Virtual space providing device, virtual space providing method, and program

Abstract

A virtual space-providing device (2000) obtains reference three-dimensional data (40), which is three-dimensional data which expresses a target location (10) at a reference time. The virtual space-providing device (2000) determines a replacement time, which is a time which differs from the reference time, and a replacement region, which is a region constituting the target location (10) where an object included therein should undergo replacement by replacing the object at the reference time with the object at the replacement time. The virtual space-providing device (2000) obtains replacement three-dimensional data (50), which is three-dimensional data that expresses the replacement region at the replacement time. The virtual space-providing device (2000) generates virtual space data (30), which is three-dimensional data which expresses the virtual space (20), by combining the reference three-dimensional data (40) and the replacement three-dimensional data (50) with one another.

Description

VIRTUAL SPACE PROVIDING DEVICE, VIRTUAL SPACE PROVIDING METHOD, AND NON-TRANSITORY COMPUTER-READABLE MEDIUM

[0001] The present disclosure relates to a virtual space providing device, a virtual space providing method, and a non-transitory computer-readable medium.

[0002] Systems have been developed that use three-dimensional data of real-world buildings to provide users with images of three-dimensional space. For example, Patent Literature 1 discloses a system that provides a user who uses a periscope to view a specific location in real space with an augmented reality image in which a building that existed in the past is synthesized with an image of the real space.

[0003] JP 2018-128815 A

[0004] In Patent Document 1, historical buildings and the like to be synthesized with specific locations in real space are determined in advance. The present disclosure has been made in consideration of this problem, and one of its purposes is to provide a new technology for providing a virtual space.

[0005] The virtual space providing device of the present disclosure includes a first acquisition means for acquiring reference three-dimensional data, which is three-dimensional data representing a target location at a reference time point; a determination means for determining a replacement time point, which is a time point different from the reference time point, and a replacement area, which is an area of ​​the target location where objects contained therein should be replaced from objects at the reference time point to objects at the replacement time point; a second acquisition means for acquiring replacement three-dimensional data, which is three-dimensional data representing the replacement area at the replacement time point; and a generation means for combining the reference three-dimensional data and the replacement three-dimensional data to generate virtual space data, which is three-dimensional data representing a virtual space.

[0006] A virtual space providing method according to the present disclosure is executed by a computer and includes a first acquisition step of acquiring reference three-dimensional data, which is three-dimensional data representing a target location at a reference time point, a determination step of determining a replacement time point that is a time point different from the reference time point and a replacement area, which is an area of ​​the target location where objects included in the target location should be replaced from the objects at the reference time point with objects at the replacement time point, a second acquisition step of acquiring replacement three-dimensional data, which is three-dimensional data representing the replacement area at the replacement time point, and a generation step of combining the reference three-dimensional data and the replacement three-dimensional data to generate virtual space data, which is three-dimensional data representing a virtual space.

[0007] The non-transitory computer-readable medium of the present disclosure stores a program that causes a computer to execute the disclosed virtual space providing method.

[0008] According to the present disclosure, a new technology for providing a virtual space is provided.

[0009] FIG. 1 is a diagram illustrating an example of an outline of the operation of a virtual space providing device of embodiment 1. FIG. 2 is a block diagram illustrating an example of the functional configuration of a virtual space providing device of embodiment 1. FIG. 3 is a block diagram illustrating an example of the hardware configuration of a computer that realizes the virtual space providing device of embodiment 1. FIG. 4 is a flowchart illustrating an example of a flow of processing executed by the virtual space providing device of embodiment 1. FIG. 5 is a diagram illustrating an example of a usage environment of a virtual space providing device. FIG. 6 is a diagram illustrating an example of the configuration of schedule information. FIG. 7 is a diagram illustrating an example of how a replacement time and a replacement area are specified by a user. FIG. 8 is a first diagram illustrating an example of a case where a replacement area and a replacement time are determined by a statement of an avatar. FIG. 9 is a second diagram illustrating an example of a case where a replacement area and a replacement time are determined by a statement of an avatar. FIG. 10 is a diagram illustrating an example of a case where a replacement time and a replacement area are determined based on the attributes of an avatar.

[0010] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In each drawing, the same or corresponding elements are designated by the same reference numerals, and duplicate explanations will be omitted as necessary for clarity. Furthermore, unless otherwise specified, predetermined values ​​such as predetermined values ​​and threshold values ​​are stored in advance in a storage device accessible from a device that uses the values. Furthermore, unless otherwise specified, the storage unit is composed of one or any number of storage devices.

[0011] [First Embodiment] <Overview> Fig. 1 is a diagram illustrating an example of an outline of the operation of a virtual space providing device 2000 according to a first embodiment. Fig. 1 is a diagram for facilitating understanding of the outline of the virtual space providing device 2000, and the operation of the virtual space providing device 2000 is not limited to that shown in Fig. 1.

[0012] The virtual space providing device 2000 generates virtual space data 30, which is three-dimensional data representing a virtual three-dimensional space. The virtual space represented by the virtual space data 30 is called a virtual space 20. Here, the virtual three-dimensional space can also be expressed as a metaverse or the like. The three-dimensional data includes, for example, polygon data representing the shape of each object and texture information representing the texture of each object.

[0013] The virtual space 20 represents a space in which the scenery of an actual place (hereinafter referred to as the target place 10) at a certain reference time point is blended with the scenery of the target place 10 at a replacement time point, which is a time point different from the reference time point. For example, in the virtual space 20, some objects at the reference time point are replaced with objects at the replacement time point. The replaced objects may be, for example, buildings, signs, vehicles, or roads.

[0014] Assume that the replacement time is a time earlier than the reference time. In this case, some buildings, etc. in the virtual space 20 are replaced with buildings, etc. that were in that location earlier than the reference time. Thus, in the virtual space 20, the scenery at the reference time (e.g., the current scenery) and the past scenery are blended together. This makes it possible to reproduce, for example, the target location 10 in the past in the virtual space 20.

[0015] On the other hand, suppose the replacement time is a time in the future from the reference time. In this case, some buildings, etc. in the virtual space 20 are replaced with buildings, etc. that are expected to be at that location in the future from the reference time. Thus, in the virtual space 20, the scenery at the reference time (e.g., the current scenery) can be blended with the future scenery. This makes it possible to simulate, for example, the target location 10 in the future in the virtual space 20. As a more specific example, if construction of a new facility is planned at the target location 10, it is possible to see in the virtual space 20 what the target location 10 will look like after the new building is completed.

[0016] To generate the virtual space data 30, the virtual space provision device 2000 performs, for example, the following processing. First, the virtual space provision device 2000 acquires reference three-dimensional data 40, which is three-dimensional data representing the target location 10 at a reference time. The virtual space provision device 2000 also determines a replacement time and a replacement area. The replacement area is an area of ​​the target location 10 where objects contained in that area should be replaced from those at the reference time with those at the replacement time. The virtual space provision device 2000 acquires replacement three-dimensional data 50, which is three-dimensional data representing the replacement area at the replacement time. The virtual space provision device 2000 then generates virtual space data 30 representing the virtual space 20 by combining the reference three-dimensional data 40 and the replacement three-dimensional data 50.

[0017] <Example of Action and Effect> The virtual space providing device 2000 of this embodiment determines a replacement time and a replacement area. The replacement time is a time different from the reference time. The replacement area is an area in the reference three-dimensional data 40 representing the target location 10 at the reference time that should be replaced from three-dimensional data corresponding to the reference time with three-dimensional data corresponding to the replacement time. The virtual space providing device 2000 then combines the replacement three-dimensional data 50 representing the replacement area at the replacement time with the reference three-dimensional data 40 to generate virtual space data 30 representing the virtual space 20 to be provided to the user.

[0018] The virtual space providing device 2000 provides a new technology for providing virtual space, in which "when replacing a part of a virtual space with three-dimensional data from a past or future point in time, which area should be replaced with three-dimensional data from a certain point in time is dynamically determined."

[0019] The virtual space providing device 2000 of this embodiment will be described in more detail below.

[0020] <Example of Functional Configuration> Figure 3 is a block diagram illustrating an example of the functional configuration of the virtual space providing device 2000 of embodiment 1. The virtual space providing device 2000 has a first acquisition unit 2020, a determination unit 2040, a second acquisition unit 2060, and a generation unit 2080. The first acquisition unit 2020 acquires reference three-dimensional data 40. The determination unit 2040 determines the replacement time and replacement area. The second acquisition unit 2060 acquires replacement three-dimensional data 50 that represents the replacement area at the replacement time. The generation unit 2080 generates virtual space data 30 using the reference three-dimensional data 40 and the replacement three-dimensional data 50.

[0021] <Example of Hardware Configuration> Each functional component of the virtual space providing device 2000 may be realized by hardware that realizes the functional component (e.g., a hardwired electronic circuit, etc.), or may be realized by a combination of hardware and software (e.g., a combination of an electronic circuit and a program that controls it, etc.). Below, a case where each functional component of the virtual space providing device 2000 is realized by a combination of hardware and software will be further described.

[0022] 3 is a block diagram illustrating an example of the hardware configuration of a computer 1000 that realizes the virtual space providing device 2000. The computer 1000 is any computer. For example, the computer 1000 is a stationary computer such as a PC (Personal Computer) or a server machine. Alternatively, the computer 1000 may be a portable computer such as a smartphone or a tablet terminal. The computer 1000 may be a dedicated computer designed to realize the virtual space providing device 2000, or may be a general-purpose computer.

[0023] For example, by installing a predetermined application on the computer 1000, each function of the virtual space providing device 2000 is realized on the computer 1000. The application is configured as a program for realizing each functional component of the virtual space providing device 2000. Note that any method for acquiring the program is possible. For example, the program can be acquired from a storage medium (such as a DVD disc or USB memory) on which the program is stored. Alternatively, the program can be acquired by downloading the program from a server device that manages the storage device on which the program is stored.

[0024] The computer 1000 has a bus 1020, a processor 1040, a memory 1060, a storage device 1080, an input / output interface 1100, and a network interface 1120. The bus 1020 is a data transmission path for the processor 1040, the memory 1060, the storage device 1080, the input / output interface 1100, and the network interface 1120 to transmit and receive data to and from each other. However, the method of connecting the processor 1040 and the like to each other is not limited to bus connection.

[0025] The processor 1040 is a processor such as a central processing unit (CPU), a graphics processing unit (GPU), or a field-programmable gate array (FPGA). The memory 1060 is a main storage device realized using a random access memory (RAM) or the like. The storage device 1080 is an auxiliary storage device realized using a hard disk, a solid state drive (SSD), a memory card, a read only memory (ROM), or the like.

[0026] The input / output interface 1100 is an interface for connecting the computer 1000 to an input / output device. For example, the input / output interface 1100 is connected to an input device such as a keyboard and an output device such as a display device.

[0027] The network interface 1120 is an interface for connecting the computer 1000 to a network. This network may be a LAN (Local Area Network) or a WAN (Wide Area Network).

[0028] The storage device 1080 stores programs (programs that realize the above-mentioned applications) that realize the functional components of the virtual space providing device 2000. The processor 1040 reads these programs into the memory 1060 and executes them to realize the functional components of the virtual space providing device 2000.

[0029] The virtual space providing device 2000 may be realized by one computer 1000, or may be realized by multiple computers 1000. In the latter case, the configurations of the computers 1000 do not need to be the same, and can be different from each other.

[0030] <Processing Flow> Figure 4 is a flowchart illustrating the processing flow executed by the virtual space providing device 2000 of embodiment 1. The first acquisition unit 2020 acquires reference three-dimensional data 40 (S102). The determination unit 2040 determines the replacement time and replacement area (S104). The second acquisition unit 2060 acquires replacement three-dimensional data 50 (S106). The generation unit 2080 generates virtual space data 30 using the reference three-dimensional data 40 and the replacement three-dimensional data 50 (S108).

[0031] <Example of use of virtual space 20> To make it easier to understand the virtual space providing device 2000, an example of use of the virtual space 20 provided by the virtual space providing device 2000 will be described. The example of use described here is just one example, and examples of use of the virtual space providing device 2000 are not limited to those described below.

[0032] 5 is a diagram illustrating an example of a usage environment for the virtual space providing device 2000. A user 60 uses a user terminal 70 to access the virtual space providing device 2000. In FIG. 5, a user 60-1 uses a user terminal 70-1, and a user 60-2 uses a user terminal 70-2.

[0033] The virtual space 20 is a virtual space in which each user 60 can participate using an avatar. In Fig. 5, an avatar 80-1 and an avatar 80-2 are the avatars of the user 60-1 and the user 60-2, respectively.

[0034] Users 60 can move freely within the virtual space 20 by operating their own avatars 80. Users 60 can also converse with other users 60 via their avatars 80. Note that various existing technologies can be used as specific technologies for realizing the movement of avatars in the virtual space and for realizing conversations between avatars in the virtual space.

[0035] The virtual space providing device 2000 stores avatar data 90, which is information about each avatar 80, in addition to virtual space data 30 representing the virtual space 20. The avatar data 90 indicates, for each avatar 80, the identification information of the user 60 operating the avatar 80, definition information about the appearance of the avatar 80, attribute information about the avatar 80, and the position and posture of the avatar 80 in the virtual space 20. The attribute information about the avatar 80 includes, for example, age, gender, personality, hobbies, language, hometown, or current address. Note that if multiple languages ​​are available, multiple languages ​​may be set as the language of the avatar 80.

[0036] The attributes of the avatar 80 may or may not match the real-life attributes of the user 60 who operates the avatar 80. In the former case, if the user 60 is 30 years old, for example, the age of the avatar 80 will automatically be 30 years old. On the other hand, in the latter case, even if the user 60 is 30 years old, the age of the avatar 80 can be set to any age, such as 20 years old or 40 years old.

[0037] The virtual space providing device 2000 provides a display screen to the user terminal 70 of each user 60. The display screen provided to the user 60 includes, for example, an image obtained by capturing an image of a predetermined range including the avatar 80 of the user 60 from a camera virtually placed in the virtual space 20 (hereinafter referred to as a virtual camera). Here, various existing technologies can be used as specific techniques for providing an image of the virtual space including the avatar placed in the virtual space to the user of the avatar.

[0038] For example, the virtual space providing device 2000 first provides a virtual space 20 that represents a target location 10 at a reference time. That is, the virtual space providing device 2000 acquires reference three-dimensional data 40 and generates virtual space data 30 that is configured from the reference three-dimensional data 40, thereby generating the virtual space 20. Therefore, the virtual space 20 that is first provided represents the target location 10 as a whole at the reference time.

[0039] For example, assume that the target location 10 is an area called Area A. Also assume that the latest three-dimensional data prepared for Area A is three-dimensional data representing Area A at 1:00 PM on February 27, 2023.

[0040] In this case, for example, the virtual space providing device 2000 acquires the three-dimensional data generated for Area A at 1:00 PM on February 27, 2023 as reference three-dimensional data 40. Here, the reference time is 1:00 PM on February 27, 2023. The virtual space providing device 2000 then generates virtual space data 30 composed of this reference three-dimensional data 40.

[0041] User 60 uses user terminal 70 to access virtual space providing device 2000 and adds his / her own avatar to virtual space 20 by logging in to the service provided by virtual space providing device 2000. In this way, user 60 can use his / her own avatar to experience a virtual space simulating District A at 1:00 PM on February 27, 2023.

[0042] Thereafter, the virtual space providing device 2000 dynamically replaces a part of the virtual space 20 with the replaced three-dimensional data 50. As a result, the virtual space 20 changes from a virtual space representing the target location 10 at the reference time to a space in which the target location 10 at the reference time and the target location 10 at a time point in the past or future relative to the reference time are merged.

[0043] For example, the user 60 specifies a portion of the virtual space 20 (such as a specific building) on ​​the display screen displayed on the display device of the user terminal 70. The user 60 then specifies the time at which the specified portion should be replaced (i.e., the replacement time). As a result, in the virtual space 20, the portion specified by the user 60 is replaced from the portion at the reference time with the portion at the specified replacement time. In this way, the user 60 can replace the scenery of a portion of the target location 10 in the virtual space 20 with the scenery at the past or future time specified by the user.

[0044] For example, suppose a building that the user 60 has visited in the past no longer exists at the present time. In such a case, the user 60 specifies the location where the building once existed as the replacement area, and also specifies the time when the user previously visited the building as the replacement time. In this way, the user 60 can look at the places the user has visited in the past in the virtual space 20 and reminisce about the past.

[0045] In addition, for example, the virtual space providing device 2000 dynamically replaces a portion of the virtual space 20 in response to a statement made by an avatar. For example, suppose an avatar says, "There used to be a movie theater around here." In this case, the virtual space providing device 2000 acquires the three-dimensional data of this movie theater as replacement three-dimensional data 50 and replaces a portion of the virtual space data 30 with the replacement three-dimensional data 50. As a result, the movie theater that existed in the past, as expressed by the avatar's statement, is reproduced in the virtual space 20.

[0046] By replacing a part of the virtual space 20 with a building or the like from the past or future in response to the avatar's utterances in this way, it is possible to appropriately replace a part of the virtual space 20 with something from the past or future in accordance with the avatar's utterances. In this way, for example, when avatars are gathered together and having a conversation, the scenery in the virtual space 20 can be appropriately changed to something from the past or future in accordance with the topic of the conversation. Thus, the user 60 can easily and naturally experience the target location 10 from the past or future.

[0047] The method for dynamically replacing a part of the virtual space 20 is not limited to the above-mentioned methods. Each of the above-mentioned methods will be described in more detail later.

[0048] Each operation of the virtual space providing device 2000 will be explained in more detail below.

[0049] <Acquisition of Reference Three-Dimensional Data 40: S102> The first acquisition unit 2020 acquires the reference three-dimensional data 40 (S102). As described above, the reference three-dimensional data 40 is three-dimensional data that represents the target location 10 at a reference time. Here, the reference time and the target location 10 may be predetermined or may be specifiable. In the former case, the first acquisition unit 2020 acquires the predetermined reference three-dimensional data 40.

[0050] If the target location 10 can be specified, the first acquisition unit 2020 accepts the specification of the target location 10. For example, the virtual space providing device 2000 provides the user of the virtual space providing device 2000 with an input interface (e.g., an input screen) for specifying the target location 10. The virtual space providing device 2000 determines the location specified using the input interface as the target location 10.

[0051] For example, the input interface is an input screen on which a portion of a map can be specified. The user of the virtual space providing device 2000 inputs information on the map displayed on the input screen to specify the range that the user wants to treat as the target location 10 (in other words, the range that the user wants to represent in the virtual space 20). The first acquisition unit 2020 determines, as the target location 10, a location that corresponds to the area specified by the user.

[0052] The method of specifying the target location 10 is not limited to using a map. For example, the target location 10 may be specified by an address. For example, by specifying the address "Shibuya Ward, Tokyo," the entire Shibuya Ward, Tokyo can be specified as the target location 10.

[0053] The target location 10 may be selected from predetermined options. In this case, for example, the first acquisition unit 2020 provides the user with an input screen showing options for the target location 10. The user selects a location that they want to treat as the target location 10 from the options shown on the input screen.

[0054] When a target location 10 can be specified, three-dimensional data at a reference time point for all locations that can be specified as the target location 10 is stored in advance in a storage unit accessible from the virtual space providing device 2000. For example, assume that any location in Japan can be specified as the target location 10. In this case, three-dimensional data at the reference time point is prepared for the whole of Japan. The first acquisition unit 2020 acquires, from the three-dimensional data of Japan at the reference time point, three-dimensional data for the range specified by the user as the target location 10 as reference three-dimensional data 40.

[0055] If the reference time point can be specified, the first acquisition unit 2020 provides the user of the virtual space providing device 2000 with an input interface (e.g., an input screen) for specifying the reference time point. The virtual space providing device 2000 determines the time point specified using the input interface as the reference time point. The input interface for specifying the reference time point is, for example, a screen including an interface that allows a time point to be specified, such as a text box, a drum roll interface, or a calendar.

[0056] When the reference time point can be specified, three-dimensional data of the target location 10 for each of a plurality of time points is stored in advance in a storage unit accessible from the virtual space providing device 2000. For example, the determination unit 2040 acquires, as the reference three-dimensional data 40, the three-dimensional data corresponding to the time point closest to the reference time point from among the plurality of three-dimensional data prepared for the target location 10.

[0057] 5, the virtual space 20 can be shared by multiple users. For example, the first acquisition unit 2020 receives a target location 10 and a reference time point from each of the multiple users. In this case, the virtual space providing device 2000 generates a group for each user who has the same pair of target location 10 and reference time point. The virtual space providing device 2000 then generates a virtual space 20 for each group.

[0058] Alternatively, for example, a group of users may be generated in advance before the target location 10 and the reference time point are specified. In this case, for example, the virtual space providing device 2000 determines the target location 10 and the reference time point for each group. For example, the target location 10 and the reference time point for each group are specified by the representative user of the group.

[0059] Alternatively, for example, the pair of the target location 10 and the reference time point may be specified by the administrator of the virtual space providing device 2000.

[0060] <Determining Replacement Time and Replacement Area: S104> The determination unit 2040 determines the replacement time and replacement area (S104). There are various methods for determining the replacement time and replacement area. Various methods for determining the replacement time and replacement area will be described below.

[0061] <<Designation by User>> The determination unit 2040 accepts designation of a replacement area and a replacement time point from the user of the virtual space providing device 2000. In this case, the determination unit 2040 provides the user terminal with a user interface that enables the designation of a replacement area and a replacement time point. The determination unit 2040 determines the area and time point designated using the user interface as the replacement area and the replacement time point, respectively. Below, a description is given of how to designate the replacement area and the replacement time point, respectively.

[0062] For example, the replacement area can be specified using a map of the target location 10. In this case, the determination unit 2040 provides the user terminal with an input screen including the map of the target location 10. The user specifies a partial area of ​​the map on the input screen as the replacement area.

[0063] Alternatively, for example, the user may designate a portion of the virtual space 20 on a display screen (a screen on which the user can operate an avatar) on which the virtual space 20 is displayed as the replacement area. In this case, for example, the user designates a specific object (e.g., a building or a sign) displayed on the display screen as the replacement area. Alternatively, for example, the user may designate a replacement area by designating an area of ​​any shape, such as a rectangle, on the display screen.

[0064] Alternatively, the user may specify the name or type of an object to be treated as a replacement area. Specifically, the user can specify the name or type of facility (e.g., restaurant, movie theater, or station) to specify the replacement area.

[0065] When the names and types of facilities are used in this way, for example, the three-dimensional data indicates the attributes of each object in association with the shape data of the object, etc. The attributes of the object indicate the name, type, etc. of the object.

[0066] Alternatively, for example, the user may specify the replacement area by address.

[0067] The replacement time point can be specified, for example, by using any user interface that can specify a time point. Examples of user interfaces that can be used to specify a time point are as described above.

[0068] The input interface for the replacement time point is added to the screen used to specify the replacement area, for example, in response to the specification of the replacement area. In this case, the determination unit 2040 accepts the specification of the replacement area and the replacement time point in this order.

[0069] The timing at which the input interface for the replacement time is provided to the user is arbitrary and is not limited to after the replacement area is specified. For example, the replacement time may be specified before the replacement area is specified.

[0070] When the user specifies a plurality of replacement areas, one replacement time point common to all replacement areas may be specified, or a replacement time point may be specified for each replacement area.

[0071] 7 is a diagram illustrating how a replacement time and replacement area are specified by a user. In the example of FIG. 7, the user can specify a replacement area by specifying a building or the like on the display screen 200. Specifically, a building 110 is specified. In this example, a dot pattern is superimposed on the object specified by the user so that the object specified by the user can be easily identified.

[0072] Furthermore, the user specifies the replacement time point using the drum roll type input interface 120. Specifically, "April 13, 1993" is specified as the replacement time point.

[0073] The replacement is performed in accordance with the specified replacement area and replacement time. Specifically, the specified building 110 is replaced with the building 130 that stood in that location on April 13, 1993.

[0074] <<Designation by Schedule>> The execution of replacement may be scheduled in advance so that a part of the virtual space 20 is automatically replaced at a specific date and time. In this case, schedule information indicating the replacement schedule is stored in a storage unit accessible from the virtual space providing device 2000.

[0075] 6 is a diagram illustrating an example of the configuration of schedule information. In FIG. 6, schedule information 100 is composed of three attributes: a replacement area 102, a replacement time point 104, and an execution time point 106. The execution time point 106 indicates the time point at which the replacement is executed.

[0076] For example, the record on the first line in Figure 6 indicates that the replacement "replace area X from the reference time with that of April 1, 1973, 10:00" will be performed at 10:00 on April 1, 2023. Therefore, when the current time becomes 10:00 on April 1, 2023, the determination unit 2040 determines "area X" and "10:00 on April 1, 1973" as the replacement area and replacement time, respectively. Then, from the three-dimensional data representing the target location 10 at 10:00 on April 1, 1973, the three-dimensional data of area X is acquired as replacement three-dimensional data 50, and replacement is performed.

[0077] This method of performing replacement at a specific date and time is particularly useful when, for example, it is desired to replace a portion of the virtual space 20 in relation to a past or future event. For example, a building or the like related to a past event can be automatically replaced with a building or the like that existed at the time of the event on the same date and time as the event.

[0078] For example, suppose an event celebrating the 50th anniversary of department store D, which opened at 10:00 on April 1, 1973, is to be held in virtual space 20. In this case, for example, replacement of the pair "(replacement area, replacement time) = (area X around department store D, 10:00 on April 1, 1973)" is scheduled for 10:00 on April 1, 2023. By doing so, it is possible to automatically perform an effect in virtual space 20 in which "the area around department store D is replaced with the appearance at the time of department store D's opening, exactly 50 years after the department store D opened."

[0079] <<Determination Based on Avatar's Statement>> The determination unit 2040 may determine the replacement time point and replacement area based on the content of the avatar's statement. In this case, for example, in response to a statement made by each avatar, the determination unit 2040 detects a keyword related to a specific object in the target location 10 from the statement. The keyword may be, for example, the name of a facility or the type of facility.

[0080] If a keyword is detected from the avatar's utterance, the determining unit 2040 determines the area corresponding to the keyword as the area to be replaced. The area corresponding to the keyword may be, for example, an area representing a facility represented by the keyword.

[0081] Here, the detection of the area corresponding to the keyword may be performed for the entire virtual space 20, or may be performed for a part of the virtual space 20. In the latter case, for example, the determination unit 2040 detects the area corresponding to the keyword for an area of ​​a predetermined size centered on the avatar that made the comment. Alternatively, for example, the determination unit 2040 detects the area corresponding to the keyword for an area of ​​a predetermined size in the direction in which the avatar's line of sight or face is facing.

[0082] The replacement time may be determined based on a facility identified from the avatar's utterance, or based on a statement by the avatar regarding time. In the former case, for example, the determination unit 2040 identifies a time when a facility with that name existed based on the name of the facility detected from the avatar's utterance. The determination unit 2040 then treats the identified time as the replacement time.

[0083] Furthermore, even if a facility exists at the reference point in time, the characteristics of that facility may be different from those of the past or future. For example, suppose that the color of a certain facility B is brown at the reference point in time, but was gray five years ago. In such a case, suppose that the avatar says, "The color of facility B used to be gray."

[0084] In this case, the determining unit 2040 detects the name of the facility and the characteristics of the facility from the avatar's utterance, and then determines the time when the facility identified from the avatar's utterance had the characteristics identified from the avatar's utterance, and determines that time as the replacement time.

[0085] When the replacement time point is determined based on a statement made by an avatar regarding time, the determination unit 2040 detects a keyword related to time from the statement made by the avatar, and then determines the replacement time point using the keyword.

[0086] For example, suppose an avatar says, "Facility B was located here five years ago." In this case, the determining unit 2040 detects "five years ago" as a keyword related to time. Then, the determining unit 2040 determines a point in time five years before the current time as the replacement point in time.

[0087] The determination unit 2040 may perform replacement based on an avatar's utterance when avatars are conversing with each other, rather than when an avatar makes a statement alone. In this case, for example, the determination unit 2040 first detects an avatar's utterance regarding a specific object. Then, the determination unit 2040 detects a utterance related to the utterance from another avatar's utterance. Then, when a utterance related to the utterance is detected from another avatar's utterance, the determination unit 2040 determines a replacement area and a replacement time point based on the utterance from the other avatar.

[0088] Here, whether two statements are related can be determined by, for example, determining whether the degree of relatedness between these two statements is equal to or greater than a threshold. The degree of relatedness between two statements can be calculated using, for example, a machine learning model (hereinafter referred to as a relatedness calculation model).

[0089] The relevance calculation model is configured to output the relevance between two utterances in response to their input. The relevance calculation model is trained in advance using a plurality of training data. Each training data is composed of, for example, a pair of input data indicating a combination of two utterances and the relevance of those utterances (i.e., ground-truth output data).

[0090] 8 is a first diagram illustrating a case in which a replacement area and a replacement time point are determined based on an avatar's statement. In this example, the avatar says, "ABC Building used to be a coffee shop." Based on this statement, the determination unit 2040 replaces the ABC Building (building 140) with a coffee shop (building 150) that existed in the location where the ABC Building currently exists, prior to the reference time point.

[0091] 9 is a second diagram illustrating a case in which the replacement area and replacement time are determined by an avatar's utterance. In this example, the avatar utters, "That building used to be a coffee shop." Unlike the example in FIG. 8, the avatar's utterance in the example in FIG. 9 does not include the specific name of the building to be replaced.

[0092] On the other hand, the avatar's utterance includes the word "building," which indicates a type of building, and the demonstrative "that." Because the demonstrative "that" is used, it can be predicted that the replacement area is located in the avatar's line of sight. Therefore, the determination unit 2040 identifies, as the replacement area, an area representing a building from among the areas located in the avatar's line of sight. As a result, the building 160 is identified as the replacement area. Then, the building 160 is replaced at the location where the building 160 currently exists with a coffee shop (building 170) that existed before the reference time. Note that the direction of the face may be used instead of the line of sight.

[0093] <<Determination Based on Avatar Attributes>> The determination unit 2040 may determine the replacement area and replacement time based on the attributes of the avatar. The avatar attribute used to determine the replacement area and replacement time is, for example, the age of the avatar.

[0094] For example, when a predetermined number or more of avatars of the same age group are gathered in an area of ​​a predetermined size in the virtual space 20, the determination unit 2040 determines the replacement area and replacement time point based on the age groups of these avatars (hereinafter referred to as the avatar group). The age group is, for example, each numerical range obtained by dividing the numerical range of age into multiple ranges. An example of such a numerical range is the 30s (i.e., 30 years old or older and younger than 40 years old).

[0095] For example, the determination unit 2040 determines the replacement time point based on the age group of the avatar group and a reference age group. The reference age group indicates the age range at which the user wants to see the scenery.

[0096] For example, suppose you want to show a user a scene from when they were in their teens, to evoke memories of their youth. Also, suppose the age group of the avatar group is in their 40s. In this case, by providing the user with a scene from approximately 30 years ago (=40-10), it is possible to provide the user with a scene from when they were in their teens. Therefore, the determination unit 2040 determines a time 30 years ago from the current time as the replacement time, based on the age group of the avatar group, which is in their 40s, and the reference age group, which is in their teens.

[0097] The replacement region is, for example, a region around the avatar group. For example, the determination unit 2040 determines an area of ​​a predetermined size centered on the avatar group as the reference region. Alternatively, for example, the determination unit 2040 may determine an area of ​​a predetermined size in the gaze direction or face direction of the avatar group as the reference region. Note that the gaze direction of the avatar group is calculated as, for example, a statistical value (for example, an average) of the gaze directions of each of the multiple avatars included in the avatar group. The same applies to the face direction.

[0098] 10 is a diagram illustrating a case in which the replacement time and replacement area are determined based on the attributes of the avatars. In the example of Fig. 10, when three or more avatars of the same age group gather together, buildings and the like present in the line of sight of the avatars are replaced with buildings and the like that existed when the avatars were in their teens.

[0099] In this regard, in the example of Fig. 10, three avatars are gathered together in the same age group, in their 40s. Therefore, a building 180 that exists in the line of sight of these avatars is identified as the replacement area. Furthermore, the time when the avatars were in their teens, i.e., 30 years ago, is identified as the replacement time. Therefore, building 180 is replaced with building 190 that existed in the location of building 180 30 years ago.

[0100] <Acquisition of replacement three-dimensional data 50: S106> The second acquisition unit 2060 acquires replacement three-dimensional data 50 representing the replacement area at the time of replacement (S106). For example, the second acquisition unit 2060 determines the three-dimensional data closest to the time of replacement from among the three-dimensional data representing the target location 10 prepared for each of a plurality of time points. Then, the determination unit 2040 acquires, from the determined three-dimensional data, the three-dimensional data of a portion corresponding to the replacement area as the replacement three-dimensional data 50.

[0101] <Generation of Virtual Space Data 30: S108> The generation unit 2080 generates the virtual space data 30 by combining the reference three-dimensional data 40 and the replacement three-dimensional data 50 (S108). Specifically, the generation unit 2080 generates the virtual space data 30 by replacing the portion of the reference three-dimensional data 40 that corresponds to the replacement area with the replacement three-dimensional data 50.

[0102] When replacing a portion of the reference three-dimensional data 40 with the replacement three-dimensional data 50, the generating unit 2080 may perform interpolation processing or the like to smoothly connect the reference three-dimensional data 40 and the replacement three-dimensional data 50. Here, existing technology can be used as a specific technique for performing interpolation processing or the like to smoothly connect the two pieces of three-dimensional data.

[0103] Here, when replacement using the replacement three-dimensional data 50 is repeatedly performed, the virtual space data 30 may already have been generated. In this case, generating the virtual space data 30 also includes updating the already generated virtual space data 30. Specifically, the generation unit 2080 updates the virtual space data 30 by replacing a part of the virtual space data 30 (a part of the reference three-dimensional data 40 included in the virtual space data 30) with the replacement three-dimensional data 50.

[0104] <Output of Results> The virtual space providing device 2000 outputs information using the generated virtual space data 30. The information output by the virtual space providing device 2000 is, for example, video data of the virtual space 20 generated using the virtual space data 30.

[0105] For example, the virtual space providing device 2000 repeatedly generates, for each avatar, image data of the virtual space 20 within the imaging range of the virtual camera corresponding to that avatar, using the virtual space data 30. As a result, video data to be provided to the user of that avatar is generated for each avatar. The virtual space providing device 2000 transmits the video data generated for each avatar to the user terminal of the user corresponding to the avatar. In this way, an image showing the state of the virtual space 20 is provided to the user, allowing the user to use the avatar to use the virtual space 20.

[0106] The process of generating video data using the virtual space data 30 may be performed by a user terminal. In this case, the virtual space providing device 2000 provides the virtual space data 30 to the user terminal. The user terminal generates the above-mentioned video data using the virtual space data 30 provided by the virtual space providing device 2000.

[0107] Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

[0108] In the above examples, the program includes instructions (or software code) that, when loaded into a computer, cause the computer to perform one or more functions described in the embodiments. The program may be stored in a non-transitory computer-readable medium or a tangible storage medium. By way of example and not limitation, computer-readable media or tangible storage media include random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technologies, CD-ROM, digital versatile disc (DVD), Blu-ray (registered trademark) disc or other optical disk storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device. The program may also be transmitted on a transitory computer-readable medium or communication medium. By way of example and not limitation, transitory computer-readable media or communication media include electrical, optical, acoustic, or other forms of propagated signals.

[0109] Some or all of the above embodiments may be described as, but are not limited to, the following supplementary notes. (Supplementary Note 1) A virtual space provision device comprising: a first acquisition means for acquiring reference three-dimensional data, which is three-dimensional data representing a target location at a reference time; a determination means for determining a replacement time, which is a time different from the reference time, and a replacement area, which is an area of ​​the target location where an object included in the target location should be replaced from the object at the reference time with an object at the replacement time; a second acquisition means for acquiring replacement three-dimensional data, which is three-dimensional data representing the replacement area at the replacement time; and a generation means for combining the reference three-dimensional data and the replacement three-dimensional data to generate virtual space data, which is three-dimensional data representing a virtual space. (Supplementary Note 2) The virtual space provision device according to Supplementary Note 1, wherein the determination means acquires the replacement area and the replacement time, which are associated with the execution time that coincides with the current time, from schedule information in which a replacement execution time is associated with a pair of the replacement area and the replacement time. (Supplementary Note 3) The virtual space providing device of Supplementary Note 1, wherein the determination means determines the replacement area or the replacement time point based on utterances or attributes of an avatar used in the virtual space. (Supplementary Note 4) The virtual space providing device of Supplementary Note 3, wherein the determination means detects an area to be treated as the replacement area from an area within a predetermined range that includes the avatar, or an area within a predetermined range in the direction in which the avatar's line of sight or face is facing. (Supplementary Note 5) The virtual space providing device of Supplementary Note 3 or 4, wherein the determination means detects a first keyword that indicates the name or type of an object from the avatar's utterances, and determines, as the replacement area, an area in which an object of the name or type identified by the first keyword existed in the target location at the replacement time. (Supplementary Note 6) The virtual space providing device of Supplementary Note 5, wherein the determination means determines, as the replacement time point, a time point at which an object of the name or type identified by the first keyword existed in the target location.(Supplementary Note 7) The virtual space providing device according to Supplementary Note 5, wherein the determining means detects a second keyword representing a time point from the utterance of the avatar, and determines the time point specified by the second keyword as the replacement time point. (Supplementary Note 8) The virtual space providing device according to Supplementary Note 3 or 4, wherein the determining means determines, when a predetermined number or more avatars belonging to the same age group exist within a predetermined range in the virtual space, a time point when those avatars belonged to a standard age group as the replacement time point. (Supplementary Note 9) A virtual space provision method executed by a computer, comprising: a first acquisition step of acquiring reference three-dimensional data, which is three-dimensional data representing a target location at a reference time point; a determination step of determining a replacement time point, which is a time point different from the reference time point, and a replacement area, which is an area of ​​the target location where objects included in the target location should be replaced from the objects at the reference time point with objects at the replacement time point; a second acquisition step of acquiring replacement three-dimensional data, which is three-dimensional data representing the replacement area at the replacement time point; and a generation step of generating virtual space data, which is three-dimensional data representing a virtual space, by combining the reference three-dimensional data and the replacement three-dimensional data. (Supplementary Note 10) The virtual space provision method according to Supplementary Note 9, wherein, in the determination step, the replacement area and the replacement time point associated with the execution time that coincides with the current time are acquired from schedule information in which execution times of replacement are associated with pairs of the replacement area and the replacement time point. (Supplementary Note 11) The virtual space provision method according to Supplementary Note 9, wherein, in the determination step, the replacement area or the replacement time point is determined based on a statement or an attribute of an avatar used in the virtual space. (Appendix 12) A virtual space providing method according to Appendix 11, wherein in the determination step, an area to be treated as the replacement area is detected from an area within a predetermined range that includes the avatar, or an area within a predetermined range in the direction in which the avatar's line of sight or face is facing.(Supplementary Note 13) The virtual space provision method according to Supplementary Note 11 or 12, wherein in the determining step, a first keyword representing the name or type of an object is detected from the avatar's utterance, and an area in the target location at the replacement time where an object of the name or type specified by the first keyword existed is determined as the replacement area. (Supplementary Note 14) The virtual space provision method according to Supplementary Note 13, wherein in the determining step, a time when an object of the name or type specified by the first keyword existed in the target location is determined as the replacement time. (Supplementary Note 15) The virtual space provision method according to Supplementary Note 13, wherein in the determining step, a second keyword representing a time point is detected from the avatar's utterance, and the time point specified by the second keyword is determined as the replacement time. (Supplementary Note 16) The virtual space providing method according to Supplementary Note 11 or 12, wherein in the determining step, if a predetermined number or more avatars belonging to the same age group are present within a predetermined range in the virtual space, a time point at which those avatars belonged to a reference age group is determined to be the replacement time point. (Supplementary Note 17) A non-transitory computer-readable medium having stored thereon a program that causes a computer to execute the following steps: a first obtaining step of obtaining reference three-dimensional data, which is three-dimensional data representing a target location at a reference time point; a determining step of determining a replacement time point, which is a time point different from the reference time point, and a replacement area, which is an area of ​​the target location where objects included therein should be replaced from objects at the reference time point with objects at the replacement time point; a second obtaining step of obtaining replacement three-dimensional data, which is three-dimensional data representing the replacement area at the replacement time point; and a generating step of combining the reference three-dimensional data and the replacement three-dimensional data to generate virtual space data, which is three-dimensional data representing a virtual space. (Appendix 18) The medium described in Appendix 17, wherein in the determination step, the replacement area and the replacement time that are associated with the execution time that matches the current time are obtained from schedule information in which the execution time of the replacement is associated with the pair of the replacement area and the replacement time.(Supplementary Note 19) The medium of Supplementary Note 17, wherein, in the determining step, the replacement area or the replacement time point is determined based on utterances or attributes of an avatar used in the virtual space. (Supplementary Note 20) The medium of Supplementary Note 19, wherein, in the determining step, an area to be treated as the replacement area is detected from an area within a predetermined range that includes the avatar, or an area within a predetermined range in the direction in which the avatar's line of sight or face is facing. (Supplementary Note 21) The medium of Supplementary Note 19 or 20, wherein, in the determining step, a first keyword that represents the name or type of object is detected from the avatar's utterances, and an area in which an object of the name or type identified by the first keyword existed in the target location at the replacement time is determined to be the replacement area. (Supplementary Note 22) The medium of Supplementary Note 21, wherein, in the determining step, the time point at which an object of the name or type identified by the first keyword existed in the target location is determined to be the replacement time point. (Supplementary Note 23) The medium according to Supplementary Note 21, wherein in the determining step, a second keyword representing a time point is detected from the utterance of the avatar, and the time point specified by the second keyword is determined as the replacement time point. (Supplementary Note 24) The medium according to Supplementary Note 19 or 20, wherein in the determining step, when a predetermined number or more avatars belonging to the same age group exist within a predetermined range in the virtual space, the time point when those avatars belonged to a standard age group is determined as the replacement time point.

[0110] 10 Target location 20 Virtual space 30 Virtual space data 40 Reference three-dimensional data 50 Replacement three-dimensional data 60 User 70 User terminal 80 Avatar 90 Avatar data 100 Schedule information 102 Replacement area 104 Replacement time 106 Execution time 110 Building 120 Input interface 130 Building 140 Building 150 Building 160 Building 170 Building 180 Building 190 Building 200 Display screen 1000 Computer 1020 Bus 1040 Processor 1060 Memory 1080 Storage device 1100 Input / output interface 1120 Network interface 2000 Virtual space providing device 2020 First acquisition unit 2040 Determination unit 2060 Second acquisition unit 2080 Generation unit

Claims

1. a first acquisition means for acquiring reference three-dimensional data that is three-dimensional data representing a target location at a reference time point; a determination means for determining a replacement time point that is a time point different from the reference time point and a replacement area that is an area of ​​the target location in which an object included therein should be replaced from the object at the reference time point to the object at the replacement time point; a second acquisition means for acquiring replaced three-dimensional data that is three-dimensional data representing the replaced area at the time of the replacement; a generating means for generating virtual space data, which is three-dimensional data representing a virtual space, by combining the reference three-dimensional data and the replacement three-dimensional data.

2. The virtual space providing device of claim 1, wherein the determination means obtains the replacement area and the replacement time that are associated with the execution time that matches the current time from schedule information in which the execution time of the replacement is associated with the pair of the replacement area and the replacement time.

3. The virtual space providing device according to claim 1 , wherein the determining means determines the replacement area or the replacement time point based on a statement or an attribute of an avatar used in the virtual space.

4. The virtual space providing device according to claim 3, wherein the determination means detects an area to be treated as the replacement area from an area within a predetermined range that includes the avatar, or an area within a predetermined range in the direction in which the avatar's line of sight or face is facing.

5. The determining means detecting a first keyword representing a name or a type of an object from the utterance of the avatar; The virtual space providing device according to claim 3 , wherein an area in the target location at the time of replacement where an object of a name or type specified by the first keyword existed is determined as the replacement area.

6. The virtual space providing device according to claim 5 , wherein the determining means determines, as the replacement time, a time when an object of a name or type specified by the first keyword was present at the target location.

7. The determining means detecting a second keyword representing a time point from the utterance of the avatar; The virtual space providing device according to claim 5 , wherein the time point specified by the second keyword is determined as the replacement time point.

8. The virtual space providing device according to claim 3 or 4, wherein the determination means determines the time when a predetermined number or more of avatars belonging to the same age group exist within a predetermined range in the virtual space as the replacement time.

9. a first acquisition step of acquiring reference three-dimensional data that is three-dimensional data representing a target location at a reference time point; a determination step of determining a replacement time point that is a time point different from the reference time point and a replacement area that is an area of ​​the target location where an object included therein should be replaced from the object at the reference time point to the object at the replacement time point; a second acquisition step of acquiring replaced three-dimensional data that is three-dimensional data representing the replaced area at the time of the replacement; a generating step of combining the reference three-dimensional data and the replacement three-dimensional data to generate virtual space data, which is three-dimensional data representing a virtual space, executed by a computer.

10. 10. The virtual space providing method of claim 9, wherein in the determination step, the replacement area and the replacement time that are associated with the execution time that matches the current time are obtained from schedule information in which the execution time of the replacement is associated with the pair of the replacement area and the replacement time.

11. The virtual space providing method according to claim 9 , wherein in the determining step, the replacement area or the replacement time point is determined based on a statement or an attribute of an avatar used in the virtual space.

12. The virtual space providing method according to claim 11, wherein in the determination step, an area to be treated as the replacement area is detected from an area within a predetermined range that includes the avatar, or an area within a predetermined range in the direction in which the avatar's line of sight or face is facing.

13. In the determining step, detecting a first keyword representing a name or a type of an object from the utterance of the avatar; The virtual space providing method according to claim 11 or 12, wherein an area in the target location at the time of replacement where an object of a name or type specified by the first keyword existed is determined as the replacement area.

14. The virtual space providing method according to claim 13 , wherein in the determining step, a time when an object of a name or type specified by the first keyword was present at the target location is determined as the replacement time.

15. In the determining step, detecting a second keyword representing a time point from the utterance of the avatar; The virtual space providing method according to claim 13 , wherein a time point specified by the second keyword is determined as the replacement time point.

16. A virtual space providing method as described in claim 11 or 12, wherein in the determination step, if a predetermined number or more avatars belonging to the same age group exist within a predetermined range in the virtual space, the time when those avatars belonged to a standard age group is determined to be the replacement time.

17. a first acquisition step of acquiring reference three-dimensional data that is three-dimensional data representing a target location at a reference time point; a determination step of determining a replacement time point that is a time point different from the reference time point and a replacement area that is an area of ​​the target location where an object included therein should be replaced from the object at the reference time point to the object at the replacement time point; a second acquisition step of acquiring replaced three-dimensional data that is three-dimensional data representing the replaced area at the time of the replacement; a generating step of combining the reference three-dimensional data and the replaced three-dimensional data to generate virtual space data, which is three-dimensional data representing a virtual space.

18. 18. The program according to claim 17, wherein in the determination step, the replacement area and the replacement time that are associated with the execution time that matches the current time are obtained from schedule information in which the execution time of the replacement is associated with the pair of the replacement area and the replacement time.

19. 18. The program according to claim 17, wherein in the determining step, the replacement area or the replacement time point is determined based on a statement or an attribute of an avatar used in the virtual space.

20. 20. The program according to claim 19, wherein in the determination step, an area to be treated as the replacement area is detected from an area within a predetermined range that includes the avatar, or an area within a predetermined range in the direction in which the avatar's line of sight or face is facing.

21. In the determining step, detecting a first keyword representing a name or a type of an object from the utterance of the avatar; 21. The program according to claim 19, wherein an area in the target location at the time of replacement where an object of a name or type specified by the first keyword existed is determined as the replacement area.

22. 22. The program according to claim 21, wherein in the determining step, a time when an object of a name or type specified by the first keyword was present at the target location is determined as the replacement time.

23. In the determining step, detecting a second keyword representing a time point from the utterance of the avatar; 22. The program according to claim 21, wherein a time point specified by the second keyword is determined as the replacement time point.

24. 21. The program according to claim 19, wherein in the determination step, if a predetermined number or more of avatars belonging to the same age group exist within a predetermined range in the virtual space, the time when those avatars belonged to a reference age group is determined to be the replacement time.