Processing server, processing method, and program

Abstract

A processing server, a processing method, and a program are provided. [Solution] In one embodiment of the processing system 10, the processing server includes a material data acquisition unit that acquires material data provided by a user, including captured images captured at a location where the user was located at each of a plurality of time points, and a spatio-temporal data generation unit that generates, based on the material data, past spatio-temporal data representing the user's past space-time, which is composed of spatial data of the user's location at the plurality of time points. According to one embodiment, the processing server includes a material data acquisition unit that acquires material data including images related to a specific object at each of a plurality of time points and at least one of papers, research materials, newspaper articles, and minutes related to the object, and a spatio-temporal data generation unit that generates, based on the material data, past spatio-temporal data representing the object's past space-time, which is composed of spatial data of the object at the plurality of time points.

Description

[Technical field]

[0001] The present invention relates to a processing server, a processing method, and a program. [Background technology]

[0002] Patent Document 1 describes an image generation AI that generates an AI (Artificial Intelligence) image according to input text information or the like. [Prior art document] [Patent documents] [Patent Document 1] JP 2024-043680 A Summary of the Invention [Means for solving the problem]

[0003] According to one embodiment of the present invention, there is provided a processing server. The processing server may include a material data acquisition unit that acquires material data provided by a user, the material data including captured images captured at a location where the user was located at each of a plurality of time points. The processing server may include a spatio-temporal data generation unit that generates, based on the material data, past spatio-temporal data representing the user's past space-time, the past space-time data being composed of spatial data of the location of the user at the plurality of time points.

[0004] In the processing server, the spatio-temporal data generation unit may generate spatial data representing the situation of the point where the user was located at each of the multiple points in time using the multiple captured images included in the material data, generate instruction data including an instruction to generate the spatio-temporal data using the multiple generated spatial data, and generate the past spatio-temporal data by inputting the instruction data to a generation AI.

[0005] In the processing server, the spatio-temporal data generation unit may generate spatial data representing the points where the user was located at each of the multiple time points using the multiple captured images included in the material data, generate spatial data for a time point between consecutive time points among the multiple time points using the spatial data for the consecutive time points, generate the instruction data including an instruction to generate the past spatio-temporal data using the generated multiple spatial data, and input the instruction data to the generation AI.

[0006] In any of the processing servers, the material data may further include map data including information on the locations, and the spatiotemporal data generation unit may generate the spatial data representing the locations where the user was located at each of the multiple points in time using the multiple captured images and the map data included in the material data, generate the instruction data including an instruction to generate the past spatiotemporal data using the generated multiple spatial data, and input the instruction data to the generation AI.

[0007] In any of the processing servers, the material data may further include diary data including diaries for the multiple time points, and the spatio-temporal data generation unit may use the multiple captured images included in the material data to generate spatial data representing the points where the user was located at each of the multiple time points, use the diary data to estimate the user's level of interest in multiple areas within the spatial data for each of the multiple spatial data, correct the spatial data according to the estimated level of interest, and then generate the instruction data including an instruction to generate the past spatio-temporal data using the multiple spatial data, and input the instruction data to the generation AI.

[0008] In any of the processing servers, the spatio-temporal data generation unit may estimate an interest of the user based on the material data, and generate future spatio-temporal data representing a future space-time of the user based on the estimated interest. The processing server may include a spatio-temporal data providing unit that provides the future spatio-temporal data of the user generated by the spatio-temporal data generation unit to other users approved by the user.

[0009] Any one of the processing servers may include a spatio-temporal data providing unit that provides the past spatio-temporal data of the user generated by the spatio-temporal data generating unit to other users approved by the user.

[0010] In any of the processing servers, the material data acquisition unit may acquire object material data including images at multiple points in time related to a specific object, and at least one of papers, research materials, newspaper articles, and minutes related to the object, and the spatio-temporal data generation unit may generate object past spatio-temporal data representing the past space-time of the object, which is composed of spatial data of the object at multiple points in time, based on the object material data.

[0011] According to one embodiment of the present invention, there is provided a processing server. The processing server may include a material data acquisition unit that acquires material data including images at a plurality of time points related to a specific object and at least one of papers, research materials, newspaper articles, and minutes related to the object. The processing server may include a spatio-temporal data generation unit that generates, based on the material data, past spatio-temporal data representing the past space-time of the object, which is composed of spatial data of the object at a plurality of time points.

[0012] In the processing server, the spatio-temporal data generation unit may generate the spatial data for each of the multiple points in time using the multiple images included in the material data and at least one of the papers, the research materials, the newspaper articles, and the minutes included in the material data, generate instruction data including an instruction to generate the past spatio-temporal data using the generated multiple spatial data, and generate the past spatio-temporal data by inputting the instruction data to a generation AI.

[0013] In the processing server, the spatio-temporal data generation unit may generate the spatial data for each of the multiple points in time using the multiple images included in the material data and at least one of the papers, the research materials, the newspaper articles, and the minutes included in the material data, generate spatial data for a point in time between consecutive points in time among the multiple points in time using the spatial data for the consecutive points in time, generate the instruction data including an instruction to generate the past spatio-temporal data using the generated multiple spatial data, and input the instruction data to the generation AI.

[0014] In the processing server, the spatio-temporal data generation unit may identify repeated events in the history of the subject based on at least one of the papers, the research materials, the newspaper articles, and the minutes included in the material data, and generate future spatio-temporal data representing the future space-time of the subject based on the identified events.

[0015] In the processing server, the space-time data generation unit may generate a plurality of the future space-time data, estimate the user's interests based on user-related information related to a user who will experience any of the plurality of future space-time data, and determine the future space-time data to be experienced by the user from the plurality of future space-time data based on the estimated interests.

[0016] In the processing server, the spatio-temporal data generation unit may estimate the user's interests based on user-related information related to the user experiencing the generated past spatio-temporal data, and correct the past spatio-temporal data based on the estimated interests.

[0017] According to one embodiment of the present invention, there is provided a processing method executed by a computer. The processing method may include a material data acquisition step of acquiring material data provided by a user, the material data including captured images captured at points where the user was located at each of a plurality of time points. The processing method may include a spatio-temporal data generation step of generating, based on the material data, past spatio-temporal data representing the user's past space-time, the past space-time data being composed of spatial data where the user was located at the plurality of time points.

[0018] According to one embodiment of the present invention, a processing method executed by a computer may be provided. The processing method may include a material data acquisition step of acquiring material data including images at a plurality of time points related to a specific object and at least one of papers, research materials, newspaper articles, and minutes related to the object. The processing method may include a spatio-temporal data generation step of generating, based on the material data, past spatio-temporal data representing the past space-time of the object, the past space-time data being composed of spatial data of the object at a plurality of time points.

[0019] According to one embodiment of the present invention, there is provided a program for causing a computer to execute the processing method.

[0020] The above summary of the invention does not list all of the necessary features of the present invention. Also, subcombinations of these features may also be inventions. [Brief description of the drawings]

[0021] [Figure 1] An example of a processing system 10 is shown generally. [Diagram 2]FIG. 2 is an explanatory diagram for explaining the processing contents by the processing server 100. [Diagram 3] FIG. 2 is an explanatory diagram for explaining the processing contents by the processing server 100. [Figure 4] FIG. 2 is an explanatory diagram for explaining the processing contents by the processing server 100. [Diagram 5] FIG. 2 is an explanatory diagram for explaining the processing contents by the processing server 100. [Figure 6] 2 illustrates an example of a functional configuration of the processing server 100. [Figure 7] 1 shows an example of a processing flow by the processing server 100. [Figure 8] 1 shows an example of a processing flow by the processing server 100. [Figure 9] An example of the hardware configuration of a computer 1200 functioning as the processing server 100 is shown in schematic form. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0022] The present invention will be described below through embodiments of the invention, but the following embodiments do not limit the scope of the invention according to the claims. Furthermore, not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention.

[0023] It is considered very useful to reproduce a life log of a specific user by using a 3D video, so that the user can experience it, or to allow other users to experience it with the user's approval. It is also considered very useful to predict the future from the past history of a specific user, generate a 3D video representing the predicted future, so that the user can experience it, or to allow other users to experience it with the user's approval. It is also considered very useful to reproduce the history of a specific region, corporation, etc. by using a 3D video, so that the user can experience it, or to predict the future from the past history of a specific region, corporation, etc., generate a 3D video representing the predicted future, so that the user can experience it. The processing system 10 according to the present embodiment provides such an experience. The processing system 10 provides such an experience by, for example, XR (Cross Reality). The processing server 100 may generate these 3D videos as content for an MR (Mixed Reality) device. The processing server 100 may generate these 3D videos as content for an AR (Augmented Reality) device. The processing server 100 may generate these 3D videos as content for a VR (Virtual Reality) device.

[0024] 1 illustrates an example of a processing system 10. The processing system 10 includes a processing server 100. The processing system 10 includes a generative AI 180. The processing system 10 includes a DB server 190.

[0025] The processing server 100 may have a function of generating spatiotemporal data representing past and future time-space. The spatiotemporal data may be 3D video. The processing server 100 may have a function of generating spatiotemporal data for individuals. The processing server 100 may have a function of generating spatiotemporal data for corporations.

[0026] The personal spatiotemporal data generation function will be described. The processing server 100 generates past spatiotemporal data representing the past space-time of the user 200, for example, based on material data provided by the user 200. The processing server 100 may receive the material data from a communication terminal 300 of the user 200. The communication terminal 300 may be any terminal capable of communication, such as a smartphone, a tablet terminal, or a PC (Personal Computer). The processing server 100 may receive the material data from eyewear 400 of the user 200.

[0027] The material data includes captured images captured at a location where the user was previously located. The material data may include captured images at each of a plurality of time points. The captured images may be still images. The captured images may be videos. The material data may include map data including information on the locations where the captured images were captured. The material data may include diary data including diaries at a plurality of time points where the captured images were captured. The diary data may include, for each of a plurality of dates and times, the date and time, the location, and the contents of the diary written by the user 200. The material data may include data other than these.

[0028] The past time-space data may be a 3D video representing a past experience of the user 200. The past time-space data may be content for an XR device. For example, the past time-space data is content for an MR device. For example, the past time-space data is content for an AR device. For example, the past time-space data is content for a VR device.

[0029] The processing server 100 may generate past time-space data using the generation system AI 180. The generation system AI 180 may be any type capable of receiving a prompt input including a still image, a video, and a text instruction, and outputting a 3D video. The text may be input via an input device such as a keyboard or a touch panel, may be input by voice input, or may be input by other input methods. The generation system AI 180 may be a so-called video generation AI. The processing server 100 may use an external generation system AI 180. The processing server 100 may communicate with the generation system AI 180 via a network 50. The network 50 may include the Internet. The network 50 may include a cloud. The network 50 may include a mobile communication network. The mobile communication network may comply with any of a 5G (5th Generation) communication method, an LTE (Long Term Evolution) communication method, and a 6G (6th Generation) or later communication method. The network 50 may include a LAN (Local Area Network). The processing server 100 may include the generation AI 180. The processing server 100 may include the generation AI 180 generated by another device, or may include the generation AI 180 generated by the processing server 100 itself.

[0030] The processing server 100 may input prompt information including material data and an instruction to generate past spatio-temporal data using the material data to the generation AI 180, and acquire the past spatio-temporal data from the generation AI 180. The processing server 100 may store the material data provided by the user 200 by itself. The processing server 100 may store the material data provided by the user 200 in the DB server 190. The processing server 100 may store the generated past spatio-temporal data by itself. The processing server 100 may store the generated past spatio-temporal data in the DB server 190.

[0031] The processing server 100 provides the past time-space data of the user 200 to the user 200. The processing server 100 may transmit the past time-space data of the user 200 to the communication terminal 300. The processing server 100 may transmit the past time-space data of the user 200 to the eyewear 400. The user 200 may view the past time-space data using the communication terminal 300. The user 200 may view the past time-space data using the eyewear 400.

[0032] The processing server 100 generates future spatio-temporal data representing the future space-time of the user 200, for example, based on material data provided by the user 200. The future spatio-temporal data may be a 3D video predicting the future of the user 200. The future spatio-temporal data may be a 3D video representing the situation of the user 200 when the wish of the user 200 is fulfilled, for example. The processing server 100 may input prompt information including material data and an instruction to generate future spatio-temporal data using the material data to the generation AI 180, and obtain the future spatio-temporal data from the generation AI 180. The processing server 100 may store the generated future spatio-temporal data by itself. The processing server 100 may store the generated future spatio-temporal data in the DB server 190.

[0033] The processing server 100 provides the future time-space data of the user 200 to the user 200. The processing server 100 may transmit the future time-space data of the user 200 to the communication terminal 300. The processing server 100 may transmit the future time-space data of the user 200 to the eyewear 400. The user 200 may view the future time-space data using the communication terminal 300. The user 200 may view the future time-space data using the eyewear 400.

[0034] The spatio-temporal data generation function for corporations will be described. The processing server 100 generates past spatio-temporal data representing the past space-time of a target using material data about a specific target (corporation, region, etc.) provided by, for example, a corporation, an individual belonging to a corporation, or an individual not belonging to a corporation. The processing server 100 may receive material data from the communication terminal 300 of the user 200.

[0035] The material data includes images of the object. The material data may include images at each of a plurality of points in time. The images may be still images. The images may be videos. The material data may include map data including information on the locations where the images were captured. The material data may include paper data including papers about the object. The material data may include research material data including research materials about the object. The material data may include newspaper article data including newspaper articles about the object. The material data may include minutes data including minutes of a meeting about the object. The material data may include data other than the above.

[0036] The past time-space data may be a 3D video showing the history of the target. The past time-space data may be content for an XR device. For example, the past time-space data is content for an MR device. For example, the past time-space data is content for an AR device. For example, the past time-space data is content for a VR device.

[0037] The processing server 100 may generate the past spatio-temporal data using the generation system AI 180. The processing server 100 may input material data and prompt information including an instruction to generate the past spatio-temporal data using the material data to the generation system AI 180, and acquire the past spatio-temporal data from the generation system AI 180.

[0038] The processing server 100 may store the material data provided by the user 200 by itself. The processing server 100 may store the material data provided by the user 200 in the DB server 190. The processing server 100 may store the generated past time-space data by itself. The processing server 100 may store the generated past time-space data in the DB server 190.

[0039] The processing server 100 provides the target past time-space data to the user 200. The processing server 100 may transmit the target past time-space data to the communication terminal 300. The processing server 100 may transmit the target past time-space data to the eyewear 400. The user 200 may view the past time-space data using the communication terminal 300. The user 200 may view the past time-space data using the eyewear 400.

[0040] The processing server 100 generates future spatio-temporal data representing the future space-time of the target based on, for example, material data provided by the user 200. The future spatio-temporal data may be a 3D video predicting the future of the target. The future spatio-temporal data may be a 3D video predicted from events occurring repeatedly in the history of the target. The processing server 100 may input prompt information including material data and an instruction to generate future spatio-temporal data using the material data to the generation AI 180, and obtain the future spatio-temporal data from the generation AI 180. The processing server 100 may store the generated future spatio-temporal data by itself. The processing server 100 may store the generated future spatio-temporal data in the DB server 190.

[0041] The processing server 100 provides the target future-time spatio-temporal data to the user 200. The processing server 100 may transmit the target future-time spatio-temporal data to the communication terminal 300. The processing server 100 may transmit the target future-time spatio-temporal data to the eyewear 400. The user 200 may view the future-time spatio-temporal data using the communication terminal 300. The user 200 may view the future-time spatio-temporal data using the eyewear 400.

[0042] The processing server 100 may be arranged in a so-called MEC (Multi-access Edge Computing). The processing server 100 may be applied to a system that realizes control of a Radio Access Network (RAN) and AI processing by a management infrastructure and a plurality of distributed infrastructures managed by the management infrastructure.

[0043] The RAN referred to here may be a virtualized vRAN (Virtual RAN) or a physical RAN. The AI ​​processing referred to here may include RAN control AI processing and non-RAN control AI processing. An example of the RAN control AI processing is a RAN Intelligent Controller (RIC). Note that the RAN control AI processing is not limited to the RIC. The non-RAN control AI processing may correspond to a so-called MEC application. The processing server 100 may be arranged on a distributed platform. The processing server 100 may be arranged on a management platform.

[0044] The distributed infrastructure may be a data center located in various locations. The distributed infrastructure may be configured in a plurality of devices. The distributed infrastructure may be realized on a virtualization infrastructure including a plurality of devices. The distributed infrastructure may be realized by a single device. In other words, the distributed infrastructure may be a distributed device.

[0045] The management infrastructure may be a data center that manages multiple distributed infrastructures. The management infrastructure may be configured in multiple devices. The management infrastructure may be realized on a virtualization infrastructure that includes multiple devices. The management infrastructure may be realized by one device. In other words, the management infrastructure may be a management device.

[0046] The management infrastructure may be called the Core Brain, and the distributed infrastructure may be called the Regional Brain. Multiple hierarchical levels of distributed infrastructure may be placed under the management infrastructure. For example, when two hierarchical levels of distributed infrastructure are placed under the management infrastructure, the management infrastructure may be called the Core Brain, the distributed infrastructure at the lower level may be called the Regional Brain, and the distributed infrastructure at the lower level may be called the Sub Regional Brain.

[0047] The distributed infrastructure may have one or more central processing units (CPUs). The distributed infrastructure may have one or more GPUs. The distributed infrastructure may have a plurality of super chips, each of which has a CPU and a GPU connected via an interconnect. The interconnect may have memory consistency and may be capable of achieving high bandwidth and low latency. In this way, the distributed infrastructure may have CPU resources and GPU resources as computational resources.

[0048] FIG. 2 is an explanatory diagram for explaining the spatiotemporal data generation function for individuals by the processing server 100. In the example shown in FIG. 2, the processing server 100 receives material data 210 from the communication terminal 300 of the user 200. The material data 210 includes captured images captured at a location where the user 200 was located in the past. The material data 210 may include captured images at each of a plurality of time points. The material data 210 may include map data including information on the location where the captured images were captured. The material data 210 may include diary data including diaries at a plurality of time points where the captured images were captured. The material data 210 may include other data.

[0049] The processing server 100 uses the generation system AI 180 to generate past time-space data representing the past time-space of the user 200, using the material data 210. For example, the processing server 100 uses an image included in the material data 210 to generate spatial data 502 representing the situation of the location where the user 200 was located, including the surrounding scenery. The processing server 100 may generate the spatial data 502 by inputting prompt information including an image included in the material data 210 and an instruction to generate the spatial data 502 representing the situation of the location where the user 200 was located, including the surrounding scenery, using the image to the generation system AI 180. The processing server 100 may generate the spatial data 502 at multiple time points using multiple images at multiple time points. Since the spatial data 502 at multiple time points represents the situation of the location where the user 200 was located at each of the multiple time points, if the locations where the user 200 was located at the multiple time points are different, the multiple spatial data 502 will represent the situations of the different locations. If the location where the user 200 was located at multiple points in time was the same, then the multiple pieces of spatial data 502 will represent the situation at the same location.

[0050] The processing server 100 may further use the material data 210 provided by the other users 200 to generate the spatial data 502. For example, the processing server 100 generates the spatial data 502 for a certain location by using an image included in the material data 210 of the user 200 and an image included in the material data 210 of the other users 200. The processing server 100 may generate the spatial data 502 by inputting prompt information including an image included in the material data 210 of the user 200, an image included in the material data 210 of the other users 200, and an instruction to generate the spatial data 502 using these images to the generation AI 180.

[0051] The processing server 100 may automatically complement a time point where there is no image. For example, the processing server 100 generates spatial data 504 by complementing an image at a time point where there is no image from images at the time point before and after the time point. The processing server 100 may generate the spatial data 504 by inputting prompt information to the generation system AI 180, the prompt information including images at the time points before and after the time point and an instruction to generate spatial data 504 by complementing images at the time points between the time points from the images at the time points before and after the time point.

[0052] The processing server 100 may generate the spatial data 502 and the spatial data 504 by inputting prompt information to the generation system AI 180, the prompt information including an image included in the material data 210 of the user 200, an instruction to generate spatial data 502 using the image, and, for a point in time when there is no image, an instruction to generate spatial data 504 by complementing the image at that point in time from images at previous and subsequent points in time.

[0053] The processing server 100 may generate past spatio-temporal data using the generated spatial data 502 and spatial data 504. The processing server 100 may generate past spatio-temporal data by inputting the generated spatial data 502 and spatial data 504 and prompt information including an instruction to generate past spatio-temporal data using them to the generation AI 180.

[0054] The processing server 100 may generate future spatio-temporal data 508 representing the future spatio-temporal space of the user 200, which is a prediction of the future of the user 200, based on the material data 210 of the user 200. For example, the processing server 100 may identify the past history of the user 200 based on the material data 210 of the user 200, and generate the future spatio-temporal data 508 of the user 200 based on the identified history. The processing server 100 may generate the future spatio-temporal data 508 by inputting the material data 210 of the user 200 and prompt information including an instruction to identify the past history of the user 200 based on the material data 210 and generate the future spatio-temporal data 508 of the user 200 based on the identified history to the generation AI 180.

[0055] FIG. 3 is an explanatory diagram for explaining the spatiotemporal data generation function for corporations by the processing server 100. In the example shown in FIG. 3, the processing server 100 receives material data 220 about a specific object from the communication terminal 300 of the user 200. The material data 220 includes an image about the object. The material data 220 may include images at each of a plurality of time points. The material data 220 may include map data including information on the point where the image was captured. The material data 220 may include paper data including a paper about the object. The material data 220 may include research material data including research materials about the object. The material data 220 may include newspaper article data including newspaper articles about the object. The material data 220 may include minutes data including minutes of a meeting about the object. The material data 220 may include data other than these.

[0056] The processing server 100 utilizes the generation system AI 180 to generate past time-space data representing the past time-space of the target using the material data 220. For example, the processing server 100 generates spatial data 512 representing the target including the surrounding scenery using an image included in the material data 220. The processing server 100 may generate the spatial data 512 by inputting prompt information including an image included in the material data 220 and an instruction to generate the spatial data 512 representing the target including the surrounding scenery using the image to the generation system AI 180. The processing server 100 may generate the spatial data 512 at multiple points in time using multiple images at multiple points in time.

[0057] The processing server 100 may automatically complement a time point where there is no image. For example, the processing server 100 generates spatial data 514 by complementing an image at a time point where there is no image from images at the time point before and after the time point. The processing server 100 may generate the spatial data 514 by inputting prompt information to the generation system AI 180, the prompt information including images at the time points before and after the time point and an instruction to generate spatial data 514 by complementing images at the time points between the time points from the images at the time points before and after the time point.

[0058] The processing server 100 may generate the spatial data 512 and the spatial data 514 by inputting prompt information to the generation system AI 180, the prompt information including an image included in the material data 220, an instruction to generate spatial data 512 using the image, and, for a point in time when no image is available, an instruction to generate spatial data 514 that complements the image at that point in time from images at the previous and following points in time.

[0059] The processing server 100 may generate past spatio-temporal data using the generated spatial data 512 and spatial data 514. The processing server 100 may generate past spatio-temporal data by inputting the generated spatial data 512 and spatial data 514 and prompt information including an instruction to generate past spatio-temporal data using them to the generation AI 180.

[0060] The processing server 100 may generate future spatio-temporal data 518 representing the future space-time of the target, which is a prediction of the future of the target, based on the material data 220. For example, the processing server 100 may identify repeated events in the history of the target based on the material data 220, and generate the future spatio-temporal data 518 of the target based on the identified events. The processing server 100 may generate the future spatio-temporal data 518 by inputting the material data 220 and prompt information including an instruction to identify repeated events in the history of the target based on the material data 220 and generate the future spatio-temporal data 518 based on the identified events to the generation AI 180.

[0061] The processing server 100 may generate a plurality of different future-time space data 518 based on the material data 220. For example, the processing server 100 generates future-time space data 518 representing a time space in which the future of the target is normally predicted, future-time space data 518 representing a time space in which the good future of the target is predicted, and future-time space data 518 representing a time space in which the bad future of the target is predicted, based on the material data 220. The processing server 100 may input prompt information including the material data 220 and an instruction to generate the future-time space data 518 representing the time space in which the future of the target is normally predicted, the future-time space data 518 representing the time space in which the good future of the target is predicted, and the future-time space data 518 representing the time space in which the bad future of the target is predicted, based on the material data 220, to the generation AI 180, and obtain three future-time space data 518 from the generation AI 180.

[0062] 4 is an explanatory diagram for explaining the correction of spatio-temporal data by the processing server 100. In the spatio-temporal data generation function for individuals, the processing server 100 may generate past spatio-temporal data and future spatio-temporal data corrected based on the interests of the user 200. In the spatio-temporal data generation function for corporations, the processing server 100 may generate past spatio-temporal data and future spatio-temporal data corrected based on the interests of the user 200 who experiences it.

[0063] First, a case where the user 200 generates spatio-temporal data from his / her own material in the spatio-temporal data generation function for individuals will be described. In the generation of past spatio-temporal data, if no correction is performed, the processing server 100 generates past spatio-temporal data representing a past virtual space-time from an image included in the material data 210. If correction is performed, the processing server 100 generates past spatio-temporal data from an image included in the material data 210, and identifies an area in the space of the past spatio-temporal data in which the user 200 was highly interested by using data other than images included in the material data 210 (diary, etc.), and corrects the past spatio-temporal data so as to emphasize the identified area. As a specific example, in a space in which a tower surrounded by a moat and a bridge over the moat exist, if the user 200 was highly interested in the tower, the processing server 100 applies correction to emphasize the tower. The method of emphasis may be any method that can emphasize an object, such as emphasis by making the color conspicuous or emphasis by increasing the size. The processing server 100 may correct the past spatio-temporal data so as to make less noticeable an area in the space of the past spatio-temporal data in which the user 200 had little interest. As a specific example, in a space in which a tower surrounded by a moat and a bridge over the moat exist, if the user 200 had little interest in the bridge, the processing server 100 applies a correction to make the bridge less noticeable. The method of making it less noticeable may be any method that can make the target less noticeable, such as making the color less noticeable, reducing the size, deleting information about the target part and then generating a less noticeable and natural area based on information about the surrounding environment. In the generation of the future spatio-temporal data, if no correction is made, the processing server 100 identifies the past history of the user 200 based on the material data 210, predicts the future from the identified history, and generates the future spatio-temporal data. When correction is made, the processing server 100 identifies the past history, interests, and level of interest of the user 200 based on the material data 210, and takes into account the identified interests and level of interest, predicts the future from the identified history, and generates future time-space data.As a specific example, if the user 200 has toured a garden in the past and has a high level of interest in a tower within the garden, the processing server 100 generates future time-space data representing a future time-space in which the user 200 will stroll through a garden that has a famous or special tower.

[0064] Next, a case where spatio-temporal data is jointly generated from materials of multiple people in the spatio-temporal data generation function for individuals will be described. In the generation of past spatio-temporal data, if no correction is performed, the processing server 100 generates past spatio-temporal data representing a past virtual space-time from images included in the multiple material data 210. If correction is performed, the processing server 100 generates past spatio-temporal data from images included in the multiple material data 210, and identifies areas in the space of the past spatio-temporal data that each of the multiple users 200 was interested in and was not interested in, using data other than images included in the multiple material data 210 (diaries, etc.), and corrects the past spatio-temporal data so that each user can confirm their areas of interest and areas of no interest. As a specific example, in a space where a tower surrounded by a moat and a bridge over the moat exist, if the first user 200 is interested in the tower and the second user 200 is not interested in the bridge, the processing server 100 corrects the past spatio-temporal data so that it is possible to confirm that the first user 200 is interested in the tower and the second user 200 is not interested in the bridge. For example, the processing server 100 adds an explanatory text to the tower in the past spatio-temporal data explaining that the first user 200 was interested in it, and adds an explanatory text to the bridge in the past spatio-temporal data explaining that the second user 200 was not interested in it. The processing server 100 may have a function of generating past spatio-temporal data reflecting the interest of a selected user 200 from among the multiple users 200. In this case, when the first user 200 is selected, the processing server 100 applies a correction to emphasize the tower part in the past spatio-temporal data. Furthermore, when the second user 200 is selected, the processing server 100 applies a correction to make the bridge portion in the past-time space-time data less noticeable. In generating future-time space-time data, when no correction is performed, the processing server 100 identifies the past history of the multiple users 200 based on the multiple material data 210, predicts the future from the identified history, and generates future-time space-time data. When correction is performed, the processing server 100 identifies the past history and common interests of the multiple users 200 based on the multiple material data 210, and generates future-time space-time data by predicting the future from the identified history, taking into account the identified interests.As a specific example, if multiple users 200 have toured a garden in the past and a common interest is a tower in the garden, the processing server 100 generates future time-space data representing a future time-space in which the multiple users 200 will stroll through a garden where a famous or special tower is located.

[0065] Next, a case where spatio-temporal data is generated from a corporation's material in the spatio-temporal data generation function for corporations will be described. In the generation of past spatio-temporal data, in the case where no correction is performed, the processing server 100 generates past spatio-temporal data representing a past virtual space-time from an image included in the material data 220 provided by the corporation. In the case where correction is performed, the processing server 100 generates past spatio-temporal data from an image included in the material data 220, and corrects the past spatio-temporal data according to the interest of the user 200 who experiences the past spatio-temporal data. For example, the processing server 100 may acquire material data 210 of the user 200 who experiences the past spatio-temporal data, and use the material data 210 to identify the interest of the user 200. For example, the processing server 100 identifies an interest of the user 200 among a plurality of matters in the space of the past spatio-temporal data, and corrects the past spatio-temporal data so as to emphasize an area corresponding to the identified interest. For example, in a case where the corporation is a research institute and the past spatio-temporal data represents the history of multiple types of research by the research institute, the processing server 100 identifies, from among the multiple types of research, a research that matches the user 200's interests, and corrects the past spatio-temporal data so as to emphasize the area corresponding to the identified research. In the generation of future spatio-temporal data, in a case where no correction is performed, the processing server 100 predicts multiple possible futures from various past histories of the corporation based on the material data 220, and generates multiple future spatio-temporal data representing each of the predicted multiple futures. In a case where correction is performed, the processing server 100 predicts multiple possible futures from various past histories of the corporation based on the material data 220, and narrows down the predicted multiple futures according to the interests of the user 200 who experiences the future spatio-temporal data, and generates future spatio-temporal data representing the narrowed down futures.

[0066] Fig. 5 shows an example of a processing flow in the processing system 10. In the example shown in Fig. 5, first, the user 200 performs a prompt input to the communication terminal 300. The user 200 performs a prompt input including an instruction of a target time and place and material data 210 such as a still image, a video, etc. to the communication terminal 300. The communication terminal 300 transmits the prompt input to the processing server 100 and stores it in the DB server 190. The DB server 190 stores the specified time and place in association with the material data 210. The processing server 100 uses the generative AI 180 to generate past time-space data representing the time-space of the specified time and place using the material data 210.

[0067] The processing server 100 transmits the generated past time-space data to the communication terminal 300 and presents it to the user 200. The processing server 100 may transmit the past time-space data to the eyewear 400 and present it to the user 200. The user 200 checks the past time-space data and selects OK or NG. If OK is selected, the communication terminal 300 notifies the processing server 100 that it is OK, and the processing server 100 stores the past time-space data in the DB server 190. The DB server 190 may store the past time-space data in a time-space library.

[0068] When NG is selected, the communication terminal 300 prompts the user 200 to additionally input the material data 210. The user 200 additionally inputs the material data 210. The communication terminal 300 transmits the additionally input material data 210 to the processing server 100. The processing server 100 reads the existing material data 210 from the DB server 190, and regenerates the past spatio-temporal data by combining it with the additionally input material data 210. The processing server 100 stores the regenerated past spatio-temporal data in the DB server 190. The DB server 190 may store the past spatio-temporal data in the spatio-temporal library. Note that the processing server 100 may present the regenerated past spatio-temporal data to the user 200 again, and when OK is selected, store the past spatio-temporal data in the DB server 190, and when NG is selected, accept additional input of further material data 210.

[0069] The communication terminal 300 may receive a prompt input from the user 200, including an instruction for a time and place to be a target, material data 210 of a material for correction such as a diary, and an instruction for correction. The communication terminal 300 transmits the received prompt input to the processing server 100. Based on the received instruction, the processing server 100 may read the existing material data 210 from the DB server 190, and generate and correct past spatio-temporal data by combining it with the material data 210 of the material for correction. The processing server 100 stores the past spatio-temporal data in the DB server 190. The DB server 190 may store the past spatio-temporal data in a spatio-temporal library.

[0070] 6 shows an example of a schematic functional configuration of the processing server 100. The processing server 100 includes a storage unit 102, a reception unit 104, a material data acquisition unit 106, a spatio-temporal data generation unit 108, a spatio-temporal data provision unit 110, and an AI management unit 112. Note that it is not essential that the processing server 100 includes all of these units.

[0071] The storage unit 102 stores various data. When the processing server 100 has the generation AI 180, the storage unit 102 stores the generation AI 180.

[0072] The reception unit 104 receives various inputs from the user 200. The reception unit 104 stores the received inputs from the user 200 in the storage unit 102. The reception unit 104 may store the received inputs from the user 200 in the DB server 190. The reception unit 104 receives the inputs from the user 200 from the communication terminal 300, for example. The reception unit 104 receives the inputs from the user 200 from the eyewear 400, for example.

[0073] The receiving unit 104 receives, for example, an instruction from the user 200 and the material data 210. The instruction from the user 200 may include an instruction to generate past spatio-temporal data. The instruction to generate past spatio-temporal data may include information on a target time and place. The instruction from the user 200 may include an instruction to generate future spatio-temporal data. The instruction to generate future spatio-temporal data may include information on a target time and place. The information on the target time may indicate a period, for example, from xx years ago to xx years ago. The information on the target place may indicate an area indicated by a place name, an address, and latitude and longitude. The receiving unit 104 may receive material data 210 from another user 200 in addition to the instruction from the user 200 and the material data 210.

[0074] The material data acquisition unit 106 acquires material data 210 used when generating past-time spatiotemporal data or future-time spatiotemporal data. The material data acquisition unit 106 may acquire the material data 210 from the storage unit 102. The material data acquisition unit 106 may acquire the material data 210 from the DB server 190.

[0075] The spatio-temporal data generation unit 108 generates spatio-temporal data of the user 200. For example, the spatio-temporal data generation unit 108 generates past spatio-temporal data of the user 200. The spatio-temporal data generation unit 108 generates the past spatio-temporal data by using the instruction accepted by the acceptance unit 104 and the material data 210. The spatio-temporal data generation unit 108 may generate the past spatio-temporal data of the user 200 based on the material data 210, the past spatio-temporal data being composed of spatial data in which the user 200 was located at a plurality of time points.

[0076] The spatio-temporal data generating unit 108 may generate the past spatio-temporal data by using the generative AI 180. For example, the spatio-temporal data generating unit 108 generates instruction data including an instruction to generate the past spatio-temporal data, and inputs the generated instruction data to the generative AI 180 to generate the past spatio-temporal data. The instruction data may be a so-called prompt input. The instruction data may include an instruction to generate spatial data representing the situation of the location where the user 200 was located at a plurality of time points using the material data 210 and a plurality of captured images included in the material data 210, and to generate the spatio-temporal data using the generated plurality of spatial data. By giving the generative AI 180 an instruction to generate the past spatio-temporal data representing the past space-time of the user 200, rather than simply giving the generative AI 180 an instruction to generate captured images at a plurality of time points in the past and spatial data at a plurality of time points from the plurality of captured images, and to generate the past spatio-temporal data using the plurality of spatial data, it is possible to reproduce the spatial data at the time points when the captured images exist in the material data 210 with a relatively high accuracy, and to generate a 3D video representing the past experience of the user 200 by linking them in the time axis direction.

[0077] The spatiotemporal data generating unit 108 may instruct the generative AI 180 of the "true emotion" of the user 200 regarding the event that can only be understood by the person involved when the user 200 experienced the event in the past. The "true emotion" may be a true emotion, not an imaginary emotion or a false emotion. For example, the receiving unit 104 receives material data 210 including captured images captured at points where the user 200 was located at each of a plurality of past time points and the "true emotion" of the user 200 regarding the event that the user 200 experienced at each of the points, and the spatiotemporal data generating unit 108 uses the material data 210, the captured images included in the material data 210, and the "true emotion" of the user 200 included in the material data 210 to generate spatial data representing the situation of each of the points where the user 200 was located at a plurality of time points, and inputs instruction data including an instruction to generate spatiotemporal data using the generated plurality of spatial data to the generative AI 180.

[0078] The spatio-temporal data generation unit 108 stores the generated spatial data and the past spatio-temporal data in the storage unit 102. The spatio-temporal data generation unit 108 may store the generated spatial data and the past spatio-temporal data in the DB server 190.

[0079] The time-space data providing unit 110 provides the past time-space data generated by the time-space data generating unit 108 to the user 200. The time-space data providing unit 110 may transmit the past time-space data to the communication terminal 300. The time-space data providing unit 110 may transmit the past time-space data to the eyewear 400.

[0080] The receiving unit 104 may receive feedback from the user 200 on the past spatio-temporal data provided by the spatio-temporal data providing unit 110. For example, the receiving unit 104 receives feedback including designation of an area that matches the memory of the user 200 and an area that does not match the memory of the user 200 in the space-time represented by the past spatio-temporal data. The spatio-temporal data generating unit 108 may correct the past spatio-temporal data based on the feedback from the user 200. For example, the spatio-temporal data generating unit 108 corrects the past spatio-temporal data by inputting the past spatio-temporal data and instruction data including an instruction to correct the past spatio-temporal data based on the feedback from the user 200 to the generation AI 180.

[0081] The spatio-temporal data generating unit 108 may generate the past spatio-temporal data while having the user 200 confirm the spatial data. For example, the spatio-temporal data generating unit 108 inputs the material data 210 and instruction data including an instruction to generate spatial data representing the situation of a point where the user 200 was located at a plurality of time points using a plurality of captured images included in the material data 210 to the generating AI 180, and acquires a plurality of spatial data from the generating AI 180. The spatio-temporal data providing unit 110 provides the plurality of spatial data generated by the spatio-temporal data generating unit 108 to the user 200. The receiving unit 104 receives feedback from the user 200 on the spatial data provided by the spatio-temporal data providing unit 110. The spatio-temporal data generating unit 108 corrects the spatial data based on the feedback from the user 200. For example, the spatio-temporal data generating unit 108 corrects the spatial data by inputting the spatial data and instruction data including an instruction to correct the spatial data based on the feedback from the user 200 to the generating AI 180. After completing the correction of the plurality of spatial data, the spatio-temporal data generation unit 108 generates past spatio-temporal data using the plurality of spatial data. This makes it possible to generate past spatio-temporal data after generating spatial data corresponding to each of the plurality of captured images included in the material data 210 and correcting erroneous areas, thereby making it possible to generate past spatio-temporal data with a high degree of reproducibility.

[0082] The spatiotemporal data generating unit 108 may generate spatial data representing the locations where the user 200 was located at each of the multiple time points using the multiple captured images and map data included in the material data 210, generate instruction data including an instruction to generate past spatiotemporal data using the multiple generated spatial data, and input the instruction data to the generation system AI 180. This makes it possible to correct or supplement the surrounding image of the location where the user 200 was located using information from the map data, which can contribute to improving the reproduction of the spatiotemporal data.

[0083] The spatiotemporal data generating unit 108 may generate spatial data representing the locations where the user was located at multiple time points using multiple captured images included in the material data 210, estimate the interest level of the user 200 in multiple areas in the spatial data for each of the multiple spatial data using the diary data, correct the spatial data according to the estimated interest level, and then generate instruction data including an instruction to generate spatiotemporal data using the multiple spatial data, and input the instruction data to the generation system AI 180. The spatiotemporal data generating unit 108 may, for example, include an instruction to emphasize an area of ​​the multiple areas of the spatial data that is of high interest to the user 200 in the instruction data. The spatiotemporal data generating unit 108 may, for example, include an instruction to make an area of ​​the multiple areas of the spatial data that is of low interest to the user 200 less noticeable in the instruction data. The diary data includes the date and time, the location, and the contents of the diary written by the user 200. That is, the diary data includes information on the interests and interest level of the user 200 in the locations where the user 200 was located at each of the multiple time points. By inputting diary data to the generation system AI 180 in addition to multiple captured images, it is possible to estimate the user's 200 interests and level of interest at the time and place when each of the multiple captured images was captured, making it possible to generate past time-space data that reflects the interests and level of interest.

[0084] The spatio-temporal data generation unit 108 may supplement the spatial data for a time point where the material data 210 does not include a captured image by using a captured image included in the material data 210. For example, the spatio-temporal data generation unit 108 generates spatial data representing a point where the user was located at each of a plurality of time points by using a plurality of captured images included in the material data 210, generates spatial data for a time point between consecutive time points among the plurality of time points by using spatial data for the consecutive time points, generates instruction data including an instruction to generate past spatio-temporal data by using the generated plurality of spatial data, and inputs the instruction data to the generation system AI 180. For example, when the user 200 was located at a first location at a first time point and at a second location at a second time point, the spatio-temporal data generation unit 108 causes the generation system AI 180 to generate spatial data corresponding to the third time point by assuming that the user 200 was located at a third location between the first location and the second location at a third time point between the first time point and the second time point. This makes it possible to generate spatial data at a time when no captured image existed, although accurate reproduction is difficult, and it is possible to generate past time-space data with high temporal continuity, i.e., data that is not discontinuous in time, thereby contributing to improving the quality of experience of user 200.

[0085] When generating the past spatio-temporal data of the user 200, if the receiving unit 104 receives material data 210 from other users 200 as well, the spatio-temporal data generating unit 108 may generate the past spatio-temporal data of the user 200 using the material data 210 of the multiple users 200 including the user 200 and the other users 200. The spatio-temporal data generating unit 108 may estimate the interests and the degree of interest of each of the multiple users 200 using the material data 210 of each of the multiple users 200, and generate the past spatio-temporal data of the user 200 using the estimated interests and the degree of interest. For each of the multiple spatial data, the spatio-temporal data generating unit 108 may include an instruction to highlight an area that was of high interest to any of the multiple users 200 among multiple areas included in the spatial data, and to make it possible to confirm which user 200 was of high interest, in the instruction data. The spatio-temporal data generating unit 108 may include in the instruction data an instruction to make less noticeable an area that has low interest of any of the multiple users 200 among the multiple areas included in the spatial data for each of the multiple spatial data, and to make it possible to confirm which user 200 has low interest. The spatio-temporal data providing unit 110 provides the past spatio-temporal data generated by the generative AI 180 in response to the instruction data. This makes it possible to provide past spatio-temporal data that allows the user 200 to confirm areas in the past time-space of the user 200 that have high or low interest, as well as areas in the past time-space of the user 200 that have high or low interest. Note that the spatio-temporal data generating unit 108 may generate multiple pieces of past spatio-temporal data corresponding to each of the multiple users 200. For example, the spatio-temporal data generating unit 108 generates past spatio-temporal data of the user 200, and includes in the instruction data an instruction to generate multiple pieces of past spatio-temporal data by correcting the past spatio-temporal data in accordance with the interests of each of the multiple users 200. The spatio-temporal data providing unit 110 provides a plurality of past spatio-temporal data generated by the generative AI 180 in response to the instruction data.The time-space data providing unit 110, for example, receives a designation from the experiencing user 200 as to which of the multiple users 200's interests the user wishes to be provided with past time-space data reflecting that interest, and provides the experiencing user 200 with the past time-space data reflecting the interest of the designated user 200.

[0086] The spatio-temporal data providing unit 110 may provide the past spatio-temporal data of the user 200 to other users 200 approved by the user 200. The processing server 100 may manage the past spatio-temporal data of the user 200 so as not to provide it to other users 200 unless approval is obtained from the user 200. This makes it possible to provide an environment in which other people can simulate the past experiences of a person with the person's consent.

[0087] The spatio-temporal data providing unit 110 may make the past spatio-temporal data of the user 200 public when approval is obtained from the user 200. For example, the spatio-temporal data providing unit 110 makes the past spatio-temporal data of the user 200 public in the metaverse when approval is obtained from the user 200. This makes it possible to realize a metaverse space in which the past experiences of various users 200 can be simulated, on the condition that approval is obtained from the user 200.

[0088] The spatio-temporal data generation unit 108 generates, for example, future spatio-temporal data of the user 200. The spatio-temporal data generation unit 108 generates the future spatio-temporal data using the instruction and the material data 210 accepted by the acceptance unit 104. The spatio-temporal data generation unit 108 generates the future spatio-temporal data by generating instruction data including an instruction to generate the future spatio-temporal data and inputting the generated instruction data to the generative AI 180. The instruction data may be a so-called prompt input. The instruction data may include the material data 210 and an instruction to estimate the situation of the user 200 at each of multiple points in time using the material data 210, estimate the past history of the user 200, and predict the future from the past history of the user 200 to generate the future spatio-temporal data. Rather than simply giving instructions to the generative AI 180 to generate future time-space data representing the future time-space of the user 200, by giving instruction data including the material data 210 and instructions to estimate the past history of the user 200 and predict the future from the past history, it may be possible to generate future time-space data that is in line with the future situation of the user 200 and the future aspirations of the user 200.

[0089] The spatio-temporal data generating unit 108 may estimate the user 200's interests based on the material data 210, and generate future spatio-temporal data of the user 200 based on the estimated interests. For example, the spatio-temporal data generating unit 108 inputs instruction data to the generating AI 180, including the material data 210, an instruction to estimate the status of the user 200 at multiple time points using the material data 210, estimate the user 200's past history, and predict the future from the user 200's past history, an instruction to estimate the user 200's interests at multiple time points using the material data 210, and an instruction to generate future spatio-temporal data using the predicted future and the estimated interests. The spatio-temporal data generating unit 108 may input instruction data to the generating AI 180, including an instruction to make a correction to emphasize an area related to the user 200's interests among areas in the future space-time. This makes it possible to generate future spatio-temporal data in which an area in which the user 200 shows interest is emphasized with respect to the future predicted from the user 200's past history.

[0090] The spatio-temporal data generation unit 108 may estimate the interests of the multiple users 200 based on the material data 210 of the multiple users 200 including the user 200 and other users 200, and generate future spatio-temporal data of the user 200 based on the estimated interests. For example, the spatio-temporal data generation unit 108 inputs to the generation system AI 180 instruction data including the material data 210 of the user 200, an instruction to estimate the status of each user 200 at multiple time points using the material data 210 of the user 200, estimate the past history of the user 200, and predict the future from the past history of the user 200, an instruction to estimate the interests of the user 200 at multiple time points using the material data 210 of the user 200, an instruction to estimate the interests of the other users 200 at multiple time points using the material data 210 of the other users 200, and an instruction to generate future spatio-temporal data using the predicted future and the estimated interests of the user 200 and the other users 200. This makes it possible to generate future time-space data that reflects the interests of a plurality of users 200 with respect to the future predicted from the past history of the users 200.

[0091] The time-space data generating unit 108 stores the generated future time-space data in the storage unit 102. The time-space data generating unit 108 may store the generated future time-space data in the DB server 190. The time-space data providing unit 110 provides the future time-space data generated by the time-space data generating unit 108 to the user 200. The time-space data providing unit 110 may transmit the future time-space data to the communication terminal 300. The time-space data providing unit 110 may transmit the future time-space data to the eyewear 400. The time-space data providing unit 110 may provide the future time-space data of the user 200 to another user 200 approved by the user 200. The time-space data providing unit 110 may manage the future time-space data of the user 200 so as not to provide it to another user 200 unless approval by the user 200 is obtained. The spatio-temporal data providing unit 110 may publish the future spatio-temporal data of the user 200 when approval by the user 200 is obtained. For example, the spatio-temporal data providing unit 110 publishes the future spatio-temporal data of the user 200 in the metaverse when approval by the user 200 is obtained. Compared to the past, the future spatio-temporal data has a wider range of uncertainties, and therefore the need for approval by the user 200 is considered to be higher.

[0092] The receiving unit 104 receives, for example, an instruction from the user 200 and the material data 220. The instruction from the user 200 may include an instruction to generate target past-time-space data. The instruction from the user 200 may include an instruction to generate target future-time-space data. The target time information may indicate a period, for example, from xx years ago to xx years ago. The target location information may indicate an area indicated by a place name, an address, and latitude and longitude.

[0093] The material data acquisition unit 106 acquires material data 220 used when generating past-time spatiotemporal data or future-time spatiotemporal data. The material data acquisition unit 106 may acquire the material data 220 from the storage unit 102. The material data acquisition unit 106 may acquire the material data 220 from the DB server 190.

[0094] The spatio-temporal data generation unit 108 generates spatio-temporal data of the target. For example, the spatio-temporal data generation unit 108 generates past spatio-temporal data of the target. The spatio-temporal data generation unit 108 generates the past spatio-temporal data by using the instruction accepted by the acceptance unit 104 and the material data 220. The spatio-temporal data generation unit 108 may generate the past spatio-temporal data of the target, which is composed of spatial data of the target at multiple points in time, based on the material data 220.

[0095] The spatio-temporal data generation unit 108 may generate past spatio-temporal data of the target using the generative AI 180. For example, the spatio-temporal data generation unit 108 generates instruction data including an instruction to generate past spatio-temporal data of the target, and inputs the generated instruction data to the generative AI 180 to generate past spatio-temporal data of the target. The instruction data may be a so-called prompt input. The instruction data may include an instruction to generate spatial data for each of a plurality of time points using the material data 220, a plurality of images included in the material data 220, and at least one of a paper, research material, newspaper article, and minutes included in the material data 220, and to generate past spatio-temporal data using the generated plurality of spatial data. Rather than simply instructing the generation system AI 180 to generate past spatio-temporal data that represents the past time and space of the object, the generation system AI 180 is instructed to generate spatial data for multiple points in time using images of the object at multiple points in the past and at least one of papers, research materials, newspaper articles, and minutes related to the object, and to generate past spatio-temporal data using the multiple spatial data.This makes it possible to reproduce the spatial data at the time when the image exists in the material data 220 with relatively high accuracy, and to generate a 3D video that represents the history of the object by linking these data along the time axis.

[0096] The material data 220 may include, for example, paper data of a paper that discusses the history of the subject. By inputting the material data 220 including such paper data to the generative AI 180, it may be possible to reflect the history of the subject that cannot be determined from an image alone in the past spatio-temporal data. The material data 220 may further include the impact factor of the academic journal in which the paper is published. The spatio-temporal data generating unit 108 may instruct the generative AI 180 to generate past spatio-temporal data by placing more importance on the content of the paper data of a paper published in an academic journal with a higher impact factor than the content of the paper data of a paper published in an academic journal with a lower impact factor. This may enable the generation of more reliable past spatio-temporal data.

[0097] The material data 220 may include, for example, research material data of research materials that have been studied over a long period of time. By inputting the material data 220 including such research material data into the generative AI 180, it may be possible to reflect the history of the object, which cannot be determined from the image alone, in the past spatiotemporal data.

[0098] The material data 220 may include, for example, newspaper article data including newspaper articles containing content about the target. By inputting the material data 220 including such newspaper article data to the generative AI 180, it may be possible to reflect highly credible facts and the like written by newspapers in the past spatio-temporal data. The material data 220 may further include the circulation of the newspaper in which the newspaper article is published. The spatio-temporal data generating unit 108 may instruct the generative AI 180 to generate past spatio-temporal data by placing more importance on the content of newspaper articles published in newspapers with a larger circulation than the content of newspaper articles published in newspapers with a smaller circulation. This may enable the generation of more credible past spatio-temporal data.

[0099] The material data 220 may include, for example, minutes data including minutes of a meeting about the subject. By inputting the material data 220 including such minutes data to the generative AI 180, it may be possible to reflect various official thoughts about the subject in the past spatio-temporal data.

[0100] The spatio-temporal data generation unit 108 stores the generated spatial data and the past spatio-temporal data in the storage unit 102. The spatio-temporal data generation unit 108 may store the generated spatial data and the past spatio-temporal data in the DB server 190.

[0101] The time-space data providing unit 110 provides the past time-space data generated by the time-space data generating unit 108 to the user 200. The time-space data providing unit 110 may transmit the past time-space data to the communication terminal 300. The time-space data providing unit 110 may transmit the past time-space data to the eyewear 400.

[0102] The receiving unit 104 may receive feedback from the user 200 on the past spatio-temporal data provided by the spatio-temporal data providing unit 110. For example, the receiving unit 104 receives feedback including designation of an area that matches with facts and an area that does not match with facts in the space-time represented by the past spatio-temporal data. The spatio-temporal data generating unit 108 may correct the past spatio-temporal data based on the feedback from the user 200. For example, the spatio-temporal data generating unit 108 corrects the past spatio-temporal data by inputting the past spatio-temporal data and instruction data including an instruction to correct the past spatio-temporal data based on the feedback from the user 200 to the generative AI 180.

[0103] The spatio-temporal data generating unit 108 may generate the past spatio-temporal data while having the user 200 confirm the spatial data. For example, the spatio-temporal data generating unit 108 inputs instruction data including an instruction to generate spatial data at a plurality of time points using the material data 220, a plurality of images included in the material data 220, and at least one of a paper, a research material, a newspaper article, and a meeting minutes included in the material data 220, to the generative AI 180, and acquires a plurality of spatial data from the generative AI 180. The spatio-temporal data providing unit 110 provides the plurality of spatial data generated by the spatio-temporal data generating unit 108 to the user 200. The receiving unit 104 receives feedback from the user 200 on the spatial data provided by the spatio-temporal data providing unit 110. The spatio-temporal data generating unit 108 corrects the spatial data based on the feedback from the user 200. For example, the spatio-temporal data generation unit 108 corrects the spatial data by inputting instruction data including the spatial data and an instruction to correct the spatial data based on feedback from the user 200 to the generative AI 180. After completing the correction of the plurality of spatial data, the spatio-temporal data generation unit 108 generates past spatio-temporal data using the plurality of spatial data. In this way, it is possible to generate the plurality of spatial data using the material data 220, correct erroneous areas, and generate past spatio-temporal data with a high degree of reproducibility.

[0104] When the material data 220 includes map data, the spatiotemporal data generation unit 108 may generate spatial data for multiple points in time using multiple images and map data included in the material data 220, generate instruction data including an instruction to generate past spatiotemporal data using the generated multiple spatial data, and input the instruction data to the generation system AI 180. This makes it possible to correct or supplement the surrounding image of the target using information from the map data, which can contribute to improving the reproduction of the spatiotemporal data.

[0105] The spatio-temporal data generating unit 108 may supplement the spatial data for a time point where no image is included in the material data 220, using an image included in the material data 220. For example, the spatio-temporal data generating unit 108 generates spatial data for each of a plurality of time points using a plurality of images included in the material data 220, generates spatial data for a time point between the consecutive time points using spatial data for the consecutive time points among the plurality of time points, generates instruction data including an instruction to generate past spatio-temporal data using the generated plurality of spatial data, and inputs the instruction data to the generation system AI 180. This makes it possible to generate spatial data for a time point where no image exists, although accurate reproduction is difficult, and can generate past spatio-temporal data with high temporal continuity, i.e., without being discontinuous in time, which can contribute to improving the quality of experience of the user 200 who experiences the past spatio-temporal data.

[0106] The spatio-temporal data generating unit 108 may estimate the interests of the user 200 based on user-related information related to the user 200 who experiences the generated past spatio-temporal data, and may correct the past spatio-temporal data based on the estimated interests. The user-related information of the user 200 may be, for example, data posted by the user 200 on a social networking service (SNS). The spatio-temporal data generating unit 108 may acquire and use data posted by the user 200 on the SNS used by the user 200 from the communication terminal 300 of the user 200. The user-related information may be material data 210 provided by the user 200. For example, the spatio-temporal data generating unit 108 corrects the past spatio-temporal data so as to emphasize an area corresponding to the user's interests among a plurality of areas of space included in the past spatio-temporal data. The spatio-temporal data generation unit 108 may input instruction data to the generation system AI 180, the instruction data including past spatio-temporal data and an instruction to emphasize an area corresponding to a subject of interest of the user 200 among multiple areas of space included in the past spatio-temporal data, and obtain corrected past spatio-temporal data from the generation system AI 180.

[0107] The spatio-temporal data providing unit 110 may provide the past spatio-temporal data of the target to other users 200 approved by the user 200. The processing server 100 may manage the past spatio-temporal data of the target not to be provided to other users 200 unless approval is obtained from the user 200. In this way, if approval is obtained from the user 200 related to the target such as a region or a corporation, an environment can be provided in which other people can simulate an experience of the past history of the target.

[0108] The spatio-temporal data providing unit 110 may make the past spatio-temporal data of the target public when approval is obtained from the user 200. For example, the spatio-temporal data providing unit 110 makes the past spatio-temporal data of the target public in the metaverse when approval is obtained from the target. This makes it possible to realize a metaverse space in which the history of the target can be simulated, on the condition that approval is obtained from the user 200 related to the target, such as a region or a corporation.

[0109] The spatio-temporal data generating unit 108 generates, for example, future spatio-temporal data of the target. The spatio-temporal data generating unit 108 generates future spatio-temporal data using the instruction and the material data 220 received by the receiving unit 104. The spatio-temporal data generating unit 108 generates the future spatio-temporal data by generating instruction data including an instruction to generate future spatio-temporal data and inputting the generated instruction data to the generative AI 180. The instruction data may be a so-called prompt input. The instruction data may include the material data 220 and an instruction to identify an event that has repeatedly occurred in the history of the target using the material data 220 and generate future spatio-temporal data of the target based on the identified event. By giving the generative AI 180 instruction data including the material data 220 and an instruction to identify an event that has repeatedly occurred in the history of the target and generate future spatio-temporal data based on the identified event, rather than simply giving an instruction to generate future spatio-temporal data representing the future space-time of the target, it may be possible to generate highly credible future spatio-temporal data based on events that have repeatedly occurred in history.

[0110] The spatio-temporal data generation unit 108 may use the instruction and the material data 220 received by the reception unit 104 to generate a plurality of different future spatio-temporal data.

[0111] For example, the spatio-temporal data generation unit 108 generates future spatio-temporal data representing a space-time predicted for the target's future, future spatio-temporal data representing a space-time predicted for a good future for the target, and future spatio-temporal data representing a space-time predicted for a bad future for the target, based on the material data 220. The processing server 100 may input the material data 220 and instruction data including an instruction to generate, based on the material data 220, future spatio-temporal data representing a space-time predicted for the target's future, future spatio-temporal data representing a space-time predicted for a good future for the target, and future spatio-temporal data representing a space-time predicted for a bad future for the target, to the generation AI 180, and obtain three future spatio-temporal data from the generation AI 180.

[0112] Also, for example, the reception unit 104 receives input of a plurality of themes by the user 200, and the time-space data generation unit 108 generates a plurality of different future time-space data corresponding to each of the plurality of themes. As a specific example, when the reception unit 104 receives a first theme, a second theme, and a third theme, the time-space data generation unit 108 inputs the material data 220, and instruction data including an instruction to generate future time-space data representing a time-space in which the future of the target is predicted according to the first theme using the material data 220, future time-space data representing a time-space in which the future of the target is predicted according to the second theme using the material data 220, and future time-space data representing a time-space in which the future of the target is predicted according to the third theme using the material data 220, to the generative AI 180, and acquires three future time-space data from the generative AI 180. The theme may be text indicating the direction in which the target will move, or the like.

[0113] When the spatio-temporal data generation unit 108 generates a plurality of different future-time space-time data, the spatio-temporal data generation unit 108 may estimate an interest of the user 200 based on user-related information related to the user 200 who will experience any one of the plurality of different future-time space-time data, and may determine the future-time space-time data to be experienced by the user 200 from the plurality of future-time space-time data based on the estimated interest. For example, the spatio-temporal data generation unit 108 corrects the future-time space-time data so as to emphasize an area corresponding to the interest of the user 200 among a plurality of areas of space included in the future-time space-time data. The spatio-temporal data generation unit 108 may input the future-time space-time data and instruction data including an instruction to emphasize an area corresponding to the interest of the user 200 among a plurality of areas of space included in the future-time space-time data to the generation AI 180, and acquire the corrected future-time space-time data from the generation AI 180.

[0114] The spatio-temporal data generating unit 108 stores the generated target future-time-space data in the storage unit 102. The spatio-temporal data generating unit 108 may store the generated target future-time-space data in the DB server 190. The spatio-temporal data providing unit 110 provides the target future-time-space data generated by the spatio-temporal data generating unit 108 to the user 200 who instructed the generation of the future-time-space data. The spatio-temporal data providing unit 110 may transmit the target future-time-space data to the communication terminal 300. The spatio-temporal data providing unit 110 may transmit the target future-time-space data to the eyewear 400. The spatio-temporal data providing unit 110 may provide the target future-time-space data to another user 200 approved by the user 200 who instructed the generation of the future-time-space data. The spatio-temporal data providing unit 110 may manage the target future-time-space data so as not to provide it to another user 200 unless approval is obtained from the user 200 who instructed the generation of the future-time-space data. The spatio-temporal data providing unit 110 may publish the target future spatio-temporal data when approval is obtained from the user 200 who instructed the generation of the future spatio-temporal data. For example, the spatio-temporal data providing unit 110 publishes the target future spatio-temporal data in the metaverse when approval is obtained from the user 200 who instructed the generation of the future spatio-temporal data.

[0115] The AI ​​manager 112 manages the generative AI 180. The AI ​​manager 112 may update the generative AI 180. The AI ​​manager 112 may tune the generative AI 180.

[0116] The AI ​​management unit 112 may provide the past spatio-temporal data to the user 200, and update or tune the generative AI 180 according to the feedback from the user 200. For example, when past spatio-temporal data is generated using the same material data 210, the AI ​​management unit 112 uses feedback including indication of an erroneous part of the past spatio-temporal data generated using the material data 210 to update or tune the generative AI 180 so that erroneous generation does not occur again.

[0117] The AI ​​management unit 112 may provide the future space-time data to the user 200, and update or tune the generative AI 180 according to the content of feedback from the user 200. For example, when future space-time data is generated using material data 210 using feedback including an indication of an unlikely part of the future space-time data generated using the same material data 210, the AI ​​management unit 112 updates or tunes the generative AI 180 so that the indicated part is not included in the future space-time data.

[0118] The AI ​​management unit 112 may update or tune the generation system AI 180 so as to improve the generation accuracy when generating the target past spatio-temporal data using the target material data 220.

[0119] For example, when using the paper data included in the material data 220, the AI ​​management unit 112 updates or tunes the generative AI 180 by treating the impact factor of the academic journal in which the paper is published as index data. Here, the impact factor of the academic journal in which the paper is published may be described as the impact factor of the paper. In other words, the impact factor of the paper means the impact factor of the academic journal in which the paper is published. When multiple papers with different impact factors are input, the AI ​​management unit 112 may update, tune, or perform additional learning of the parameters of the generative AI 180 so as to generate spatio-temporal data with emphasis on the content of the paper with a high impact factor. For example, in the case of a paper with a high impact factor, the AI ​​management unit 112 updates, tunes, or performs additional learning of the generative AI 180 in such a way that the paper data is treated as data with high importance during learning. For example, in the case of a paper with a low impact factor, the AI ​​management unit 112 updates, tunes, or performs additional learning of the generative AI 180 in such a way that the paper data is treated as data with low importance during learning. The AI ​​management unit 112 may update, tune, or perform additional learning on the generative AI 180 using a dataset including multiple papers with different impact factors and spatiotemporal data generated with an emphasis on the content of papers with high associated impact factors.

[0120] For example, when using newspaper article data included in the material data 220, the AI ​​management unit 112 updates or tunes the generative AI 180 by treating the number of copies of the newspaper in which the newspaper article is published as index data. When a plurality of newspaper articles with different numbers of copies of the newspaper articles published are input, the AI ​​management unit 112 may update, tune, or perform additional learning of the parameters of the generative AI 180 so as to generate spatio-temporal data with emphasis on the content of the newspaper article with a large number of copies. For example, in the case of newspaper articles published in newspapers with a large number of copies, the AI ​​management unit 112 updates, tunes, or performs additional learning of the generative AI 180 in such a way that the newspaper article data is treated as data with a high degree of importance during learning. For example, in the case of newspaper articles published in newspapers with a small number of copies, the AI ​​management unit 112 updates, tunes, or performs additional learning of the generative AI 180 in such a way that the newspaper article data is treated as data with a low degree of importance during learning. The AI ​​management unit 112 may update, tune, or perform additional learning on the generative AI 180 using a dataset including multiple newspaper articles with different circulations in the newspapers in which they are published, and spatiotemporal data generated with an emphasis on the content of newspaper articles with large circulations in the newspapers in which they are published that are linked to the newspaper articles.

[0121] It may be possible to select which index is to be adopted as a weighting, which is included in the material data 220. For example, an administrator of the processing server 100 or the like selects an index to be adopted as a weighting from among a plurality of data included in the material data 220, and the AI ​​management unit 112 uses the selected index to update or tune the generative AI 180.

[0122] 7 is a schematic diagram showing an example of a processing flow by the processing server 100. Here, an example of a processing flow in which the processing server 100 repeatedly generates past spatio-temporal data until approval by the user 200 is obtained will be described.

[0123] In step (sometimes abbreviated to S) 102, the reception unit 104 receives material data provided by the user 200 and an instruction to generate past time-space data. The reception unit 104 stores the received material data and the instruction in the storage unit 102. The reception unit 104 may also store the received material data and the instruction in the DB server 190.

[0124] In S104, the spatio-temporal data generation unit 108 generates past spatio-temporal data from the material data using the generative AI 180. In S106, the spatio-temporal data provision unit 110 provides the past spatio-temporal data generated by the spatio-temporal data generation unit 108 in S104 to the user 200. If OK is obtained from the user 200, the process proceeds to S110.

[0125] If the result is NG, the process returns to S102. In this case, the spatio-temporal data generation unit 108 may request additional material data from the user 200. In response to the reception unit 104 receiving the additional material data, the spatio-temporal data generation unit 108 uses the generative AI 180 to generate past spatio-temporal data using the material data acquired up to that point and the additional material data.

[0126] In S110, the spatio-temporal data generation unit 108 registers the past spatio-temporal data approved by the user 200 in the storage unit 102. The spatio-temporal data generation unit 108 may also register the past spatio-temporal data in the DB server 190.

[0127] 8 is a schematic diagram showing an example of a processing flow by the processing server 100. Here, an example of a processing flow will be described in which the processing server 100 generates spatial data for each of a plurality of time points while confirming with the user 200, and generates past spatio-temporal data using the plurality of spatial data.

[0128] In S202, the reception unit 104 receives the material data and an instruction to generate past time-space data provided by the user 200. The reception unit 104 stores the received material data and the instruction in the storage unit 102. The reception unit 104 may also store the received material data and the instruction in the DB server 190.

[0129] In S204, the material data acquisition unit 106 acquires a portion of the material data. The material data acquisition unit 106 acquires a portion of the material data corresponding to one of the multiple time points.

[0130] In S206, the spatio-temporal data generation unit 108 generates spatial data corresponding to one point in time from the material data acquired by the material data acquisition unit 106 in S204, using the generative AI 180. In S208, the spatio-temporal data provision unit 110 provides the user 200 with the spatial data generated by the spatio-temporal data generation unit 108 in S206.

[0131] If the spatial data needs to be corrected, proceed to S212, and if not, proceed to S214. A case where correction is necessary may be a case where a correction instruction is received from the user 200. The correction instruction may include an area to be corrected among multiple areas included in the spatial data, and correction content indicating how to correct the area. In S212, the spatiotemporal data generation unit 108 corrects the spatial data according to the correction instruction. The spatiotemporal data generation unit 108 may input instruction data including the spatial data and the correction instruction to the generation system AI 180 to obtain the corrected spatial data.

[0132] If the generation of spatial data for all of the multiple time points for which images are included in the material data received in S202 has been completed (YES in S214), the process proceeds to S216, and if not completed, the process proceeds to S204. In S204, the material data acquisition unit 106 acquires a portion of the material data corresponding to the next time point among the multiple time points.

[0133] In S216, the spatio-temporal data generating unit 108 determines whether or not the spatial data needs to be complemented. The spatio-temporal data generating unit 108 may determine whether or not the spatial data is sufficient when generating past spatio-temporal data. For example, the spatio-temporal data generating unit 108 determines that the spatial data is sufficient if spatial data exists for all of the preset time intervals, and otherwise determines that the spatial data is insufficient. The preset time interval may be set in any unit such as minutes, hours, or days. If it is determined that complementation is necessary, the process proceeds to S218, and if it is not determined that complementation is necessary, the process proceeds to S220.

[0134] In S218, the spatio-temporal data generating unit 108 complements the spatial data using the generated spatial data at the multiple time points. For example, when the time interval between the spatial data at consecutive time points is greater than a predetermined threshold, the spatio-temporal data generating unit 108 generates the spatial data at a time point between the consecutive time points using the spatial data at the consecutive time points.

[0135] In S220, the spatio-temporal data generation unit 108 generates past spatio-temporal data using the generated spatial data and, if there is spatial data generated in S218, the spatial data. The spatio-temporal data generation unit 108 may input instruction data including the generated spatial data, the complemented spatial data, and an instruction to generate past spatio-temporal data using these spatial data to the generation AI 180, and obtain the past spatio-temporal data from the generation AI 180.

[0136] 9 is a schematic diagram showing an example of a hardware configuration of a computer 1200 functioning as the processing server 100. A program installed on the computer 1200 can cause the computer 1200 to function as one or more "parts" of an apparatus according to the present embodiment, or cause the computer 1200 to execute operations associated with the apparatus according to the present embodiment or one or more "parts" thereof, and / or cause the computer 1200 to execute a process according to the present embodiment or steps of the process. Such a program can be executed by the CPU 1212 to cause the computer 1200 to execute specific operations associated with some or all of the blocks of the flowcharts and block diagrams described herein.

[0137] The computer 1200 according to this embodiment includes a CPU 1212, a RAM 1214, and a graphics controller 1216, which are connected to each other by a host controller 1210. The computer 1200 also includes input / output units such as a communication interface 1222, a storage device 1224, a DVD drive, and an IC card drive, which are connected to the host controller 1210 via an input / output controller 1220. The DVD drive may be a DVD-ROM drive, a DVD-RAM drive, etc. The storage device 1224 may be a hard disk drive, a solid state drive, etc. The computer 1200 also includes a legacy input / output unit such as a ROM 1230 and a keyboard, which are connected to the input / output controller 1220 via an input / output chip 1240.

[0138] The CPU 1212 operates according to a program stored in the ROM 1230 and the RAM 1214, thereby controlling each unit. The graphic controller 1216 acquires image data generated by the CPU 1212 into a frame buffer or the like provided in the RAM 1214 or into itself, and causes the image data to be displayed on the display device 1218.

[0139] The communication interface 1222 communicates with other electronic devices via a network. The storage device 1224 stores programs and data used by the CPU 1212 in the computer 1200. The DVD drive reads programs or data from a DVD-ROM or the like and provides them to the storage device 1224. The IC card drive reads programs and data from an IC card and / or writes programs and data to an IC card.

[0140] The ROM 1230 stores therein a boot program or the like that is executed by the computer 1200 upon activation, and / or a program that depends on the hardware of the computer 1200. The input / output chip 1240 may also connect various input / output units to the input / output controller 1220 via a USB port, a parallel port, a serial port, a keyboard port, a mouse port, and the like.

[0141] The programs are provided by a computer-readable storage medium such as a DVD-ROM or an IC card. The programs are read from the computer-readable storage medium, installed in the storage device 1224, the RAM 1214, or the ROM 1230, which are also examples of computer-readable storage media, and executed by the CPU 1212. Information processing described in these programs is read by the computer 1200, and brings about cooperation between the programs and the various types of hardware resources described above. An apparatus or method may be constructed by implementing operations or processing of information according to the use of the computer 1200.

[0142] For example, when communication is performed between the computer 1200 and an external device, the CPU 1212 may execute a communication program loaded in the RAM 1214 and instruct the communication interface 1222 to perform communication processing based on the processing described in the communication program. Under the control of the CPU 1212, the communication interface 1222 reads transmission data stored in a transmission buffer area provided in the RAM 1214, the storage device 1224, a DVD-ROM, or a recording medium such as an IC card, and transmits the read transmission data to the network, or writes reception data received from the network to a reception buffer area or the like provided on the recording medium.

[0143] Furthermore, the CPU 1212 may cause all or a necessary portion of a file or database stored in an external recording medium such as the storage device 1224, a DVD drive (DVD-ROM), an IC card, etc. to be read into the RAM 1214, and perform various types of processing on the data on the RAM 1214. The CPU 1212 may then write back the processed data to the external recording medium.

[0144] Various types of information, such as various types of programs, data, tables, and databases, may be stored in the recording medium and undergo information processing. The CPU 1212 may perform various types of processing on the data read from the RAM 1214, including various types of operations, information processing, conditional judgment, conditional branching, unconditional branching, information search / replacement, etc., as described throughout this disclosure and specified by the instruction sequence of the program, and write back the results to the RAM 1214. The CPU 1212 may also search for information in a file, database, etc. in the recording medium. For example, when a plurality of entries each having an attribute value of a first attribute associated with an attribute value of a second attribute are stored in the recording medium, the CPU 1212 may search for an entry whose attribute value of the first attribute matches a specified condition from among the plurality of entries, read the attribute value of the second attribute stored in the entry, and thereby obtain the attribute value of the second attribute associated with the first attribute that satisfies a predetermined condition.

[0145] The above-described programs or software modules may be stored in a computer-readable storage medium on the computer 1200 or in the vicinity of the computer 1200. In addition, a recording medium such as a hard disk or a RAM provided in a server system connected to a dedicated communication network or the Internet can be used as a computer-readable storage medium, thereby providing the programs to the computer 1200 via the network.

[0146] The blocks in the flowcharts and block diagrams in the present embodiment may represent stages of a process in which an operation is performed or "parts" of an apparatus responsible for performing the operation. Particular stages and "parts" may be implemented by dedicated circuitry, programmable circuitry provided with computer-readable instructions stored on a computer-readable storage medium, and / or a processor provided with computer-readable instructions stored on a computer-readable storage medium. The dedicated circuitry may include digital and / or analog hardware circuits, and may include integrated circuits (ICs) and / or discrete circuits. The programmable circuitry may include reconfigurable hardware circuits, such as, for example, field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), and the like, including AND, OR, XOR, NAND, NOR, and other logical operations, flip-flops, registers, and memory elements.

[0147] A computer-readable storage medium may include any tangible device capable of storing instructions that are executed by a suitable device, such that a computer-readable storage medium having instructions stored thereon comprises an article of manufacture that includes instructions that can be executed to create means for performing the operations specified in the flowcharts or block diagrams. Examples of computer-readable storage media may include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, and the like. More specific examples of computer-readable storage media may include floppy disks, diskettes, hard disks, random access memories (RAMs), read-only memories (ROMs), erasable programmable read-only memories (EPROMs or flash memories), electrically erasable programmable read-only memories (EEPROMs), static random access memories (SRAMs), compact disk read-only memories (CD-ROMs), digital versatile disks (DVDs), Blu-ray disks, memory sticks, integrated circuit cards, and the like.

[0148] The computer readable instructions may include either assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk®, JAVA®, C++, etc., and conventional procedural programming languages ​​such as the “C” programming language or similar programming languages.

[0149] The computer-readable instructions may be provided to a general-purpose computer, a special-purpose computer, or a processor of another programmable data processing device, or a programmable circuit, locally or over a wide area network (WAN) such as a local area network (LAN), the Internet, etc., so that the processor of the programmable data processing device, such as a computer, or the programmable circuit executes the computer-readable instructions to generate means for performing the operations specified in the flowchart or block diagram. Here, the computer may be a PC (personal computer), a tablet computer, a smartphone, a workstation, a server computer, a general-purpose computer, or a special-purpose computer, or may be a computer system in which multiple computers are connected. Such a computer system in which multiple computers are connected is also called a distributed computing system, and is a computer in the broad sense. In a distributed computing system, multiple computers collectively execute a program by each executing a part of the program and transferring data during program execution between the computers as necessary.

[0150] Examples of processors include computer processors, central processing units, processing units, microprocessors, digital signal processors, controllers, microcontrollers, etc. A computer may have one processor or multiple processors. In a multiprocessor system with multiple processors, each processor executes a part of a program, and the multiple processors collectively execute a program by passing data during program execution between the processors as necessary. For example, in executing multitasks, each of the multiple processors may execute a part of each task in small chunks by switching tasks for each time slice. In this case, which part of a program each processor executes changes dynamically. Which part of a program each of the multiple processors executes may be statically determined by programming that takes the multiprocessor into consideration.

[0151] Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It is clear to those skilled in the art that various modifications and improvements can be made to the above embodiments. It is clear from the description of the claims that such modifications and improvements can also be included in the technical scope of the present invention.

[0152] It should be noted that the order of execution of each process, such as operations, procedures, steps, and stages, in the devices, systems, programs, and methods shown in the claims, specifications, and drawings is not specifically stated as "before" or "prior to," and that the process may be performed in any order unless the output of a previous process is used in a later process. Even if the operational flow in the claims, specifications, and drawings is described using "first," "next," etc. for convenience, it does not mean that the process must be performed in this order. [Explanation of symbols]

[0153] 10 processing system, 50 network, 100 processing server, 102 memory unit, 104 reception unit, 106 material data acquisition unit, 108 spatiotemporal data generation unit, 110 spatiotemporal data provision unit, 112 AI management unit, 180 generative AI, 190 DB server, 200 user, 210 material data, 220 material data, 300 communication terminal, 400 eyewear, 502 spatial data, 504 spatial data, 508 spatial data, 512 spatial data, 514 spatial data, 518 spatial data, 1200 computer, 1210 host controller, 1212 CPU, 1214 RAM, 1216 graphic controller, 1218 display device, 1220 input / output controller, 1222 communication interface, 1224 storage device, 1230 ROM, 1240 input / output chip

Claims

1. a material data acquisition unit that acquires material data including captured images captured at a location where the user was located at each of a plurality of time points provided by the user; a time-space data generating unit that generates past time-space data representing the user's past time-space, the past time-space data being composed of spatial data in which the user was located at a plurality of time points, based on the material data; Equipped with The material data further includes diary data including the diaries at the multiple points in time, the spatio-temporal data generation unit generates spatial data representing the situation of a point where the user was located at each of the multiple time points using the multiple captured images included in the material data, estimates the user's level of interest in multiple areas within the spatial data for each of the multiple spatial data using the diary data, corrects the spatial data according to the estimated level of interest, and then generates instruction data including an instruction to generate the past spatio-temporal data using the multiple spatial data, and generates the past spatio-temporal data by inputting the instruction data to a generation AI. Processing server.

2. 2. The processing server of claim 1, wherein the spatiotemporal data generation unit generates spatial data representing the points where the user was located at each of the multiple time points using the multiple captured images included in the material data, generates spatial data for a time point between the multiple time points using the spatial data for consecutive time points among the multiple time points, generates the instruction data including an instruction to generate the past spatiotemporal data using the multiple generated spatial data, and inputs the instruction data to the generation AI.

3. The material data further includes map data including information on the location, The processing server according to claim 1, wherein the spatiotemporal data generation unit generates the spatial data representing the points where the user was located at each of the multiple points in time using the multiple captured images and the map data included in the material data, generates the instruction data including an instruction to generate the past spatiotemporal data using the generated multiple spatial data, and inputs the instruction data to the generation AI.

4. A material data acquisition unit that acquires material data including captured images captured at a location where the user was located at each of a plurality of time points provided by the user; a time-space data generating unit that generates past time-space data representing the user's past time-space, the past time-space data being composed of spatial data in which the user was located at a plurality of time points, based on the material data; Equipped with the spatio-temporal data generation unit uses the material data of a plurality of users including the user and other users to estimate interests and degrees of interest of each of the plurality of users, and generates past spatio-temporal data of the user using the estimated interests and degrees of interest; Processing server.

5. The processing server described in claim 4, wherein the spatiotemporal data generation unit generates spatial data representing the situation of the point where each of the users was located at the multiple time points using the multiple captured images included in the material data of the multiple users, and for each of the multiple generated spatial data, generates instruction data including instructions to highlight an area among multiple areas included in the spatial data that was of high interest to any of the multiple users, make it possible to confirm which user was of high interest, and instructions to generate the past spatiotemporal data using the multiple spatial data, and generates the past spatiotemporal data by inputting the instruction data into a generation AI.

6. The processing server described in claim 4, wherein the spatiotemporal data generation unit generates spatial data representing the situation of the location where each of the users was located at the multiple time points using the multiple captured images included in the material data of the multiple users, and generates instruction data for each of the multiple generated spatial data, for a multiple area included in the spatial data in which any of the multiple users had low interest, to make the area less noticeable and to make it possible to confirm which user had low interest, and an instruction to generate the past spatiotemporal data using the multiple spatial data, and generates the past spatiotemporal data by inputting the instruction data into a generation AI.

7. The processing server according to claim 1 , wherein the spatiotemporal data generation unit estimates the user's interests based on the material data, and generates future spatiotemporal data representing the user's future space-time based on the estimated interests.

8. a spatio-temporal data providing unit that provides the future spatio-temporal data of the user generated by the spatio-temporal data generating unit to other users approved by the user; The processing server of claim 7 .

9. a spatio-temporal data providing unit that provides the past spatio-temporal data of the user generated by the spatio-temporal data generating unit to other users approved by the user; The processing server according to claim 1 , comprising:

10. The material data acquisition unit acquires object material data including images at a plurality of times related to a specific object and at least one of a paper, a research document, a newspaper article, and a minutes related to the object; The processing server according to claim 1 , wherein the spatio-temporal data generation unit generates, based on the object material data, object past spatio-temporal data that is composed of spatial data of the object at a plurality of points in time and represents a past space-time of the object.

11. A material data acquisition unit that acquires material data including images of a specific object at a plurality of time points and at least one of a paper, a research document, a newspaper article, and a meeting minutes related to the object; a spatio-temporal data generating unit that generates past spatio-temporal data representing a past space-time of the object, the past space-time data being composed of spatial data of the object at a plurality of points in time, based on the raw data; Equipped with the spatio-temporal data generation unit identifies events that have repeatedly occurred in the history of the target based on at least one of the papers, the research materials, the newspaper articles, and the minutes included in the material data, and generates future spatio-temporal data representing a future space-time of the target based on the identified events; Processing server.

12. The processing server according to claim 11, wherein the spatio-temporal data generation unit generates the spatial data for each of the multiple points in time using the multiple images included in the material data and at least one of the papers, the research materials, the newspaper articles, and the minutes included in the material data, generates instruction data including an instruction to generate the past spatio-temporal data using the generated multiple spatial data, and generates the past spatio-temporal data by inputting the instruction data to a generation AI.

13. The processing server according to claim 12, wherein the spatio-temporal data generation unit generates the spatial data for each of the multiple points in time using the multiple images included in the material data and at least one of the papers, the research materials, the newspaper articles, and the minutes included in the material data, generates spatial data for a point in time between consecutive points in time among the multiple points in time using the spatial data for the consecutive points in time, generates the instruction data including an instruction to generate the past spatio-temporal data using the generated multiple spatial data, and inputs the instruction data to the generation AI.

14. The processing server according to any one of claims 11 to 13, wherein the time-space data generation unit generates a plurality of the future-time-space data, estimates the user's interests based on user-related information related to a user who experiences any of the plurality of future-time-space data, and determines the future-time-space data to be experienced by the user from the plurality of future-time-space data based on the estimated interests.

15. The processing server according to any one of claims 11 to 13, wherein the time-space data generation unit estimates the user's interests based on user-related information related to the user who experiences the generated past time-space data, and corrects the past time-space data based on the estimated interests.

16. A material data acquisition unit that acquires material data including images of a specific object at a plurality of time points, paper data of a paper that discusses the history of the object, and an impact factor of an academic journal in which the paper is published; a spatio-temporal data generating unit that generates spatial data of the object at a plurality of time points using a plurality of the images included in the raw data, the paper data, and the impact factor, and generates past spatio-temporal data representing the past space-time of the object, which is composed of the plurality of spatial data, using the plurality of the generated spatial data; Equipped with The spatio-temporal data generation unit instructs the generation AI to generate the past spatio-temporal data by placing more importance on the content of paper data of papers published in academic journals with a higher impact factor than the content of paper data of papers published in academic journals with a lower impact factor. Processing server.

17. A material data acquisition unit that acquires material data including images of a specific object at a plurality of time points, newspaper article data including newspaper articles including contents about the object, and the number of copies of the newspaper in which the newspaper articles are published; a spatio-temporal data generating unit that generates spatial data of the object at a plurality of points in time using the plurality of images included in the material data, the newspaper article data, and the circulation number, and generates past spatio-temporal data representing a past space-time of the object, the past spatio-temporal data being configured from the plurality of generated spatial data; Equipped with The spatio-temporal data generation unit instructs the generation AI to generate the past spatio-temporal data by placing more importance on the contents of newspaper articles published in newspapers with a larger circulation than the contents of newspaper articles published in newspapers with a smaller circulation. Processing server.

18. 1. A computer-implemented information processing method, comprising: a material data acquisition step of acquiring material data including captured images captured at a location where the user was located at each of a plurality of time points provided by the user; a spatio-temporal data generating step of generating past spatio-temporal data representing the past space-time of the user, the past space-time data being composed of spatial data in which the user was located at a plurality of time points, based on the material data; Equipped with The material data further includes diary data including the diaries at the multiple points in time, The spatio-temporal data generation step generates spatial data representing the situation of a location where the user was located at each of the multiple time points using the multiple captured images included in the material data, estimates the user's level of interest in multiple areas within the spatial data for each of the multiple spatial data using the diary data, corrects the spatial data according to the estimated level of interest, and then generates instruction data including an instruction to generate the past spatio-temporal data using the multiple spatial data, and generates the past spatio-temporal data by inputting the instruction data to a generation AI. Processing methods.

19. A computer-implemented information processing method, comprising: a material data acquisition step of acquiring material data including captured images captured at a location where the user was located at each of a plurality of time points provided by the user; a spatio-temporal data generating step of generating past spatio-temporal data representing the past space-time of the user, the past space-time data being composed of spatial data in which the user was located at a plurality of time points, based on the material data; Equipped with The spatio-temporal data generating step estimates interests and degrees of interest of each of the plurality of users using the material data of the plurality of users including the user and other users, and generates past spatio-temporal data of the user using the estimated interests and degrees of interest. Processing methods.

20. The processing method described in claim 19, wherein the spatiotemporal data generation step generates spatial data representing the situation of a point where each of the users was located at the multiple time points using the multiple captured images included in the material data of the multiple users, and generates instruction data for each of the multiple generated spatial data, the instruction data including an instruction to highlight an area among multiple areas included in the spatial data that was of high interest to any of the multiple users and to make it possible to confirm which user was of high interest, and an instruction to generate the past spatiotemporal data using the multiple spatial data, and generates the past spatiotemporal data by inputting the instruction data into a generation AI.

21. The processing method described in claim 19, wherein the spatiotemporal data generation step generates spatial data representing the situation of a location where each of the users was located at the multiple time points using the multiple captured images included in the material data of the multiple users, and generates instruction data for each of the multiple generated spatial data, for a multiple area included in the spatial data in which any of the multiple users had low interest, to make the area less noticeable and to make it possible to confirm which user had low interest, and an instruction to generate the past spatiotemporal data using the multiple spatial data, and generates the past spatiotemporal data by inputting the instruction data into a generation AI.

22. 1. A computer-implemented information processing method, comprising: A material data acquisition step of acquiring material data including images at a plurality of times related to a specific subject and at least one of papers, research materials, newspaper articles, and minutes related to the subject; a spatio-temporal data generation step of generating past spatio-temporal data representing the past space-time of the object, the past space-time data being composed of spatial data of the object at a plurality of points in time, based on the raw data; Equipped with The spatio-temporal data generation step includes identifying repeated events in the history of the object based on at least one of the papers, the research materials, the newspaper articles, and the minutes included in the material data, and generating future spatio-temporal data representing the future space-time of the object based on the identified events. Processing methods.

23. A computer-implemented information processing method, comprising: A material data acquisition step of acquiring material data including images of a plurality of time points related to a particular subject, paper data of a paper discussing the history of the subject, and an impact factor of an academic journal in which the paper is published; a spatio-temporal data generation step of generating spatial data of the object at multiple time points using the multiple images included in the raw data, the paper data, and the impact factor, and generating past spatio-temporal data representing the past space-time of the object, which is composed of the multiple spatial data, using the multiple spatial data generated; Equipped with The spatio-temporal data generation step instructs the generation AI to generate the past spatio-temporal data by placing more importance on the content of paper data of papers published in academic journals with a higher impact factor than on the content of paper data of papers published in academic journals with a lower impact factor. Processing methods.

24. A computer-implemented information processing method, comprising: A material data acquisition step of acquiring material data including images of a specific object at a plurality of time points, newspaper article data including newspaper articles including contents about the object, and the number of copies of newspapers in which the newspaper articles are published; a spatio-temporal data generation step of generating spatial data of the object at a plurality of points in time using the plurality of images included in the material data, the newspaper article data, and the circulation number, and generating past spatio-temporal data representing the past space-time of the object, the past spatio-temporal data being composed of the generated plurality of spatial data; Equipped with The spatio-temporal data generation step instructs the generation AI to generate the past spatio-temporal data by placing more importance on the contents of newspaper articles published in newspapers with a larger circulation than the contents of newspaper articles published in newspapers with a smaller circulation. Processing methods.

25. A program for causing a computer to execute the processing method according to any one of claims 18 to 24.

Images