Processing server, processing method, and program

Abstract

A processing server, a processing method, and a program are provided. [Solution] In one embodiment of a processing system 10, a processing server includes a material data acquisition unit that acquires material data provided by a user, including captured images captured at locations where the user was located at each of a plurality of points in time, and a spatio-temporal data generation unit that generates, based on the material data, past spatio-temporal data representing the user's past space-time, the past spatio-temporal data being composed of spatial data of the user's locations at the multiple points in time. According to one embodiment, the processing server includes a material data acquisition unit that acquires material data including images related to a specific object at each of a plurality of points in time, and at least one of papers, research materials, newspaper articles, and minutes related to the object, and a spatio-temporal data generation unit that generates, based on the material data, past spatio-temporal data representing the object's past space-time, the past spatio-temporal data being composed of spatial data of the object at the multiple points in time.

Description

[Technical Field]

[0001] The present invention relates to a processing server, a processing method, and a program. [Background technology]

[0002] Patent Document 1 describes an image generation AI that generates an AI (Artificial Intelligence) image according to input text information or the like. [Prior art document] [Patent documents] [Patent Document 1] JP 2024-043680 A Summary of the Invention [Means for solving the problem]

[0003] According to one embodiment of the present invention, there is provided a processing server. The processing server may include a material data acquisition unit that acquires material data provided by a user, the material data including captured images captured at locations where the user was located at each of a plurality of time points. The processing server may also include a spatio-temporal data generation unit that generates, based on the material data, past spatio-temporal data representing the user's past space-time, the past space-time data being composed of spatial data of where the user was located at the plurality of time points.

[0004] In the processing server, the spatiotemporal data generation unit may generate spatial data representing the situation of the location where the user was located at each of the multiple points in time using the multiple captured images included in the material data, generate instruction data including instructions to generate the spatiotemporal data using the multiple generated spatial data, and generate the past spatiotemporal data by inputting the instruction data to a generation AI.

[0005] In the processing server, the spatiotemporal data generation unit may generate spatial data representing the points where the user was located at each of the multiple time points using the multiple captured images included in the material data, generate spatial data for a time point between consecutive time points using the spatial data for consecutive time points among the multiple time points, generate the instruction data including an instruction to generate the past spatiotemporal data using the generated multiple spatial data, and input the instruction data to the generation AI.

[0006] In any of the processing servers, the material data may further include map data including information on the locations, and the spatiotemporal data generation unit may generate the spatial data representing the locations where the user was located at each of the multiple points in time using the multiple captured images and the map data included in the material data, generate the instruction data including instructions to generate the past spatiotemporal data using the multiple spatial data generated, and input the instruction data to the generation AI.

[0007] In any of the processing servers, the material data may further include diary data including diaries at the multiple points in time, and the spatiotemporal data generation unit may use the multiple captured images included in the material data to generate spatial data representing the points where the user was located at each of the multiple points in time, use the diary data to estimate the user's level of interest in multiple areas within the spatial data for each of the multiple spatial data, correct the spatial data according to the estimated level of interest, and then generate the instruction data including instructions to generate the past spatiotemporal data using the multiple spatial data, and input the instruction data to the generation AI.

[0008] In any of the processing servers, the spatio-temporal data generation unit may estimate interests of the user based on the material data, and generate future spatio-temporal data representing a future space-time of the user based on the estimated interests. The processing server may include a spatio-temporal data providing unit that provides the future spatio-temporal data of the user generated by the spatio-temporal data generation unit to other users approved by the user.

[0009] Any one of the processing servers may include a spatiotemporal data providing unit that provides the past spatiotemporal data of the user generated by the spatiotemporal data generating unit to other users approved by the user.

[0010] In any of the processing servers, the material data acquisition unit may acquire object material data including images of a specific object at each of a plurality of points in time and at least one of papers, research materials, newspaper articles, and minutes related to the object, and the spatiotemporal data generation unit may generate object past spatiotemporal data representing the past space-time of the object, which is composed of spatial data of the object at a plurality of points in time, based on the object material data.

[0011] According to one embodiment of the present invention, there is provided a processing server. The processing server may include a material data acquisition unit that acquires material data including images of a specific object at a plurality of points in time and at least one of papers, research materials, newspaper articles, and minutes related to the object. The processing server may also include a spatio-temporal data generation unit that generates, based on the material data, past spatio-temporal data representing the past space-time of the object, the past spatio-temporal data being composed of spatial data of the object at a plurality of points in time.

[0012] In the processing server, the spatiotemporal data generation unit may generate the spatial data for each of the multiple points in time using the multiple images included in the material data and at least one of the papers, research materials, newspaper articles, and minutes included in the material data, generate instruction data including instructions to generate the past spatiotemporal data using the generated multiple spatial data, and generate the past spatiotemporal data by inputting the instruction data to a generation AI.

[0013] In the processing server, the spatiotemporal data generation unit may generate the spatial data for each of the plurality of points in time using the plurality of images included in the material data and at least one of the papers, the research materials, the newspaper articles, and the minutes included in the material data, generate spatial data for a point in time between consecutive points in time among the plurality of points in time using the spatial data for the consecutive points in time, generate the instruction data including an instruction to generate the past spatiotemporal data using the generated plurality of spatial data, and input the instruction data to the generation AI.

[0014] In the processing server, the spatiotemporal data generation unit may identify events that have repeatedly occurred in the history of the subject based on at least one of the papers, the research materials, the newspaper articles, and the minutes included in the material data, and generate future spatiotemporal data that represents the future space-time of the subject based on the identified events.

[0015] In the processing server, the spatio-temporal data generation unit may generate a plurality of pieces of future spatio-temporal data, estimate the interests of the user based on user-related information related to the user who will experience any of the plurality of future spatio-temporal data, and determine the future spatio-temporal data to be experienced by the user from the plurality of future spatio-temporal data based on the estimated interests.

[0016] In the processing server, the spatiotemporal data generation unit may estimate the user's interests based on user-related information related to the user who experiences the generated past spatiotemporal data, and correct the past spatiotemporal data based on the estimated interests.

[0017] According to one embodiment of the present invention, there is provided a processing method executed by a computer. The processing method may include a material data acquisition step of acquiring material data provided by a user, the material data including captured images captured at locations where the user was located at each of a plurality of time points. The processing method may also include a spatio-temporal data generation step of generating, based on the material data, past spatio-temporal data representing the user's past space-time, the past spatio-temporal data being composed of spatial data where the user was located at the plurality of time points.

[0018] According to one embodiment of the present invention, a processing method executed by a computer may be provided. The processing method may include a material data acquisition step of acquiring material data including images of a specific object at a plurality of time points and at least one of papers, research materials, newspaper articles, and minutes related to the object. The processing method may also include a spatio-temporal data generation step of generating, based on the material data, past spatio-temporal data representing the past space-time of the object, the past spatio-temporal data being composed of spatial data of the object at a plurality of time points.

[0019] According to one embodiment of the present invention, there is provided a program for causing a computer to execute the processing method.

[0020] The above summary of the invention does not list all of the necessary features of the present invention, and subcombinations of these features may also constitute inventions. [Brief explanation of the drawings]

[0021] [Figure 1] An example of a processing system 10 is shown schematically. [Figure 2]FIG. 2 is an explanatory diagram for explaining the processing content by the processing server 100. [Figure 3] FIG. 2 is an explanatory diagram for explaining the processing content by the processing server 100. [Figure 4] FIG. 2 is an explanatory diagram for explaining the processing content by the processing server 100. [Figure 5] FIG. 2 is an explanatory diagram for explaining the processing content by the processing server 100. [Figure 6] 1 shows an example of a functional configuration of a processing server 100. [Figure 7] 10 shows an example of a processing flow by the processing server 100. [Figure 8] 10 shows an example of a processing flow by the processing server 100. [Figure 9] An example of the hardware configuration of a computer 1200 that functions as the processing server 100 is shown in schematic form. DETAILED DESCRIPTION OF THE INVENTION

[0022] The present invention will be described below through embodiments of the invention, but the following embodiments do not limit the scope of the invention according to the claims. Furthermore, not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention.

[0023] It is considered very useful to recreate a specific user's life log using 3D video to allow the user to experience it, or to allow other users to experience it with the user's approval. It is also considered very useful to predict the future based on the user's past history and generate 3D video representing the predicted future to allow the user to experience it, or to allow other users to experience it with the user's approval. It is also considered very useful to recreate the history of a specific region, corporation, etc. using 3D video to allow the user to experience it, or to predict the future based on the past history of a specific region, corporation, etc. and generate 3D video representing the predicted future to allow the user to experience it. The processing system 10 according to this embodiment provides such experiences. The processing system 10 provides such experiences using, for example, XR (Cross Reality). The processing server 100 may generate these 3D videos as content for MR (Mixed Reality) devices. The processing server 100 may generate these 3D videos as content for AR (Augmented Reality) devices. The processing server 100 may generate these 3D videos as content for a VR (Virtual Reality) device.

[0024] 1 schematically illustrates an example of a processing system 10. The processing system 10 includes a processing server 100. The processing system 10 includes a generative AI 180. The processing system 10 includes a DB server 190.

[0025] The processing server 100 may have a function to generate spatiotemporal data representing past and future space-time. The spatiotemporal data may be 3D video. The processing server 100 may have a function to generate spatiotemporal data for individuals. The processing server 100 may have a function to generate spatiotemporal data for corporations.

[0026] The personalized spatiotemporal data generation function will now be described. The processing server 100 generates past spatiotemporal data representing the past space-time of the user 200, for example, based on material data provided by the user 200. The processing server 100 may receive the material data from a communication terminal 300 of the user 200. The communication terminal 300 may be any terminal capable of communication, such as a smartphone, a tablet terminal, or a PC (Personal Computer). The processing server 100 may receive the material data from eyewear 400 of the user 200.

[0027] The material data includes captured images taken at locations where the user was previously located. The material data may include captured images at each of a plurality of points in time. The captured images may be still images. The captured images may be videos. The material data may include map data including information on locations where the captured images were taken. The material data may include diary data including diaries from a plurality of points in time when the captured images were taken. The diary data may include, for each of a plurality of dates and times, the date and time, the location, and the content of the diary written by the user 200. The material data may include data other than these.

[0028] The past spatio-temporal data may be a 3D video representing a past experience of the user 200. The past spatio-temporal data may be content for an XR device. For example, the past spatio-temporal data is content for an MR device. For example, the past spatio-temporal data is content for an AR device. For example, the past spatio-temporal data is content for a VR device.

[0029] The processing server 100 may generate past-space-time data using the generative AI 180. The generative AI 180 may be any type capable of accepting prompt input including still images, videos, and text instructions and outputting 3D videos. The text may be input via an input device such as a keyboard or touch panel, by voice input, or by other input methods. The generative AI 180 may be a so-called video generation AI. The processing server 100 may use an external generative AI 180. The processing server 100 may communicate with the generative AI 180 via a network 50. The network 50 may include the Internet. The network 50 may include a cloud. The network 50 may include a mobile communication network. The mobile communication network may comply with any of the 5th Generation (5G) communication standard, the Long Term Evolution (LTE) communication standard, and the 6th Generation (6G) or later communication standard. The network 50 may include a local area network (LAN). The processing server 100 may also include the generation AI 180. The processing server 100 may include the generation AI 180 generated by another device, or may include the generation AI 180 generated by the processing server 100 itself.

[0030] The processing server 100 may input prompt information including material data and an instruction to generate past spatio-temporal data using the material data to the generation AI 180, and acquire the past spatio-temporal data from the generation AI 180. The processing server 100 may store the material data provided by the user 200 itself. The processing server 100 may store the material data provided by the user 200 in the DB server 190. The processing server 100 may store the generated past spatio-temporal data itself. The processing server 100 may store the generated past spatio-temporal data in the DB server 190.

[0031] The processing server 100 provides the past spatio-temporal data of the user 200 to the user 200. The processing server 100 may transmit the past spatio-temporal data of the user 200 to the communication terminal 300. The processing server 100 may transmit the past spatio-temporal data of the user 200 to the eyewear 400. The user 200 may view the past spatio-temporal data using the communication terminal 300. The user 200 may view the past spatio-temporal data using the eyewear 400.

[0032] The processing server 100 generates future spatio-temporal data representing the future space-time of the user 200, for example, based on material data provided by the user 200. The future spatio-temporal data may be a 3D video that predicts the future of the user 200. The future spatio-temporal data may be a 3D video that represents the situation of the user 200 if the user's wish is fulfilled, for example. The processing server 100 may input prompt information including material data and instructions to generate future spatio-temporal data using the material data to the generation AI 180, and acquire the future spatio-temporal data from the generation AI 180. The processing server 100 may store the generated future spatio-temporal data itself. The processing server 100 may store the generated future spatio-temporal data in the DB server 190.

[0033] The processing server 100 provides the future spatio-temporal data of the user 200 to the user 200. The processing server 100 may transmit the future spatio-temporal data of the user 200 to the communication terminal 300. The processing server 100 may transmit the future spatio-temporal data of the user 200 to the eyewear 400. The user 200 may view the future spatio-temporal data using the communication terminal 300. The user 200 may view the future spatio-temporal data using the eyewear 400.

[0034] The spatiotemporal data generation function for corporations will now be described. The processing server 100 generates past spatiotemporal data representing the past space-time of a target using material data about a specific target (such as a corporation or a region) provided by, for example, a corporation, an individual belonging to a corporation, or an individual not belonging to a corporation. The processing server 100 may receive material data from the communication terminal 300 of the user 200.

[0035] The material data includes images of the object. The material data may include images at each of a plurality of points in time. The images may be still images. The images may be videos. The material data may include map data including information on the locations where the images were captured. The material data may include paper data including papers on the object. The material data may include research material data including research materials on the object. The material data may include newspaper article data including newspaper articles on the object. The material data may include meeting minutes data including meeting minutes on the object. The material data may include data other than these.

[0036] The past spatio-temporal data may be a 3D video that represents the history of the target. The past spatio-temporal data may be content for an XR device. For example, the past spatio-temporal data is content for an MR device. For example, the past spatio-temporal data is content for an AR device. For example, the past spatio-temporal data is content for a VR device.

[0037] The processing server 100 may generate the past spatio-temporal data using the generation system AI 180. The processing server 100 may input material data and prompt information including an instruction to generate the past spatio-temporal data using the material data to the generation system AI 180, and acquire the past spatio-temporal data from the generation system AI 180.

[0038] The processing server 100 may store the material data provided by the user 200 by itself. The processing server 100 may store the material data provided by the user 200 in the DB server 190. The processing server 100 may store the generated past spatio-temporal data by itself. The processing server 100 may store the generated past spatio-temporal data in the DB server 190.

[0039] The processing server 100 provides the target past spatio-temporal data to the user 200. The processing server 100 may transmit the target past spatio-temporal data to the communication terminal 300. The processing server 100 may transmit the target past spatio-temporal data to the eyewear 400. The user 200 may view the past spatio-temporal data using the communication terminal 300. The user 200 may view the past spatio-temporal data using the eyewear 400.

[0040] The processing server 100 generates future spatio-temporal data representing the future space-time of the target based on, for example, material data provided by the user 200. The future spatio-temporal data may be a 3D video that predicts the future of the target. The future spatio-temporal data may be a 3D video that is predicted from events that have repeatedly occurred in the history of the target. The processing server 100 may input prompt information including material data and instructions to generate future spatio-temporal data using the material data to the generation AI 180, and obtain the future spatio-temporal data from the generation AI 180. The processing server 100 may store the generated future spatio-temporal data itself. The processing server 100 may store the generated future spatio-temporal data in the DB server 190.

[0041] The processing server 100 provides the target future spatio-temporal data to the user 200. The processing server 100 may transmit the target future spatio-temporal data to the communication terminal 300. The processing server 100 may transmit the target future spatio-temporal data to the eyewear 400. The user 200 may view the future spatio-temporal data using the communication terminal 300. The user 200 may view the future spatio-temporal data using the eyewear 400.

[0042] The processing server 100 may be arranged in a so-called MEC (Multi-access Edge Computing). The processing server 100 may be applied to a system that realizes RAN (Radio Access Network) control and AI processing using a management infrastructure and multiple distributed infrastructures managed by the management infrastructure.

[0043] The RAN referred to here may be a virtualized vRAN (Virtual RAN) or a physical RAN. The AI ​​processing referred to here may include RAN control AI processing and non-RAN control AI processing. An example of RAN control AI processing is a RAN Intelligent Controller (RIC). Note that RAN control AI processing is not limited to RIC. Non-RAN control AI processing may correspond to a so-called MEC application. The processing server 100 may be arranged on a distributed platform. The processing server 100 may be arranged on a management platform.

[0044] The distributed infrastructure may be a data center located in various locations. The distributed infrastructure may be configured with multiple devices. The distributed infrastructure may be realized on a virtualization infrastructure with multiple devices. The distributed infrastructure may also be realized by a single device. In other words, the distributed infrastructure may be a distributed device.

[0045] The management infrastructure may be a data center that manages multiple distributed infrastructures. The management infrastructure may be configured on multiple devices. The management infrastructure may be realized on a virtualization infrastructure using multiple devices. The management infrastructure may also be realized by a single device. In other words, the management infrastructure may be a management device.

[0046] The management infrastructure may be called the Core Brain, and the distributed infrastructure may be called the Regional Brain. Multiple layers of distributed infrastructure may be placed under the management infrastructure. For example, if two layers of distributed infrastructure are placed under the management infrastructure, the management infrastructure may be called the Core Brain, the distributed infrastructure at the layer below it may be called the Regional Brain, and the distributed infrastructure at the layer below that may be called the Sub-Regional Brain.

[0047] The distributed infrastructure may be equipped with one or more central processing units (CPUs). The distributed infrastructure may be equipped with one or more GPUs. The distributed infrastructure may be equipped with multiple super chips, each of which has a CPU and a GPU connected via an interconnect. The interconnect may have memory consistency and be capable of achieving high bandwidth and low latency. In this way, the distributed infrastructure may have CPU resources and GPU resources as computational resources.

[0048] FIG. 2 is an explanatory diagram illustrating the personalized spatiotemporal data generation function performed by the processing server 100. In the example shown in FIG. 2, the processing server 100 receives material data 210 from the communication terminal 300 of the user 200. The material data 210 includes captured images captured at locations where the user 200 was previously located. The material data 210 may include captured images at each of a plurality of time points. The material data 210 may include map data including information on locations where the captured images were captured. The material data 210 may include diary data including diaries for a plurality of time points at which the captured images were captured. The material data 210 may also include data other than these.

[0049] The processing server 100 uses the generation system AI 180 to generate past spatiotemporal data representing the past space-time of the user 200, using the material data 210. For example, the processing server 100 uses an image included in the material data 210 to generate spatial data 502 representing the situation of a location where the user 200 was located, including the surrounding scenery. The processing server 100 may generate the spatial data 502 by inputting prompt information to the generation system AI 180, the prompt information including an image included in the material data 210 and an instruction to generate spatial data 502 representing the situation of a location where the user 200 was located, including the surrounding scenery, using the image. The processing server 100 may generate the spatial data 502 at multiple points in time using multiple images at multiple points in time. The spatial data 502 at multiple points in time represent the situation of a location where the user 200 was located at each of the multiple points in time. Therefore, if the locations where the user 200 was located at different points in time, the multiple spatial data 502 will represent the situations of the different locations. If the location where the user 200 was located at multiple points in time was the same, the multiple pieces of spatial data 502 will represent the situation at the same location.

[0050] The processing server 100 may further use material data 210 provided by other users 200 to generate the spatial data 502. For example, the processing server 100 generates the spatial data 502 for a certain location by using an image included in the material data 210 of the user 200 and an image included in the material data 210 of the other users 200. The processing server 100 may generate the spatial data 502 by inputting, to the generation AI 180, prompt information including the image included in the material data 210 of the user 200, the image included in the material data 210 of the other users 200, and an instruction to generate the spatial data 502 using these images.

[0051] The processing server 100 may automatically complement a time point for which no image exists. For example, the processing server 100 generates spatial data 504 by complementing an image of the time point for which no image exists from images of previous and subsequent time points. The processing server 100 may generate the spatial data 504 by inputting prompt information to the generation system AI 180, the prompt information including images of the previous and subsequent time points and instructions for generating spatial data 504 by complementing images of the intermediate time points from the images of the previous and subsequent time points.

[0052] The processing server 100 may generate the spatial data 502 and the spatial data 504 by inputting prompt information to the generation system AI 180, the prompt information including an image included in the material data 210 of the user 200, an instruction to generate spatial data 502 using the image, and, for a time point when no image is available, an instruction to generate spatial data 504 by complementing the image at that time point from images at previous and subsequent times.

[0053] The processing server 100 may generate past spatio-temporal data using the generated spatial data 502 and spatial data 504. The processing server 100 may generate past spatio-temporal data by inputting prompt information to the generation AI 180, the prompt information including the generated spatial data 502 and spatial data 504 and instructions for generating past spatio-temporal data using the generated spatial data 502 and spatial data 504.

[0054] The processing server 100 may generate future spatio-temporal data 508 representing the future space-time of the user 200, which is a prediction of the future of the user 200, based on the material data 210 of the user 200. For example, the processing server 100 may identify a past history of the user 200 based on the material data 210 of the user 200, and generate the future spatio-temporal data 508 of the user 200 based on the identified history. The processing server 100 may generate the future spatio-temporal data 508 by inputting the material data 210 of the user 200 and prompt information including an instruction to identify a past history of the user 200 based on the material data 210 and generate the future spatio-temporal data 508 of the user 200 based on the identified history to the generation AI 180.

[0055] FIG. 3 is an explanatory diagram illustrating the spatiotemporal data generation function for corporations performed by the processing server 100. In the example shown in FIG. 3, the processing server 100 receives material data 220 about a specific object from the communication terminal 300 of the user 200. The material data 220 includes images of the object. The material data 220 may include images at each of a plurality of points in time. The material data 220 may include map data including information on the locations where the images were captured. The material data 220 may include paper data including papers about the object. The material data 220 may include research material data including research materials about the object. The material data 220 may include newspaper article data including newspaper articles about the object. The material data 220 may include meeting minutes data including meeting minutes about the object. The material data 220 may also include data other than these.

[0056] The processing server 100 utilizes the generation system AI 180 to generate past spatiotemporal data representing the past space-time of an object using the material data 220. For example, the processing server 100 uses an image included in the material data 220 to generate spatial data 512 representing the object including the surrounding scenery. The processing server 100 may generate the spatial data 512 by inputting prompt information to the generation system AI 180, the prompt information including an image included in the material data 220 and an instruction to generate the spatial data 512 representing the object including the surrounding scenery using the image. The processing server 100 may generate the spatial data 512 at multiple points in time using multiple images at multiple points in time.

[0057] The processing server 100 may automatically complement a time point for which no image exists. For example, the processing server 100 generates spatial data 514 by complementing an image for a time point for which no image exists from images for previous and subsequent time points. The processing server 100 may generate the spatial data 514 by inputting prompt information to the generation system AI 180, the prompt information including images for the previous and subsequent time points and instructions for generating spatial data 514 by complementing images for intermediate time points from the images for the previous and subsequent time points.

[0058] The processing server 100 may generate the spatial data 512 and the spatial data 514 by inputting prompt information to the generation system AI 180, the prompt information including an image included in the material data 220, an instruction to generate spatial data 512 using the image, and, for a time point when no image is available, an instruction to generate spatial data 514 that complements the image at that time point from images from the previous and next time points.

[0059] The processing server 100 may generate past spatio-temporal data using the generated spatial data 512 and spatial data 514. The processing server 100 may generate past spatio-temporal data by inputting prompt information to the generation AI 180, the prompt information including the generated spatial data 512 and spatial data 514 and instructions for generating past spatio-temporal data using the generated spatial data 512 and spatial data 514.

[0060] The processing server 100 may generate future spatio-temporal data 518 representing the future space-time of the target, which is a prediction of the target's future, based on the material data 220. For example, the processing server 100 may identify recurring events in the target's history based on the material data 220, and generate the target's future spatio-temporal data 518 based on the identified events. The processing server 100 may generate the future spatio-temporal data 518 by inputting the material data 220 and prompt information to the generation AI 180, the prompt information including instructions to identify recurring events in the target's history based on the material data 220 and to generate the future spatio-temporal data 518 based on the identified events.

[0061] The processing server 100 may generate multiple different future space-time data 518 based on the material data 220. For example, the processing server 100 generates future space-time data 518 representing a space-time in which the target's future is predicted normally, future space-time data 518 representing a space-time in which a good future is predicted for the target, and future space-time data 518 representing a space-time in which a bad future is predicted for the target, based on the material data 220. The processing server 100 may input prompt information to the generation AI 180, the prompt information including instructions to generate, based on the material data 220, the future space-time data 518 representing a space-time in which the target's future is predicted normally, the future space-time data 518 representing a space-time in which a good future is predicted for the target, and the future space-time data 518 representing a space-time in which a bad future is predicted for the target, and obtain the three future space-time data 518 from the generation AI 180.

[0062] 4 is an explanatory diagram illustrating correction of spatio-temporal data by the processing server 100. In the spatio-temporal data generation function for individuals, the processing server 100 may generate past spatio-temporal data and future spatio-temporal data corrected based on the interests of the user 200. In the spatio-temporal data generation function for corporations, the processing server 100 may generate past spatio-temporal data and future spatio-temporal data corrected based on the interests of the user 200 who will experience it.

[0063] First, a case where the user 200 generates spatio-temporal data using their own material in the personalized spatio-temporal data generation function will be described. In the generation of past spatio-temporal data, if no correction is performed, the processing server 100 generates past spatio-temporal data representing a past virtual space-time from an image included in the material data 210. If correction is performed, the processing server 100 generates past spatio-temporal data from an image included in the material data 210, and uses data other than images included in the material data 210 (such as a diary) to identify an area in the space of the past spatio-temporal data that the user 200 was particularly interested in, and corrects the past spatio-temporal data to emphasize the identified area. As a specific example, in a space where there is a tower surrounded by a moat and a bridge spanning the moat, if the user 200 was particularly interested in the tower, the processing server 100 applies correction to emphasize the tower. The emphasis method may be any method that can emphasize an object, such as making the color more noticeable or increasing the size. The processing server 100 may correct the past spatio-temporal data so as to make less prominent an area of ​​the space of the past spatio-temporal data that the user 200 was less interested in. As a specific example, in a space containing a tower surrounded by a moat and a bridge spanning the moat, if the user 200 was less interested in the bridge, the processing server 100 applies a correction to make the bridge less prominent. The method of making the bridge less prominent may be any method that can make the object less prominent, such as changing the color to an inconspicuous color, reducing the size, or deleting information about the target area and then generating a less prominent and more natural area based on information about the surrounding environment. In generating future spatio-temporal data without correction, the processing server 100 identifies the past history of the user 200 based on the material data 210, predicts the future from the identified history, and generates future spatio-temporal data. If correction is made, the processing server 100 identifies the user's 200 past history, interests, and level of interest based on the material data 210, and, taking into account the identified interests and level of interest, predicts the future from the identified history and generates future spatiotemporal data.As a specific example, if user 200 has previously toured a garden and has a high level of interest in a tower in the garden, processing server 100 generates future time-space data representing a future time-space in which user 200 will stroll through a garden that has a famous or special tower.

[0064] Next, a case will be described in which, in the personal spatio-temporal data generation function, spatio-temporal data is jointly generated using materials from multiple people. In the generation of past spatio-temporal data, if no correction is performed, the processing server 100 generates past spatio-temporal data representing a past virtual space-time from images included in multiple pieces of material data 210. If correction is performed, the processing server 100 generates past spatio-temporal data from images included in multiple pieces of material data 210, and uses data other than images included in the multiple pieces of material data 210 (diaries, etc.) to identify areas in the space of the past spatio-temporal data that interested and did not interest each of the multiple users 200, and corrects the past spatio-temporal data so that each user can confirm their areas of interest and areas of disinterest. As a specific example, in a space where a tower surrounded by a moat and a bridge spanning the moat exist, if a first user 200 is interested in the tower and a second user 200 is not interested in the bridge, the processing server 100 corrects the past spatio-temporal data so that it is possible to confirm that the first user 200 is interested in the tower and the second user 200 is not interested in the bridge. For example, the processing server 100 adds a description to the tower in the past spatio-temporal data explaining that the first user 200 was interested in it, and adds a description to the bridge in the past spatio-temporal data explaining that the second user 200 was not interested in it. The processing server 100 may have a function to generate past spatio-temporal data reflecting the interests of a selected user 200 from among multiple users 200. In this case, when the first user 200 is selected, the processing server 100 applies a correction to emphasize the tower in the past spatio-temporal data. Furthermore, when a second user 200 is selected, the processing server 100 applies correction to make the bridge portion in the past spatio-temporal data less noticeable. In generating future spatio-temporal data, if no correction is performed, the processing server 100 identifies the past events of the multiple users 200 based on the multiple material data 210, predicts the future from the identified events, and generates future spatio-temporal data. If correction is performed, the processing server 100 identifies the past events and common interests of the multiple users 200 based on the multiple material data 210, and predicts the future from the identified events while taking the identified interests into consideration, and generates future spatio-temporal data.As a specific example, if multiple users 200 have toured a garden in the past and a common interest is a tower in the garden, the processing server 100 generates future time-space data representing a future time-space in which the multiple users 200 will stroll through a garden where a famous or special tower is located.

[0065] Next, a case where spatio-temporal data is generated using corporate materials in the spatio-temporal data generation function for corporations will be described. In the generation of past spatio-temporal data, if no correction is required, the processing server 100 generates past spatio-temporal data representing a past virtual space-time from images included in material data 220 provided by the corporation. If correction is required, the processing server 100 generates past spatio-temporal data from images included in the material data 220 and corrects the past spatio-temporal data according to the interests of the user 200 who experiences the past spatio-temporal data. For example, the processing server 100 may acquire material data 210 of the user 200 who experiences the past spatio-temporal data and use the material data 210 to identify the interests of the user 200. For example, the processing server 100 identifies an interest of the user 200 from among multiple interests in the space of the past spatio-temporal data, and corrects the past spatio-temporal data to emphasize the area corresponding to the identified interest. For example, if the corporation is a research institution and the past spatio-temporal data represents the history of multiple types of research conducted by the research institution, the processing server 100 identifies, from the multiple types of research, research that matches the interests of the user 200 and corrects the past spatio-temporal data to emphasize the area corresponding to the identified research. In generating future spatio-temporal data, if no correction is performed, the processing server 100 predicts multiple possible futures based on the various past histories of the corporation based on the material data 220, and generates multiple future spatio-temporal data representing each of the predicted multiple futures. If correction is performed, the processing server 100 predicts multiple possible futures based on the various past histories of the corporation based on the material data 220, and narrows down the predicted multiple futures according to the interests of the user 200 who experiences the future spatio-temporal data, and generates future spatio-temporal data representing the narrowed down futures.

[0066] Fig. 5 schematically illustrates an example of a processing flow in the processing system 10. In the example illustrated in Fig. 5, first, the user 200 provides a prompt input to the communication terminal 300. The user 200 provides the prompt input to the communication terminal 300, including an instruction for a target time and place and material data 210 such as a still image or a video. The communication terminal 300 transmits the prompt input to the processing server 100 and stores it in the DB server 190. The DB server 190 stores the specified time and place in association with the material data 210. The processing server 100 uses the generative AI 180 to generate past spatiotemporal data representing the space-time of the specified time and place using the material data 210.

[0067] The processing server 100 transmits the generated past spatio-temporal data to the communication terminal 300 and presents it to the user 200. The processing server 100 may transmit the past spatio-temporal data to the eyewear 400 and present it to the user 200. The user 200 checks the past spatio-temporal data and selects OK or NG. If OK is selected, the communication terminal 300 notifies the processing server 100 that it is OK, and the processing server 100 stores the past spatio-temporal data in the DB server 190. The DB server 190 may store the past spatio-temporal data in a spatio-temporal library.

[0068] If NG is selected, the communication terminal 300 prompts the user 200 to additionally input material data 210. The user 200 additionally inputs the material data 210. The communication terminal 300 transmits the additionally input material data 210 to the processing server 100. The processing server 100 reads the existing material data 210 from the DB server 190 and regenerates past spatio-temporal data by combining it with the additionally input material data 210. The processing server 100 stores the regenerated past spatio-temporal data in the DB server 190. The DB server 190 may store the past spatio-temporal data in a spatio-temporal library. Note that the processing server 100 may present the regenerated past spatio-temporal data to the user 200 again, and if OK is selected, store the past spatio-temporal data in the DB server 190, and if NG is selected, accept additional input of further material data 210.

[0069] The communication terminal 300 may accept a prompt input from the user 200, including an instruction for a target time and place, material data 210 of correction material such as a diary, and a correction instruction. The communication terminal 300 transmits the accepted prompt input to the processing server 100. Based on the accepted instruction, the processing server 100 may read existing material data 210 from the DB server 190, combine it with the material data 210 of the correction material, and generate and correct past spatio-temporal data. The processing server 100 stores the past spatio-temporal data in the DB server 190. The DB server 190 may store the past spatio-temporal data in a spatio-temporal library.

[0070] 6 shows an example of the functional configuration of the processing server 100. The processing server 100 includes a storage unit 102, a reception unit 104, a material data acquisition unit 106, a spatiotemporal data generation unit 108, a spatiotemporal data provision unit 110, and an AI management unit 112. Note that it is not essential for the processing server 100 to include all of these units.

[0071] The storage unit 102 stores various data. When the processing server 100 has the generation system AI 180, the storage unit 102 stores the generation system AI 180.

[0072] The reception unit 104 receives various inputs from the user 200. The reception unit 104 stores the received inputs from the user 200 in the storage unit 102. The reception unit 104 may store the received inputs from the user 200 in the DB server 190. The reception unit 104 receives inputs from the user 200 from, for example, the communication terminal 300. The reception unit 104 receives inputs from the user 200 from, for example, the eyewear 400.

[0073] The receiving unit 104 receives, for example, an instruction from the user 200 and material data 210. The instruction from the user 200 may include an instruction to generate past spatio-temporal data. The instruction to generate past spatio-temporal data may include information on a target time and location. The instruction from the user 200 may include an instruction to generate future spatio-temporal data. The instruction to generate future spatio-temporal data may include information on a target time and location. The information on the target time may indicate a period, for example, from x years ago to x years ago. The information on the target location may indicate an area indicated by a place name, an address, and latitude and longitude. In addition to the instruction from the user 200 and the material data 210, the receiving unit 104 may also receive material data 210 from another user 200.

[0074] The material data acquisition unit 106 acquires material data 210 used when generating past-time spatiotemporal data or future-time spatiotemporal data. The material data acquisition unit 106 may acquire the material data 210 from the storage unit 102. The material data acquisition unit 106 may acquire the material data 210 from the DB server 190.

[0075] The spatio-temporal data generation unit 108 generates spatio-temporal data of the user 200. For example, the spatio-temporal data generation unit 108 generates past spatio-temporal data of the user 200. The spatio-temporal data generation unit 108 generates the past spatio-temporal data using the instruction accepted by the acceptance unit 104 and the material data 210. The spatio-temporal data generation unit 108 may generate the past spatio-temporal data of the user 200, which is composed of spatial data in which the user 200 was located at multiple points in time, based on the material data 210.

[0076] The spatio-temporal data generation unit 108 may generate the past spatio-temporal data using the generative AI 180. For example, the spatio-temporal data generation unit 108 generates instruction data including an instruction to generate past spatio-temporal data and inputs the generated instruction data to the generative AI 180 to generate the past spatio-temporal data. The instruction data may be so-called prompt input. The instruction data may include an instruction to generate spatial data representing the situation of a location where the user 200 was located at each of multiple time points using the material data 210 and multiple captured images included in the material data 210, and to generate spatio-temporal data using the generated multiple spatial data. Rather than simply issuing an instruction to the generative AI 180 to generate past spatio-temporal data representing the past space-time of the user 200, the generative AI 180 is instructed to generate captured images from multiple time points in the past and spatial data from the multiple captured images, and to generate past spatio-temporal data using the multiple spatial data. This makes it possible to reproduce the spatial data at the time points when captured images exist in the material data 210 with relatively high accuracy and to generate a 3D video representing the past experiences of the user 200 by linking the captured images in the time axis direction.

[0077] The spatiotemporal data generation unit 108 may instruct the generative AI 180 about the “true feelings” of the user 200 regarding an event that could only be understood by the person who experienced the event in the past. The “true feelings” may be genuine feelings, not fictitious feelings or false feelings. For example, the reception unit 104 receives material data 210 including captured images captured at locations where the user 200 was located at each of multiple past time points and the user 200’s “true feelings” regarding the event that the user 200 experienced at each location. The spatiotemporal data generation unit 108 generates spatial data representing the situation of the location where the user 200 was located at each of multiple time points using the material data 210, the captured images included in the material data 210, and the user 200’s “true feelings” included in the material data 210, and inputs instruction data to the generative AI 180 including an instruction to generate spatiotemporal data using the generated spatial data.

[0078] The spatio-temporal data generation unit 108 stores the generated spatial data and the past spatio-temporal data in the storage unit 102. The spatio-temporal data generation unit 108 may store the generated spatial data and the past spatio-temporal data in the DB server 190.

[0079] The spatio-temporal data providing unit 110 provides the past spatio-temporal data generated by the spatio-temporal data generating unit 108 to the user 200. The spatio-temporal data providing unit 110 may transmit the past spatio-temporal data to the communication terminal 300. The spatio-temporal data providing unit 110 may transmit the past spatio-temporal data to the eyewear 400.

[0080] The receiving unit 104 may receive feedback from the user 200 regarding the past spatio-temporal data provided by the spatio-temporal data providing unit 110. For example, the receiving unit 104 receives feedback including designation of an area in the space-time represented by the past spatio-temporal data that matches the memory of the user 200 and an area that does not match the memory of the user 200. The spatio-temporal data generating unit 108 may correct the past spatio-temporal data based on the feedback from the user 200. For example, the spatio-temporal data generating unit 108 corrects the past spatio-temporal data by inputting, to the generation AI 180, instruction data including the past spatio-temporal data and an instruction to correct the past spatio-temporal data based on the feedback from the user 200.

[0081] The spatio-temporal data generation unit 108 may generate past spatio-temporal data while having the user 200 confirm the spatial data. For example, the spatio-temporal data generation unit 108 inputs, to the generation AI 180, instruction data including an instruction to generate spatial data representing the situation of a location where the user 200 was located at each of multiple time points using the material data 210 and the multiple captured images included in the material data 210, and acquires multiple pieces of spatial data from the generation AI 180. The spatio-temporal data providing unit 110 provides, to the user 200, the multiple pieces of spatial data generated by the spatio-temporal data generation unit 108. The receiving unit 104 receives feedback from the user 200 regarding the spatial data provided by the spatio-temporal data providing unit 110. The spatio-temporal data generation unit 108 corrects the spatial data based on the feedback from the user 200. For example, the spatio-temporal data generation unit 108 corrects the spatial data by inputting, to the generation AI 180, instruction data including the spatial data and an instruction to correct the spatial data based on the feedback from the user 200. After correcting the plurality of spatial data, the spatio-temporal data generation unit 108 generates past spatio-temporal data using the plurality of spatial data. This allows the generation of spatial data corresponding to each of the plurality of captured images included in the material data 210, correcting erroneous areas, and then generating past spatio-temporal data with a high degree of reproducibility.

[0082] The spatiotemporal data generation unit 108 may generate spatial data representing points where the user 200 was located at multiple points in time using the multiple captured images and map data included in the material data 210, generate instruction data including an instruction to generate past spatiotemporal data using the multiple pieces of generated spatial data, and input the instruction data to the generation AI 180. This makes it possible to correct or supplement images of the surroundings of the points where the user 200 was located using information from the map data, which can contribute to improving the reproducibility of the spatiotemporal data.

[0083] The spatiotemporal data generation unit 108 may generate spatial data representing locations where the user was located at multiple points in time using multiple captured images included in the material data 210, estimate the user's 200 level of interest in multiple areas within each of the multiple spatial data using the diary data, correct the spatial data according to the estimated level of interest, and then generate instruction data including an instruction to generate spatiotemporal data using the multiple spatial data, and input the instruction data to the generative AI 180. The spatiotemporal data generation unit 108 may, for example, include in the instruction data an instruction to emphasize an area of ​​the multiple areas of the spatial data that is of high interest to the user 200. The spatiotemporal data generation unit 108 may, for example, include in the instruction data an instruction to make an area of ​​the multiple areas of the spatial data that is of low interest to the user 200 less noticeable. The diary data includes the date and time, the location, and the contents of the diary written by the user 200. In other words, the diary data includes information on the user's 200's interests and level of interest in the locations where the user 200 was located at each of multiple points in time. By inputting diary data into the generation system AI 180 in addition to multiple captured images, it is possible to estimate the user's 200 interests and level of interest at the time and place when each of the multiple captured images was captured, making it possible to generate past time-space data that reflects the interests and level of interest.

[0084] The spatio-temporal data generation unit 108 may supplement spatial data for a time point for which no captured image is included in the material data 210, using captured images included in the material data 210. For example, the spatio-temporal data generation unit 108 generates spatial data representing points where the user was located at each of multiple time points using multiple captured images included in the material data 210, generates spatial data for a time point between consecutive time points using spatial data for the consecutive time points, generates instruction data including an instruction to generate past spatio-temporal data using the generated spatial data, and inputs the instruction data to the generation AI 180. For example, if the user 200 was located at a first location at a first time point and a second location at a second time point, the spatio-temporal data generation unit 108 determines that the user 200 was located at a third location between the first and second locations at a third time point between the first and second time points, and causes the generation AI 180 to generate spatial data corresponding to the third time point. This makes it possible to generate spatial data at a time when no captured image existed, although accurate reproduction is difficult, and it is possible to generate past spatiotemporal data with high temporal continuity, i.e., data that is not disjointed in time, thereby contributing to improving the quality of experience of user 200.

[0085] When generating past spatio-temporal data of the user 200, if the receiving unit 104 receives material data 210 from other users 200 as well, the spatio-temporal data generation unit 108 may generate the past spatio-temporal data of the user 200 using the material data 210 of multiple users 200 including the user 200 and the other users 200. The spatio-temporal data generation unit 108 may estimate the interests and levels of interest of each of the multiple users 200 using the material data 210 of each of the multiple users 200, and generate the past spatio-temporal data of the user 200 using the estimated interests and levels of interest. For each of the multiple spatial data, the spatio-temporal data generation unit 108 may highlight a region that has a high interest among multiple regions included in the spatial data, and may include in the instruction data an instruction to make it possible to confirm which user 200 has a high interest. The spatio-temporal data generation unit 108 may include in the instruction data an instruction to make less noticeable a region included in the spatial data that has received little interest from any of the multiple users 200, and to make it possible to confirm which user 200 received little interest. The spatio-temporal data providing unit 110 provides past spatio-temporal data generated by the generative AI 180 in response to the instruction data. This makes it possible to provide past spatio-temporal data that enables the user 200 to confirm regions in the user's past time-space that received high or low interest, as well as regions in the user's past time-space that received high or low interest from other users 200. The spatio-temporal data generation unit 108 may generate multiple pieces of past spatio-temporal data corresponding to each of the multiple users 200. For example, the spatio-temporal data generation unit 108 generates past spatio-temporal data for the user 200 and corrects the past spatio-temporal data in accordance with the interests of each of the multiple users 200, thereby generating multiple pieces of past spatio-temporal data. The spatiotemporal data providing unit 110 provides a plurality of pieces of past spatiotemporal data generated by the generation system AI 180 in response to the instruction data.The spatiotemporal data providing unit 110, for example, receives a designation from the experiencing user 200 as to which of the multiple users 200's interests the user 200 wishes to be provided with past spatiotemporal data that reflects that interest, and provides the experiencing user 200 with the past spatiotemporal data that reflects the interest of the designated user 200.

[0086] The spatiotemporal data providing unit 110 may provide the past spatiotemporal data of the user 200 to other users 200 who have been approved by the user 200. The processing server 100 may manage the past spatiotemporal data of the user 200 so that it is not provided to other users 200 unless approval is obtained from the user 200. This makes it possible to provide an environment in which other people can simulate the past experiences of that person, if the person's consent is obtained.

[0087] The spatiotemporal data providing unit 110 may make the past spatiotemporal data of the user 200 public when approval is obtained from the user 200. For example, the spatiotemporal data providing unit 110 makes the past spatiotemporal data of the user 200 public in the metaverse when approval is obtained from the user 200. This makes it possible to realize a metaverse space in which various past experiences of the user 200 can be simulated, provided that approval is obtained from the user 200.

[0088] The spatio-temporal data generation unit 108 generates, for example, future spatio-temporal data of the user 200. The spatio-temporal data generation unit 108 generates the future spatio-temporal data using the instruction received by the reception unit 104 and the material data 210. The spatio-temporal data generation unit 108 generates instruction data including an instruction to generate future spatio-temporal data, and inputs the generated instruction data to the generative AI 180 to generate the future spatio-temporal data. The instruction data may be so-called prompt input. The instruction data may include the material data 210 and an instruction to estimate the status of the user 200 at each of multiple points in time using the material data 210, estimate the past history of the user 200, and predict the future from the past history of the user 200 to generate the future spatio-temporal data. Rather than simply giving instructions to the generation system AI 180 to generate future time-space data representing the future time-space of the user 200, by giving instruction data including material data 210 and instructions to estimate the past history of the user 200 and predict the future from the past history, it may be possible to generate future time-space data that is in line with the future situation of the user 200 and the future aspirations of the user 200.

[0089] The spatio-temporal data generation unit 108 may estimate the user 200's interests based on the material data 210, and generate future spatio-temporal data of the user 200 based on the estimated interests. For example, the spatio-temporal data generation unit 108 inputs to the generation AI 180 instruction data including the material data 210, instructions to estimate the user 200's status at multiple points in time using the material data 210, estimate the user 200's past history, and predict the future from the user 200's past history, instructions to estimate the user 200's interests at multiple points in time using the material data 210, and instructions to generate future spatio-temporal data using the predicted future and the estimated interests. The spatio-temporal data generation unit 108 may input to the generation AI 180 instruction data including, for example, an instruction to make a correction to emphasize an area in the future space-time that is related to the user 200's interests. This makes it possible to generate future spatio-temporal data that emphasizes an area in which the user 200 is interested, based on the future predicted from the user 200's past history.

[0090] The spatio-temporal data generation unit 108 may estimate the interests of multiple users 200 based on the material data 210 of multiple users 200, including the user 200 and other users 200, and may generate future spatio-temporal data of the user 200 based on the estimated interests. For example, the spatio-temporal data generation unit 108 inputs to the generation AI 180 the material data 210 of the user 200 and instruction data including: an instruction to estimate the status of each user 200 at multiple points in time using the material data 210 of the user 200, to estimate the past history of the user 200, and to predict the future from the past history of the user 200; an instruction to estimate the interests of the user 200 at multiple points in time using the material data 210 of the user 200; an instruction to estimate the interests of other users 200 at multiple points in time using the material data 210 of the other users 200; and an instruction to generate future spatio-temporal data using the predicted future and the estimated interests of the user 200 and the other users 200. This makes it possible to generate future spatio-temporal data that reflects the interests of multiple users 200 for the future predicted from the past history of the users 200.

[0091] The spatio-temporal data generation unit 108 stores the generated future spatio-temporal data in the storage unit 102. The spatio-temporal data generation unit 108 may store the generated future spatio-temporal data in the DB server 190. The spatio-temporal data providing unit 110 provides the future spatio-temporal data generated by the spatio-temporal data generation unit 108 to the user 200. The spatio-temporal data providing unit 110 may transmit the future spatio-temporal data to the communication terminal 300. The spatio-temporal data providing unit 110 may transmit the future spatio-temporal data to the eyewear 400. The spatio-temporal data providing unit 110 may provide the future spatio-temporal data of the user 200 to another user 200 who has been approved by the user 200. The spatio-temporal data providing unit 110 may manage the future spatio-temporal data of the user 200 so that it is not provided to other users 200 unless approval is obtained from the user 200. The spatio-temporal data providing unit 110 may make public the future spatio-temporal data of the user 200 when approval is obtained from the user 200. For example, the spatio-temporal data providing unit 110 makes public the future spatio-temporal data of the user 200 in the metaverse when approval is obtained from the user 200. Compared to the past, future spatio-temporal data has a wider range of uncertainties, and therefore, it is considered that approval by the user 200 is more necessary.

[0092] The receiving unit 104 receives, for example, an instruction from the user 200 and material data 220. The instruction from the user 200 may include an instruction to generate target past spatio-temporal data. The instruction from the user 200 may include an instruction to generate target future spatio-temporal data. The target time information may indicate a period, for example, from x years ago to x years ago. The target location information may indicate an area indicated by a place name, an address, and latitude and longitude.

[0093] The material data acquisition unit 106 acquires material data 220 used when generating past-time spatiotemporal data or future-time spatiotemporal data. The material data acquisition unit 106 may acquire the material data 220 from the storage unit 102. The material data acquisition unit 106 may acquire the material data 220 from the DB server 190.

[0094] The spatio-temporal data generation unit 108 generates spatio-temporal data of the target. For example, the spatio-temporal data generation unit 108 generates past spatio-temporal data of the target. The spatio-temporal data generation unit 108 generates the past spatio-temporal data using the instruction accepted by the acceptance unit 104 and the material data 220. The spatio-temporal data generation unit 108 may generate the past spatio-temporal data of the target, which is composed of spatial data of the target at multiple points in time, based on the material data 220.

[0095] The spatio-temporal data generation unit 108 may generate past spatio-temporal data of the target using the generative AI 180. For example, the spatio-temporal data generation unit 108 generates instruction data including instructions to generate past spatio-temporal data of the target, and inputs the generated instruction data to the generative AI 180 to generate past spatio-temporal data of the target. The instruction data may be so-called prompt input. The instruction data may include instructions to generate spatial data for multiple points in time using the material data 220, multiple images included in the material data 220, and at least one of papers, research materials, newspaper articles, and minutes included in the material data 220, and to generate past spatio-temporal data using the generated multiple spatial data. Rather than simply instructing the generation system AI 180 to generate past spatiotemporal data that represents the past time and space of the object, the generation system AI 180 is instructed to generate spatial data at multiple points in time using images of the object at multiple points in time in the past and at least one of papers, research materials, newspaper articles, and minutes related to the object, and to generate past spatiotemporal data using the multiple spatial data.This makes it possible to reproduce the spatial data at the time when images exist in the material data 220 with relatively high accuracy, and to generate a 3D video that represents the history of the object by connecting these data along the time axis.

[0096] The material data 220 may include, for example, paper data of papers discussing the history of the subject. By inputting material data 220 including such paper data into the generative AI 180, it may be possible to reflect the history of the subject that cannot be determined from images alone in the past spatio-temporal data. The material data 220 may further include the impact factor of the academic journal in which the paper is published. The spatio-temporal data generation unit 108 may instruct the generative AI 180 to generate past spatio-temporal data by prioritizing the content of paper data of papers published in academic journals with higher impact factors over the content of paper data of papers published in academic journals with lower impact factors. This may enable the generation of more reliable past spatio-temporal data.

[0097] The raw data 220 may include, for example, research material data of research materials that have been obtained by studying the subject over a long period of time. By inputting the raw data 220 including such research material data into the generative AI 180, it may be possible to reflect the history of the subject, which cannot be determined from images alone, in the past spatiotemporal data.

[0098] The material data 220 may include, for example, newspaper article data including newspaper articles containing content about the target. By inputting the material data 220 including such newspaper article data into the generation AI 180, it may be possible to reflect highly credible facts, etc., written by newspapers in the past spatio-temporal data. The material data 220 may further include the circulation of the newspaper in which the newspaper article is published. The spatio-temporal data generation unit 108 may instruct the generation AI 180 to generate past spatio-temporal data by placing more importance on the content of newspaper articles published in newspapers with larger circulations than on the content of newspaper articles published in newspapers with smaller circulations. This may enable the generation of more credible past spatio-temporal data.

[0099] The material data 220 may include, for example, minutes data including minutes of a meeting about the subject. By inputting the material data 220 including such minutes data into the generative AI 180, it may be possible to reflect various official thoughts about the subject in the past spatiotemporal data.

[0100] The spatio-temporal data generation unit 108 stores the generated spatial data and the past spatio-temporal data in the storage unit 102. The spatio-temporal data generation unit 108 may store the generated spatial data and the past spatio-temporal data in the DB server 190.

[0101] The spatio-temporal data providing unit 110 provides the past spatio-temporal data generated by the spatio-temporal data generating unit 108 to the user 200. The spatio-temporal data providing unit 110 may transmit the past spatio-temporal data to the communication terminal 300. The spatio-temporal data providing unit 110 may transmit the past spatio-temporal data to the eyewear 400.

[0102] The receiving unit 104 may receive feedback from the user 200 regarding the past spatio-temporal data provided by the spatio-temporal data providing unit 110. For example, the receiving unit 104 receives feedback including designation of areas that match facts and areas that do not match facts in the space-time represented by the past spatio-temporal data. The spatio-temporal data generating unit 108 may correct the past spatio-temporal data based on the feedback from the user 200. For example, the spatio-temporal data generating unit 108 corrects the past spatio-temporal data by inputting, to the generation AI 180, instruction data including the past spatio-temporal data and an instruction to correct the past spatio-temporal data based on the feedback from the user 200.

[0103] The spatio-temporal data generation unit 108 may generate past spatio-temporal data while having the user 200 confirm the spatial data. For example, the spatio-temporal data generation unit 108 inputs instruction data to the generative AI 180, the instruction data including an instruction to generate spatial data for each of multiple time points using the material data 220, multiple images included in the material data 220, and at least one of a paper, research material, newspaper article, and minutes included in the material data 220, and acquires multiple pieces of spatial data from the generative AI 180. The spatio-temporal data providing unit 110 provides the multiple pieces of spatial data generated by the spatio-temporal data generation unit 108 to the user 200. The receiving unit 104 receives feedback from the user 200 regarding the spatial data provided by the spatio-temporal data providing unit 110. The spatio-temporal data generation unit 108 corrects the spatial data based on the feedback from the user 200. For example, the spatio-temporal data generation unit 108 corrects the spatial data by inputting instruction data including the spatial data and an instruction to correct the spatial data based on feedback from the user 200 to the generative AI 180. After correcting the plurality of spatial data, the spatio-temporal data generation unit 108 generates past spatio-temporal data using the plurality of spatial data. This makes it possible to generate past spatio-temporal data by generating a plurality of spatial data using the material data 220 and correcting erroneous areas, thereby generating past spatio-temporal data with a high degree of reproducibility.

[0104] When the material data 220 includes map data, the spatiotemporal data generation unit 108 may generate spatial data for each of a plurality of points in time using the map data and a plurality of images included in the material data 220, generate instruction data including an instruction to generate past spatiotemporal data using the generated spatial data, and input the instruction data to the generation AI 180. This makes it possible to correct or supplement the surrounding image of the target using information from the map data, which can contribute to improving the degree of spatiotemporal reproduction.

[0105] The spatio-temporal data generation unit 108 may complement spatial data for a time point for which no image is included in the material data 220, using an image included in the material data 220. For example, the spatio-temporal data generation unit 108 generates spatial data for each of a plurality of time points using a plurality of images included in the material data 220, generates spatial data for a time point between consecutive time points using spatial data for the consecutive time points, generates instruction data including an instruction to generate past spatio-temporal data using the generated plurality of spatial data, and inputs the instruction data to the generation AI 180. This makes it possible to generate spatial data for a time point for which no image exists, although accurate reproduction is difficult, and generates past spatio-temporal data with high temporal continuity, i.e., without temporal discontinuities, which can contribute to improving the quality of experience of the user 200 who experiences the past spatio-temporal data.

[0106] The spatio-temporal data generation unit 108 may estimate the interests of the user 200 based on user-related information related to the user 200 who experiences the generated past spatio-temporal data, and correct the past spatio-temporal data based on the estimated interests. The user-related information of the user 200 may be, for example, data posted by the user 200 to a social networking service (SNS). The spatio-temporal data generation unit 108 may acquire and use data posted by the user 200 on the SNS used by the user 200 from the communication terminal 300 of the user 200. The user-related information may be material data 210 provided by the user 200. For example, the spatio-temporal data generation unit 108 corrects the past spatio-temporal data so as to emphasize an area corresponding to the user's interests among multiple spatial areas included in the past spatio-temporal data. The spatiotemporal data generation unit 108 may input instruction data to the generation system AI 180, the instruction data including past spatiotemporal data and an instruction to highlight an area corresponding to the user 200's interest among multiple areas of space included in the past spatiotemporal data, and obtain corrected past spatiotemporal data from the generation system AI 180.

[0107] The spatiotemporal data providing unit 110 may provide the past spatiotemporal data of the target to other users 200 who are approved by the user 200. The processing server 100 may manage the past spatiotemporal data of the target so that it is not provided to other users 200 unless approval is obtained from the user 200. In this way, if approval is obtained from the user 200 related to the target, such as a region or a corporation, an environment can be provided in which other people can simulate the past history of the target.

[0108] The spatio-temporal data providing unit 110 may make the past spatio-temporal data of the target public when approval is obtained from the user 200. For example, the spatio-temporal data providing unit 110 makes the past spatio-temporal data of the target public in the metaverse when approval is obtained from the target. This makes it possible to realize a metaverse space that allows a user to simulate the history of the target, on the condition that approval is obtained from the user 200 related to the target, such as a region or a corporation.

[0109] The spatio-temporal data generation unit 108 generates, for example, future spatio-temporal data of the target. The spatio-temporal data generation unit 108 generates the future spatio-temporal data using the instruction and the material data 220 received by the receiving unit 104. The spatio-temporal data generation unit 108 generates the future spatio-temporal data by generating instruction data including an instruction to generate future spatio-temporal data and inputting the generated instruction data to the generation AI 180. The instruction data may be a so-called prompt input. The instruction data may include the material data 220 and an instruction to identify a recurring event in the target's history using the material data 220 and generate future spatio-temporal data of the target based on the identified event. By providing the generation AI 180 with instruction data including the material data 220 and an instruction to identify a recurring event in the target's history and generate future spatio-temporal data based on the identified event, rather than simply providing an instruction to generate future spatio-temporal data representing the target's future space-time, it may be possible to generate highly credible future spatio-temporal data based on events that have repeatedly occurred in history.

[0110] The spatio-temporal data generation unit 108 may use the instruction and the raw data 220 received by the reception unit 104 to generate a plurality of different future spatio-temporal data.

[0111] For example, the spatio-temporal data generation unit 108 generates future spatio-temporal data representing a space-time predicted for the target's future, future spatio-temporal data representing a space-time predicted for a good future for the target, and future spatio-temporal data representing a space-time predicted for a bad future for the target, based on the material data 220. The processing server 100 may input the material data 220 and instruction data including instructions to generate, based on the material data 220, future spatio-temporal data representing a space-time predicted for the target's future, future spatio-temporal data representing a space-time predicted for a good future for the target, and future spatio-temporal data representing a space-time predicted for a bad future for the target, to the generation AI 180, and obtain three pieces of future spatio-temporal data from the generation AI 180.

[0112] Alternatively, for example, the receiving unit 104 may receive input of multiple themes from the user 200, and the spatio-temporal data generation unit 108 may generate multiple different future spatio-temporal data corresponding to each of the multiple themes. As a specific example, when the receiving unit 104 receives a first theme, a second theme, and a third theme, the spatio-temporal data generation unit 108 inputs the material data 220 and instruction data including instructions to generate future spatio-temporal data representing a space-time predicted for the future of the target using the material data 220 in accordance with the first theme, future spatio-temporal data representing a space-time predicted for the future of the target using the material data 220 in accordance with the second theme, and future spatio-temporal data representing a space-time predicted for the future of the target using the material data 220 in accordance with the third theme, to the generation AI 180, and acquires three pieces of future spatio-temporal data from the generation AI 180. The themes may be text indicating the direction in which the target will move, or the like.

[0113] When the spatio-temporal data generation unit 108 generates a plurality of different future space-time data, it may estimate the interests of the user 200 based on user-related information related to the user 200 who will experience any of the plurality of different future space-time data, and may determine the future space-time data to be experienced by the user 200 from the plurality of future space-time data based on the estimated interests. For example, the spatio-temporal data generation unit 108 corrects the future space-time data so as to emphasize, among multiple regions of space included in the future space-time data, a region corresponding to the interests of the user 200. The spatio-temporal data generation unit 108 may input the future space-time data and instruction data including an instruction to emphasize, among the multiple regions of space included in the future space-time data, a region corresponding to the interests of the user 200, to the generation AI 180, and acquire the corrected future space-time data from the generation AI 180.

[0114] The spatio-temporal data generation unit 108 stores the generated target future spatio-temporal data in the storage unit 102. The spatio-temporal data generation unit 108 may store the generated target future spatio-temporal data in the DB server 190. The spatio-temporal data providing unit 110 provides the target future spatio-temporal data generated by the spatio-temporal data generation unit 108 to the user 200 who instructed the generation of the future spatio-temporal data. The spatio-temporal data providing unit 110 may transmit the target future spatio-temporal data to the communication terminal 300. The spatio-temporal data providing unit 110 may transmit the target future spatio-temporal data to the eyewear 400. The spatio-temporal data providing unit 110 may provide the target future spatio-temporal data to another user 200 who is approved by the user 200 who instructed the generation of the future spatio-temporal data. The spatio-temporal data providing unit 110 may manage the target future spatio-temporal data so that it is not provided to another user 200 unless approval is obtained from the user 200 who instructed the generation of the future spatio-temporal data. The spatio-temporal data providing unit 110 may make the target future spatio-temporal data public when approval is obtained from the user 200 who instructed the generation of the future spatio-temporal data. For example, the spatio-temporal data providing unit 110 makes the target future spatio-temporal data public in the metaverse when approval is obtained from the user 200 who instructed the generation of the future spatio-temporal data.

[0115] The AI ​​manager 112 manages the generative AI 180. The AI ​​manager 112 may update the generative AI 180. The AI ​​manager 112 may tune the generative AI 180.

[0116] The AI ​​management unit 112 may provide the past spatio-temporal data to the user 200 and update or tune the generative AI 180 according to the feedback from the user 200. For example, when past spatio-temporal data is generated using the same material data 210, the AI ​​management unit 112 uses feedback including indication of an erroneous part in the past spatio-temporal data generated using the material data 210 to update or tune the generative AI 180 so that erroneous generation does not occur again.

[0117] The AI ​​management unit 112 may provide the future space-time data to the user 200 and update or tune the generative AI 180 according to the content of feedback from the user 200. For example, when the AI ​​management unit 112 uses feedback including an indication of an unlikely part of future space-time data generated using the material data 210 to generate future space-time data using the same material data 210, the AI ​​management unit 112 updates or tunes the generative AI 180 so that the indicated part is not included in the future space-time data.

[0118] The AI ​​management unit 112 may update or tune the generation system AI 180 so as to improve the generation accuracy when generating the target past spatiotemporal data using the target material data 220.

[0119] For example, when using paper data included in the material data 220, the AI ​​management unit 112 treats the impact factor of the academic journal in which the paper is published as index data and updates or tunes the generative AI 180. Here, the impact factor of the academic journal in which the paper is published may be referred to as the paper's impact factor. In other words, the paper's impact factor refers to the impact factor of the academic journal in which the paper is published. When multiple papers with different impact factors are input, the AI ​​management unit 112 may update, tune, or perform additional learning on the parameters of the generative AI 180 so that spatio-temporal data is generated with an emphasis on the content of papers with high impact factors. For example, for papers with high impact factors, the AI ​​management unit 112 updates, tunes, or performs additional learning on the generative AI 180 so that the paper data is treated as data with high importance during learning. For example, for papers with low impact factors, the AI ​​management unit 112 updates, tunes, or performs additional learning on the generative AI 180 so that the paper data is treated as data with low importance during learning. The AI ​​management unit 112 may update, tune, or perform additional learning on the generative AI 180 using a dataset containing multiple papers with different impact factors and spatiotemporal data generated with an emphasis on the content of associated papers with high impact factors.

[0120] For example, when using newspaper article data included in material data 220, AI management unit 112 updates and tunes generative AI 180 by treating the circulation of the newspaper in which the newspaper article is published as index data. When multiple newspaper articles with different circulations are input, AI management unit 112 may update, tune, or perform additional learning on the parameters of generative AI 180 so that spatiotemporal data is generated with an emphasis on the content of newspaper articles with larger circulations. For example, in the case of newspaper articles published in newspapers with larger circulations, AI management unit 112 updates, tunes, or performs additional learning on generative AI 180 so that the newspaper article data is treated as data with higher importance during learning. For example, in the case of newspaper articles published in newspapers with lower circulations, AI management unit 112 updates, tunes, or performs additional learning on generative AI 180 so that the newspaper article data is treated as data with lower importance during learning. The AI ​​management unit 112 may update, tune, or perform additional learning on the generative AI 180 using a dataset that includes multiple newspaper articles with different circulations and spatiotemporal data generated with an emphasis on the content of the newspaper articles with the largest circulations linked to the newspaper articles in which the articles are published.

[0121] It may be possible to select which indexes included in the material data 220 to use as weights. For example, an administrator of the processing server 100 or the like selects indexes to use as weights from among the multiple data included in the material data 220, and the AI ​​management unit 112 uses the selected indexes to update and tune the generative AI 180.

[0122] 7 shows an example of the flow of processing by the processing server 100. Here, an example of the flow of processing will be described in which the processing server 100 repeatedly generates past spatio-temporal data until approval from the user 200 is obtained.

[0123] In step (sometimes abbreviated as S) 102, the reception unit 104 receives material data provided by the user 200 and an instruction to generate past spatiotemporal data. The reception unit 104 stores the received material data and instruction in the storage unit 102. The reception unit 104 may also store the received material data and instruction in the DB server 190.

[0124] In S104, the spatio-temporal data generation unit 108 generates past spatio-temporal data from the material data using the generative AI 180. In S106, the spatio-temporal data provision unit 110 provides the past spatio-temporal data generated by the spatio-temporal data generation unit 108 in S104 to the user 200. If OK is obtained from the user 200, the process proceeds to S110.

[0125] If the result is NG, the process returns to S102. In this case, the spatiotemporal data generation unit 108 may request additional material data from the user 200. In response to the reception unit 104 receiving the additional material data, the spatiotemporal data generation unit 108 uses the generative AI 180 to generate past spatiotemporal data using the material data acquired up to that point and the additional material data.

[0126] In S110, the spatio-temporal data generation unit 108 registers the past spatio-temporal data approved by the user 200 in the storage unit 102. The spatio-temporal data generation unit 108 may also register the past spatio-temporal data in the DB server 190.

[0127] 8 shows an example of the flow of processing by the processing server 100. Here, an example of the flow of processing will be described, in which the processing server 100 generates spatial data for each of a plurality of time points while confirming with the user 200, and generates past spatio-temporal data using the plurality of spatial data.

[0128] In S202, the reception unit 104 receives the material data and the instruction to generate past spatio-temporal data provided by the user 200. The reception unit 104 stores the received material data and the instruction in the storage unit 102. The reception unit 104 may also store the received material data and the instruction in the DB server 190.

[0129] In S204, the material data acquisition unit 106 acquires a portion of the material data. The material data acquisition unit 106 acquires a portion of the material data corresponding to one of the multiple time points.

[0130] In S206, the spatiotemporal data generation unit 108 generates spatial data corresponding to one point in time from the material data acquired by the material data acquisition unit 106 in S204, using the generative AI 180. In S208, the spatiotemporal data provision unit 110 provides the user 200 with the spatial data generated by the spatiotemporal data generation unit 108 in S206.

[0131] If the spatial data needs to be corrected, the process proceeds to S212; if not, the process proceeds to S214. A case where correction is needed may be a case where a correction instruction is received from the user 200. The correction instruction may include an area to be corrected from among multiple areas included in the spatial data, and correction content indicating how the correction should be made. In S212, the spatiotemporal data generation unit 108 corrects the spatial data in accordance with the correction instruction. The spatiotemporal data generation unit 108 may input instruction data including the spatial data and the correction instruction to the generation system AI 180, and acquire the corrected spatial data.

[0132] If the generation of spatial data for all of the multiple time points for which images are included in the material data received in S202 has been completed (YES in S214), the process proceeds to S216, and if not completed, the process proceeds to S204. In S204, the material data acquisition unit 106 acquires a portion of the material data corresponding to the next time point among the multiple time points.

[0133] In S216, the spatio-temporal data generation unit 108 determines whether or not the spatial data needs to be complemented. When generating past spatio-temporal data, the spatio-temporal data generation unit 108 may determine whether or not the spatial data is sufficient. For example, if spatial data exists for all preset time intervals, the spatio-temporal data generation unit 108 determines that the spatial data is sufficient; otherwise, the spatio-temporal data generation unit 108 determines that the spatial data is insufficient. The preset time interval may be set in any unit, such as minutes, hours, or days. If it is determined that complementation is necessary, the process proceeds to S218; if it is not determined that complementation is necessary, the process proceeds to S220.

[0134] In S218, the spatio-temporal data generation unit 108 complements the spatial data using the generated spatial data for the multiple time points. For example, when the time interval between spatial data for consecutive time points is greater than a predetermined threshold, the spatio-temporal data generation unit 108 generates spatial data for a time point between the consecutive time points using the spatial data for the consecutive time points.

[0135] In S220, the spatio-temporal data generation unit 108 generates past spatio-temporal data using the generated spatial data and, if there is spatial data generated in S218, the spatial data. The spatio-temporal data generation unit 108 may input instruction data including the generated spatial data, the interpolated spatial data, and an instruction to generate past spatio-temporal data using these spatial data to the generation AI 180, and obtain the past spatio-temporal data from the generation AI 180.

[0136] 9 schematically illustrates an example of the hardware configuration of a computer 1200 functioning as the processing server 100. A program installed on the computer 1200 can cause the computer 1200 to function as one or more "parts" of an apparatus according to the present embodiment, or can cause the computer 1200 to perform operations associated with the apparatus according to the present embodiment or one or more "parts" thereof, and / or can cause the computer 1200 to perform a process according to the present embodiment or steps of the process. Such a program can be executed by the CPU 1212 to cause the computer 1200 to perform specific operations associated with some or all of the blocks in the flowcharts and block diagrams described herein.

[0137] The computer 1200 according to this embodiment includes a CPU 1212, a RAM 1214, and a graphics controller 1216, which are interconnected by a host controller 1210. The computer 1200 also includes input / output units such as a communications interface 1222, a storage device 1224, a DVD drive, and an IC card drive, which are connected to the host controller 1210 via an input / output controller 1220. The DVD drive may be a DVD-ROM drive, a DVD-RAM drive, or the like. The storage device 1224 may be a hard disk drive, a solid-state drive, or the like. The computer 1200 also includes a ROM 1230 and legacy input / output units such as a keyboard, which are connected to the input / output controller 1220 via an input / output chip 1240.

[0138] The CPU 1212 operates according to programs stored in the ROM 1230 and the RAM 1214, thereby controlling each unit. The graphics controller 1216 acquires image data generated by the CPU 1212 into a frame buffer or the like provided in the RAM 1214 or into the graphics controller itself, and causes the image data to be displayed on the display device 1218.

[0139] The communication interface 1222 communicates with other electronic devices via a network. The storage device 1224 stores programs and data used by the CPU 1212 in the computer 1200. The DVD drive reads programs or data from a DVD-ROM or the like and provides them to the storage device 1224. The IC card drive reads programs and data from an IC card and / or writes programs and data to an IC card.

[0140] The ROM 1230 stores therein a boot program or the like that is executed by the computer 1200 upon activation, and / or programs that depend on the hardware of the computer 1200. The input / output chip 1240 may also connect various input / output units to the input / output controller 1220 via a USB port, a parallel port, a serial port, a keyboard port, a mouse port, etc.

[0141] The programs are provided by a computer-readable storage medium such as a DVD-ROM or an IC card. The programs are read from the computer-readable storage medium, installed in the storage device 1224, RAM 1214, or ROM 1230, which are also examples of computer-readable storage media, and executed by the CPU 1212. Information processing described in these programs is read by the computer 1200, and causes cooperation between the programs and the various types of hardware resources described above. An apparatus or method may be configured by implementing operations or processing of information in accordance with the use of the computer 1200.

[0142] For example, when communication is performed between the computer 1200 and an external device, the CPU 1212 may execute a communication program loaded into the RAM 1214 and instruct the communication interface 1222 to perform communication processing based on the processing described in the communication program. Under the control of the CPU 1212, the communication interface 1222 reads transmission data stored in a transmission buffer area provided in the RAM 1214, the storage device 1224, a DVD-ROM, or a recording medium such as an IC card, and transmits the read transmission data to the network, or writes reception data received from the network to a reception buffer area or the like provided on the recording medium.

[0143] Furthermore, the CPU 1212 may cause all or a necessary portion of a file or database stored in an external recording medium such as the storage device 1224, a DVD drive (DVD-ROM), an IC card, etc. to be read into the RAM 1214, and may perform various types of processing on the data on the RAM 1214. The CPU 1212 may then write back the processed data to the external recording medium.

[0144] Various types of information, such as various types of programs, data, tables, and databases, may be stored on the recording medium and may undergo information processing. The CPU 1212 may perform various types of processing on data read from the RAM 1214, including various types of operations, information processing, conditional judgment, conditional branching, unconditional branching, information search / replacement, etc., as described throughout this disclosure and specified by the instruction sequences of the programs, and write the results back to the RAM 1214. The CPU 1212 may also search for information in a file, database, etc. on the recording medium. For example, if multiple entries, each having an attribute value of a first attribute associated with an attribute value of a second attribute, are stored on the recording medium, the CPU 1212 may search for an entry whose attribute value of the first attribute matches a specified condition from among the multiple entries, read the attribute value of the second attribute stored in the entry, and thereby obtain the attribute value of the second attribute associated with the first attribute that satisfies a predetermined condition.

[0145] The above-described programs or software modules may be stored in a computer-readable storage medium on or near the computer 1200. A recording medium such as a hard disk or RAM provided in a server system connected to a dedicated communication network or the Internet can also be used as a computer-readable storage medium, thereby providing the programs to the computer 1200 via the network.

[0146] The blocks in the flowcharts and block diagrams in the present embodiments may represent stages of a process in which an operation is performed or "parts" of an apparatus responsible for performing the operation. Particular stages and "parts" may be implemented by dedicated circuitry, programmable circuitry provided with computer-readable instructions stored on a computer-readable storage medium, and / or a processor provided with computer-readable instructions stored on a computer-readable storage medium. The dedicated circuitry may include digital and / or analog hardware circuits, including integrated circuits (ICs) and / or discrete circuits. The programmable circuitry may include reconfigurable hardware circuits, such as field programmable gate arrays (FPGAs) and programmable logic arrays (PLAs), including AND, OR, XOR, NAND, NOR, and other logical operations, flip-flops, registers, and memory elements.

[0147] A computer-readable storage medium may include any tangible device capable of storing instructions that are executed by an appropriate device, such that a computer-readable storage medium having instructions stored thereon comprises an article of manufacture, including instructions that can be executed to create means for performing the operations specified in the flowcharts or block diagrams. Examples of computer-readable storage media may include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, etc. More specific examples of computer-readable storage media may include floppy disks, diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), electrically erasable programmable read-only memory (EEPROM), static random access memory (SRAM), compact disc read-only memory (CD-ROM), digital versatile disc (DVD), Blu-ray disc, memory stick, integrated circuit card, etc.

[0148] The computer readable instructions may include either assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state-setting data, or source or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk®, JAVA®, C++, etc., and conventional procedural programming languages ​​such as the “C” programming language or similar programming languages.

[0149] Computer-readable instructions may be provided to a general-purpose computer, a special-purpose computer, or another programmable data processing device, or a programmable circuit, either locally or via a local area network (LAN), a wide area network (WAN) such as the Internet, so that the processor of the programmable data processing device, such as a computer, or the programmable circuit executes the computer-readable instructions to generate means for performing the operations specified in the flowcharts or block diagrams. Here, the computer may be a personal computer (PC), a tablet computer, a smartphone, a workstation, a server computer, a general-purpose computer, a special-purpose computer, or the like, or may be a computer system in which multiple computers are connected. Such a computer system in which multiple computers are connected is also called a distributed computing system, and is a broad definition of computers. In a distributed computing system, multiple computers collectively execute a program by each executing a portion of the program and passing data between the computers as needed during program execution.

[0150] Examples of processors include computer processors, central processing units, processing units, microprocessors, digital signal processors, controllers, microcontrollers, etc. A computer may have one or more processors. In a multiprocessor system with multiple processors, each processor executes a portion of a program and passes data between processors as needed during program execution, allowing the multiple processors to collectively execute the program. For example, in multitasking, each of the multiple processors may execute a portion of each task in small chunks by switching tasks at time slice intervals. In this case, which portion of a program each processor executes changes dynamically. Which portion of a program each of the multiple processors executes may also be statically determined by multiprocessor-aware programming.

[0151] Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It will be apparent to those skilled in the art that various modifications and improvements can be made to the above embodiments. It is clear from the claims that such modifications and improvements can also be included within the technical scope of the present invention.

[0152] It should be noted that the execution order of each process, such as operations, procedures, steps, and stages, in the devices, systems, programs, and methods shown in the claims, specifications, and drawings is not specifically stated as "before," "prior to," etc., and that the processes can be performed in any order unless the output of a previous process is used in a later process. Even if the operational flow in the claims, specifications, and drawings is described using "first," "next," etc. for convenience, this does not mean that the processes must be performed in this order. [Explanation of symbols]

[0153] 10 processing system, 50 network, 100 processing server, 102 memory unit, 104 reception unit, 106 material data acquisition unit, 108 spatiotemporal data generation unit, 110 spatiotemporal data provision unit, 112 AI management unit, 180 generative AI, 190 DB server, 200 user, 210 material data, 220 material data, 300 communication terminal, 400 eyewear, 502 spatial data, 504 spatial data, 508 spatial data, 512 spatial data, 514 spatial data, 518 spatial data, 1200 computer, 1210 host controller, 1212 CPU, 1214 RAM, 1216 graphics controller, 1218 display device, 1220 input / output controller, 1222 communication interface, 1224 storage device, 1230 ROM, 1240 input / output chip

Claims

1. a material data acquisition unit that acquires material data provided by a user, the material data including captured images captured at points where the user was located at each of a plurality of time points; a spatio-temporal data generating unit that generates past spatio-temporal data representing the past space-time of the user, the past space-time data being composed of spatial data in which the user was located at a plurality of points in time, based on the material data; A processing server comprising:

2. The processing server of claim 1, wherein the spatiotemporal data generation unit generates spatial data representing the situation of the location where the user was located at each of the multiple points in time using the multiple captured images included in the material data, generates instruction data including instructions to generate the past spatiotemporal data using the multiple generated spatial data, and generates the past spatiotemporal data by inputting the instruction data to a generation AI.

3. The processing server of claim 2, wherein the spatiotemporal data generation unit generates spatial data representing the points where the user was located at each of the multiple time points using the multiple captured images included in the material data, generates spatial data for a time point between the multiple time points using the spatial data for consecutive time points among the multiple time points, generates the instruction data including an instruction to generate the past spatiotemporal data using the multiple generated spatial data, and inputs the instruction data to the generation AI.

4. the material data further includes map data including information on the location; The processing server according to claim 2, wherein the spatiotemporal data generation unit generates the spatial data representing the locations where the user was located at each of the multiple points in time using the multiple captured images and the map data included in the material data, generates the instruction data including an instruction to generate the past spatiotemporal data using the multiple generated spatial data, and inputs the instruction data to the generation AI.

5. the material data further includes diary data including diaries at the plurality of points in time; The processing server of claim 2, wherein the spatiotemporal data generation unit generates spatial data representing the locations where the user was located at each of the multiple time points using the multiple captured images included in the material data, estimates the user's level of interest in multiple areas within the spatial data for each of the multiple spatial data using the diary data, corrects the spatial data according to the estimated level of interest, and then generates the instruction data including instructions to generate the past spatiotemporal data using the multiple spatial data, and inputs the instruction data to the generation AI.

6. The processing server according to claim 1 , wherein the spatiotemporal data generation unit estimates the user's interests based on the material data, and generates future spatiotemporal data representing the user's future space-time based on the estimated interests.

7. a spatio-temporal data providing unit that provides the future spatio-temporal data of the user generated by the spatio-temporal data generating unit to other users approved by the user; The processing server of claim 6, comprising:

8. a spatiotemporal data providing unit that provides the past spatiotemporal data of the user generated by the spatiotemporal data generating unit to other users approved by the user; The processing server according to claim 1 , comprising:

9. the material data acquisition unit acquires object material data including images at a plurality of times related to a specific object and at least one of a paper, research material, newspaper article, and minutes related to the object; The processing server according to claim 1 , wherein the spatiotemporal data generation unit generates, based on the object material data, object past spatiotemporal data that represents a past space-time of the object and is composed of spatial data of the object at a plurality of points in time.

10. a material data acquisition unit that acquires material data including images at a plurality of times related to a specific object and at least one of papers, research materials, newspaper articles, and minutes related to the object; a spatio-temporal data generating unit that generates past spatio-temporal data representing the past space-time of the object, the past space-time data being composed of spatial data of the object at a plurality of points in time, based on the raw data; A processing server comprising:

11. The processing server according to claim 10, wherein the spatiotemporal data generation unit generates the spatial data for each of the multiple points in time using the multiple images included in the material data and at least one of the papers, research materials, newspaper articles, and minutes included in the material data, generates instruction data including instructions to generate the past spatiotemporal data using the generated multiple spatial data, and generates the past spatiotemporal data by inputting the instruction data into a generation AI.

12. The processing server according to claim 11, wherein the spatiotemporal data generation unit generates the spatial data for each of the plurality of points in time using the plurality of images included in the material data and at least one of the papers, the research materials, the newspaper articles, and the minutes included in the material data, generates spatial data for a point in time between the consecutive points in time using the spatial data for the consecutive points in time among the plurality of points in time, generates the instruction data including an instruction to generate the past spatiotemporal data using the generated plurality of spatial data, and inputs the instruction data to the generation AI.

13. 13. The processing server according to claim 10, wherein the spatiotemporal data generation unit identifies recurring events in the history of the object based on at least one of the papers, the research materials, the newspaper articles, and the minutes included in the material data, and generates future spatiotemporal data representing the future space-time of the object based on the identified events.

14. The processing server according to claim 13 , wherein the spatiotemporal data generation unit generates a plurality of pieces of future spatiotemporal data, estimates the user's interests based on user-related information related to the user who will experience any of the plurality of pieces of future spatiotemporal data, and determines the future spatiotemporal data to be experienced by the user from the plurality of future spatiotemporal data based on the estimated interests.

15. The processing server according to any one of claims 10 to 12, wherein the spatiotemporal data generation unit estimates the user's interests based on user-related information related to the user who experiences the generated past spatiotemporal data, and corrects the past spatiotemporal data based on the estimated interests.

16. 1. A computer-implemented information processing method, comprising: a material data acquisition step of acquiring material data including captured images captured at points where the user was located at each of a plurality of time points provided by the user; a spatio-temporal data generation step of generating past spatio-temporal data representing the past space-time of the user, the past space-time data being composed of spatial data in which the user was located at a plurality of points in time, based on the material data; A processing method comprising:

17. 1. A computer-implemented information processing method, comprising: a material data acquisition stage for acquiring material data including images at a plurality of time points related to a specific subject and at least one of papers, research materials, newspaper articles, and minutes related to the subject; a spatio-temporal data generation step of generating past spatio-temporal data representing the past space-time of the object, the past spatio-temporal data being composed of spatial data of the object at a plurality of points in time, based on the raw data; A processing method comprising:

18. A program for causing a computer to execute the processing method according to claim 16 or 17.

Images