system

Abstract

A system is provided.SOLUTION: A system comprising: an uploading means for a user to upload information about a scene or an event to a cloud; a verification means using a generated AI to check authenticity, morale, and privacy of the uploaded information; a categorization means for categorizing and tagging the verified information; an integration means for integrating the categorized information and generating a spatial experience using XR technology; and a providing means for providing the data and XR content hosted on the cloud to the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] As cityscapes are constantly changing due to urban development, earthquakes, and other factors, recording and preserving these changes is important for preserving history and culture. However, with existing technologies, individual data is often stored and organized manually, and checks for accuracy, morality, and privacy are insufficient, reducing the reliability and usefulness of the data. Furthermore, there is a lack of a unified platform for recreating local landscapes and events in virtual reality. This makes it difficult to effectively preserve a city's history and culture and share it with many people. [Means for solving the problem]

[0005] The present invention provides a system that includes an uploading means for users to upload data related to scenery and events to the cloud; a verification means using generation AI to check the authenticity, morality, and privacy of the uploaded data; a classification means for categorizing and tagging the verified data; an integration means for integrating the classified data and generating a spatial experience using XR technology; and a provision means for providing users with data and XR content hosted on the cloud. This system accurately preserves the history and culture of a city, promotes tourism, and strengthens ties between local communities. Furthermore, it addresses privacy and moral issues by using image analysis technology to detect and correct personal information and inappropriate content in the data. Furthermore, it enables users to effectively experience cityscapes and events by delivering content in real time in response to user requests and generating realistic spatial experiences.

[0006] The "uploading means" is a function that allows users to transfer data related to scenery and events to the cloud server.

[0007] "Verification measures" are features that use generative AI to check the authenticity, morality, and privacy of uploaded data.

[0008] The "classification means" is a function that classifies verified data into categories and organizes the data by assigning appropriate tags.

[0009] "Integration means" is a function that integrates classified data and generates a realistic spatial experience using XR technology.

[0010] "Means of provision" refers to the function of delivering data and XR content hosted on the cloud to users and allowing them to experience it.

[0011] "Generative AI" is a system that uses artificial intelligence technology to verify the authenticity of data, conduct moral checks, and perform privacy checks.

[0012] "XR technology" refers to technologies that include Augmented Reality, Virtual Reality, and Mixed Reality, and is a technology that combines digital content with the real world to provide a spatial experience.

[0013] A "cloud server" is a collection of remote servers that provide data and services over the Internet.

[0014] "Tagging" is the process of assigning relevant keywords or labels to data to make it easier to search and categorize.

[0015] "Image analysis technology" is a technology that analyzes digital images to detect and identify specific patterns and information.

[0016] "Hosting" is a service that runs data and applications on a server and makes them accessible via the Internet. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] This invention is a system that allows users to upload data about cityscapes and events to the cloud, which is then verified and organized by generative AI, and uses XR technology to provide a realistic spatial experience.

[0039] System program processing

[0040] 1. Upload your data

[0041] Users: Upload photos, videos, and audio data related to cityscapes and events to the cloud using their smartphones or PCs.

[0042] Example: A user uploads photos and videos of a local festival from their smartphone to the cloud.

[0043] 2. Data Verification

[0044] Server: Uses generative AI to verify the uploaded data, checking for authenticity, inappropriate content, and personal information.

[0045] Example: The server automatically detects and blurs the faces of individuals in uploaded festival photos.

[0046] 3. Data Classification and Tagging

[0047] Server: Categorizes the validated data and assigns appropriate tags, making it easier to search and organize.

[0048] For example, add tags such as "2023 Autumn Festival" and "Fireworks Display" to make it easier for users to find what they're looking for.

[0049] 4. Building an XR spatial experience

[0050] Server: Integrates the classified data and generates a realistic spatial experience using XR technology, including various data such as photos, videos, audio, and text information.

[0051] Example: A user uses a VR headset to experience a virtual recreation of a festival venue.

[0052] 5. Content Provision

[0053] Server: Provides users with data and XR content hosted on the cloud. Users can access this data via the Internet.

[0054] Example: Tourists can view and experience information and experiences on the cloud using their smartphones or PCs.

[0055] Specific examples

[0056] Example of data upload

[0057] Users upload photos and videos of local autumn festivals from their smartphones to the cloud, where the cloud server receives and temporarily stores them.

[0058] Specific examples of verification

[0059] The server uses generative AI to analyze uploaded photos of the autumn festival, identify faces in the images, and apply blurring to them. It also compares the event details with an existing database to confirm accuracy.

[0060] Examples of classification and tagging

[0061] Once the data has been verified, the server automatically assigns tags such as "2023 Autumn Festival" or "Fireworks Display." This classification makes it easy for users to search for data.

[0062] Specific examples of XR spatial experiences

[0063] The server integrates the classified photos, videos, and audio data and uses XR technology to generate a virtual space, allowing users to experience the festival venue in a realistic way using a VR headset.

[0064] Examples of content provision

[0065] Users access the cloud-based platform as tourists and enjoy information about the autumn festival and XR experiences. Using content provided by the server, users can take a virtual tour before actually going sightseeing.

[0066] This system effectively preserves and shares the city's history and culture, promoting tourism and strengthening ties with the local community. The integration of generative AI verification and XR technology enables the provision of high-quality content.

[0067] The processing flow will be explained below.

[0068] Step 1:

[0069] The user opens the application on their device (smartphone or PC) and selects photos, videos, and audio data related to cityscapes and events.

[0070] Specific operation: The user selects multiple image or video files through the file selection dialog on the device and clicks the upload button.

[0071] Step 2:

[0072] The terminal transmits the selected data to the cloud server.

[0073] What happens: The device compresses the selected photo or video data and sends it to the server using a secure protocol (e.g., HTTPS).

[0074] Step 3:

[0075] The server receives the uploaded data and stores it temporarily.

[0076] Specific operation: The server stores the uploaded data in a temporary storage area and records the data's meta information (upload date and time, user ID, etc.) in a database.

[0077] Step 4:

[0078] The server begins validating the uploaded data using the generated AI.

[0079] How it works: The server applies image analysis algorithms to detect personal information and inappropriate content in the data, automatically blurring out faces and displaying a warning if the data contains content that violates public order and morals.

[0080] Step 5:

[0081] The server verifies the authenticity of the uploaded data.

[0082] Specific operation: The server checks the event information in the data against the existing database to determine the reliability of the data. If the data is unreliable, the user is asked to reconfirm it.

[0083] Step 6:

[0084] The server categorizes the verified data and assigns appropriate tags.

[0085] How it works: The server analyzes the data and automatically categorizes it into categories such as landscape photos, event videos, audio guides, etc. It also assigns tags based on the event name and era.

[0086] Step 7:

[0087] The server integrates the classified data and generates a realistic spatial experience using XR technology.

[0088] How it works: The server uses 3D modeling software to create a virtual space, then places photos and videos into the 3D space. Audio data is synchronized appropriately, enhancing the sense of realism.

[0089] Step 8:

[0090] The server provides users with data and XR content hosted on the cloud.

[0091] Specific operation: The server makes the XR content publicly available through a hosting service, allowing users to access it through a VR headset or browser, and generates a link or QR code to access the data.

[0092] Step 9:

[0093] The user accesses the cloud-based platform and begins the experience by using the provided resources.

[0094] Specific operation: The user launches a browser or VR headset, accesses the URL, logs in to the XR content, begins the experience in the virtual space, and uses the navigation function to explore various content.

[0095] This series of processing steps will enable a system that allows users to easily upload information about a city's history and culture to the cloud and share it with other users as a realistic spatial experience. Verification by generative AI and the integration of XR technology will enable the provision of safe, high-quality content.

[0096] Example 1

[0097] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0098] In today's world, it has become common to upload data related to cityscapes and events to the cloud and share it with many people. However, the sheer volume of this data makes managing and organizing it difficult. Furthermore, uploaded data may contain privacy violations or inappropriate content, and manually reviewing it takes time and effort. Furthermore, generating high-quality XR spatial experiences requires appropriate data integration and real-time content delivery. The present invention aims to solve these problems and realize efficient, high-quality data management and the provision of XR spatial experiences.

[0099] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0100] In this invention, the server includes a means for users to upload data to the cloud server using a dedicated application or web browser and temporarily store the uploaded data, an image analysis means for automatically detecting personal information using generation AI and modifying or deleting it, and an XR space generation means for integrating photos, videos, and audio data and generating a virtual space using XR technology. This makes it possible to efficiently manage data related to cityscapes and events, and provide users with a high-quality XR space experience while protecting their privacy.

[0101] The "uploading means" is a means by which a user uploads data about scenery or events onto the cloud.

[0102] "Verification measures" are measures using generative AI to check the authenticity, morality, and privacy of uploaded data.

[0103] The "classification means" is a means for classifying and tagging verified data into categories.

[0104] "Integration means" refers to a means for integrating classified data and generating a spatial experience using XR technology.

[0105] "Provision means" refers to the means for providing data and XR content hosted on the cloud to users.

[0106] The "temporary storage means" is a means by which a user uploads data to a cloud server using a dedicated application or a web browser, and temporarily stores the uploaded data.

[0107] "Image analysis means" refers to a means of automatically detecting personal information using generative AI and correcting or deleting it.

[0108] An "XR space generation means" is a means for integrating photographs, videos, and audio data to generate a virtual space using XR technology.

[0109] "Authentication means" means a means for managing user access to a platform on the cloud.

[0110] This invention is a system that allows users to upload data related to cityscapes and events to the cloud, which is then verified and organized by a generative AI, and uses XR technology to provide a realistic spatial experience. Below we will explain how this system is implemented in concrete terms.

[0111] 1. Upload your data

[0112] Users use devices such as smartphones or PCs to collect photos, videos, and audio data related to cityscapes and events. This data is then uploaded to the system's cloud server via a dedicated application or web browser. Commonly used cloud platforms (e.g., Amazon S3 or Google Cloud Storage) are used for cloud storage.

[0113] Example: A user uploads photos and videos of a local autumn festival to the cloud using a smartphone app.

[0114] 2. Data Verification

[0115] The cloud server temporarily stores the received data and then verifies it using a generative AI model. This verification process includes verifying the authenticity of the data, checking morality, and detecting and correcting personal information to protect privacy. Image analysis technology is used to detect inappropriate content and personal information (e.g., faces and names) and to correct them as necessary (e.g., by mosaic processing).

[0116] Example: The server analyzes photos from an autumn festival, automatically detects individuals' faces in the images, and applies mosaic processing.

[0117] 3. Data Classification and Tagging

[0118] Once verified, the data is categorized by a generative AI model. The model analyzes the content of photos, videos, audio, and other data, and assigns appropriate tags (e.g., "2023 Autumn Festival" or "Fireworks Display"). This categorization and tagging makes it easier to search and organize the data.

[0119] Example: The server assigns tags such as "Autumn Festival 2023" and "Fireworks Display" to autumn festival data.

[0120] 4. Building an XR spatial experience

[0121] The classified data is integrated using XR technology to generate a realistic spatial experience. This includes photos, videos, audio data, and more, each of which is appropriately positioned within the virtual space. Development platforms such as Unity and Unreal Engine are used to generate the XR space.

[0122] Example: The server virtually recreates the venue of an autumn festival and provides a virtual space that users can experience using a VR headset.

[0123] 5. Content Provision

[0124] The generated XR content is hosted on the cloud and provided to users via the Internet. Users can access this data and spatial experiences using dedicated applications or web browsers. User access is managed through authentication methods.

[0125] Example: A user accesses the cloud-based platform and enjoys the provided XR experience of the Autumn Festival.

[0126] This system will efficiently preserve and share the city's history and culture, promote tourism, and strengthen ties with the local community. Furthermore, the integration of generative AI data verification and XR technology will enable the provision of high-quality content.

[0127] An example prompt might be, "Upload a photo of the autumn festival, perform facial recognition and mosaic processing, and generate a virtual space using XR technology."

[0128] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0129] Step 1: Collect and upload data

[0130] Users use devices such as smartphones or PCs to take photos, videos, and audio data of cityscapes and events.Then, they upload the data to the cloud using a dedicated application or web browser.The input includes raw data such as the photos and videos taken, and the output is the uploaded data stored in the cloud.

[0131] Specific operation: The user takes photos or videos of the autumn festival using the smartphone camera, opens the dedicated app and presses the "upload" button.

[0132] Step 2: Receiving and temporarily storing data

[0133] The server receives data uploaded by users and temporarily stores it in cloud storage. The input is photos, videos, and audio data uploaded by users, and the output is the data temporarily stored in cloud storage.

[0134] Specific operation: The server receives the upload request, receives the data, and stores it in cloud storage such as Amazon S3.

[0135] Step 3: Validate the data

[0136] The server is a process that verifies the uploaded data using a generative AI model. Here, data authenticity is verified, morality is checked, and privacy protection (detection and correction of personal information) is performed. The input is temporarily stored data, and the output is verified and corrected data.

[0137] Specific operation: The server uses generated AI to analyze photos of the autumn festival, automatically detect individual faces, and apply mosaic processing.

[0138] Step 4: Classify and tag your data

[0139] Once verified, the data is classified by category and appropriately tagged by a generative AI model. This process uses image recognition and natural language processing techniques. The input is verified data, and the output is classified and tagged data.

[0140] Specific operation: The server classifies photos and videos of autumn festivals into categories such as "Autumn Festival 2023" and "Fireworks Display," and automatically assigns corresponding tags.

[0141] Step 5: Build an XR spatial experience

[0142] The classified data is integrated to generate a realistic spatial experience using XR technology. This process involves integrating photo, video, and audio data with 3D modeling and CG. The input is classified and tagged data, and the output is the XR spatial experience content.

[0143] Specific operation: The server integrates various data and uses Unity and Unreal Engine to virtually recreate the autumn festival venue.

[0144] Step 6: Providing content

[0145] The generated XR content is hosted on the cloud and provided to users via the Internet. Users can access this data and spatial experiences through dedicated applications or web browsers. The input is the generated XR content, and the output is content hosted on the cloud that is accessible to users.

[0146] How it works: The server hosts the XR spatial experience on the cloud, and the user accesses it using a dedicated app to begin the experience.

[0147] These steps will enable efficient management of cityscape and event data, enabling users to enjoy high-quality XR spatial experiences.

[0148] (Application example 1)

[0149] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0150] Conventional tourist guide systems have problems such as improper classification of information provided by tourists and local residents, and a high risk of inappropriate content or personal information being leaked. Furthermore, there was a lack of technology for providing real-time tourist information, making it difficult to provide tourists with appropriate information immediately. Furthermore, there were limited methods for integrating a large amount of data to provide a virtual space experience, making it difficult to provide users with a satisfactory virtual experience. To solve these problems, there was a need to build a highly functional system incorporating new technologies.

[0151] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0152] In this invention, the server includes an uploading unit that allows users to upload data related to scenery and events to the cloud, a verification unit using a generation AI to check the authenticity, morality, and privacy of the uploaded data, a classification unit that classifies the verified data into categories and tags them, an integration unit that integrates the classified data and generates a spatial experience using XR technology, a provision unit that provides users with the data and XR content hosted on the cloud, a utilization unit that allows users to enjoy the XR experience in real time using a smartphone or smart glasses, and a tagging assistant unit that automatically tags tourist spots and events when uploading data. This enables information provided by tourists and local residents to be quickly and accurately classified and safely provided, and further enables tourists to enjoy high-quality XR experiences in real time.

[0153] The "uploading means" is a device or program that allows a user to upload data related to scenery or events onto the cloud.

[0154] "Verification means" is a device or program that uses generative AI to check the authenticity, morality, and privacy of uploaded data.

[0155] The "classification means" is a device or program for classifying and tagging verified data into categories.

[0156] "Integration means" refers to a device or program for integrating classified data and generating a spatial experience using XR technology.

[0157] "Providing means" refers to a device or program for providing users with data and XR content hosted on the cloud.

[0158] "Means of use" refers to a device or program that allows a user to experience an XR experience in real time using a smartphone or smart glasses.

[0159] The "tagging assistant" is a device or program that automatically tags tourist spots and events when uploading data.

[0160] "Generative AI" refers to artificial intelligence technology used to verify uploaded data.

[0161] The "cloud" is a network-based platform for providing computing resources and storage over the Internet.

[0162] "XR technology" is a comprehensive extended reality technology that includes virtual reality (VR), augmented reality (AR), and mixed reality (MR).

[0163] "Data" refers to digital information such as photos, videos, and audio related to landscapes and events.

[0164] "Tourist spots" refer to places and famous places that tourists visit.

[0165] An "event" refers to a festival or event that is held on a specific date and time.

[0166] "Personal information" is information that can identify a specific individual.

[0167] "Means of use" refers to a device or program that allows a user to experience an XR experience in real time using a smartphone or smart glasses.

[0168] These definitions provide a clear understanding of key terms in the claims and provide a concrete explanation of how the invention works.

[0169] In the embodiment of the present invention, a system is constructed that quickly and accurately classifies and safely provides tourist information provided by tourists and local residents. The specific contents of the system program and its processing are described below.

[0170] This system uses the following equipment and software:

[0171] Hardware

[0172] Smartphone: Users upload data about scenery and events and use the XR experience.

[0173] Smart glasses: A device that allows users to enjoy XR experiences in real time.

[0174] Server: The main processing unit that validates, categorizes, consolidates, and serves data.

[0175] software

[0176] Generative AI: Artificial intelligence technology that checks data for authenticity, morality, and privacy.

[0177] Cloud services (e.g., Amazon Web Services (AWS)): Data storage and high-performance computing.

[0178] XR development platform (e.g. Unity, Unreal Engine): Building virtual spaces.

[0179] Program processing explanation

[0180] Uploading data

[0181] Users use their smartphones to upload photos, videos, and audio data related to landscapes and events to the cloud, where the cloud service receives and temporarily stores the data.

[0182] Data validation

[0183] The server uses generative AI to verify the uploaded data, confirming its authenticity and detecting inappropriate content or personal information, and then correcting or deleting it. For example, it can automatically detect faces in images and apply blurring to them.

[0184] Data classification and tagging

[0185] The server categorizes the verified data and assigns appropriate tags. For example, tags such as "XX Castle" and "2023 Summer Festival" are automatically assigned. This makes it easier for users to search for data.

[0186] Building an XR spatial experience

[0187] The server integrates the classified data and generates a realistic spatial experience using XR technology. This includes various data such as photos, videos, audio, and text information. For example, users can use a VR headset to experience a virtual reproduction of a festival venue.

[0188] Content provision

[0189] The server provides users with data and XR content hosted on the cloud. Users can experience information about tourist spots and events in real time using their smartphones or smart glasses. Data and XR content can also be delivered in real time upon user request.

[0190] Specific examples

[0191] For example, if a user uploads photos or videos of a local festival from their smartphone to the cloud, the server receives and verifies them. Generative AI then blurs out private information (such as faces) in the images and automatically adds tags such as "XX Festival" or "2023." Unity is then used to build a virtual space, allowing users to experience the festival venue in real time through smart glasses.

[0192] Prompt Sentence Examples

[0193] Data uploaded by tourists and local residents must include the following: The data content must identify which tourist spot or event it relates to and be tagged appropriately.

[0194] Data includes images, videos, and audio. Please review each data and detect any inappropriate content or personal information and delete or blur it. Furthermore, please classify the data and use XR technology to generate a virtual space.

[0195] The above is an embodiment of the present invention. This system makes it possible to quickly and accurately process and safely provide information provided by tourists and local residents. Furthermore, the virtual space experience using XR technology can provide users with a new way to enjoy sightseeing.

[0196] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0197] Step 1:

[0198] Users upload data about scenery and events from their smartphones to the cloud.

[0199] Input: Photos, videos, and audio data of scenery and events

[0200] Specific operation: Use the smartphone application to select data and press the button to upload to cloud storage.

[0201] Output: Data stored in the cloud

[0202] Step 2:

[0203] The server receives the uploaded data and stores it temporarily.

[0204] Input: Data uploaded to the cloud

[0205] Specific behavior: Receives and securely stores data using a cloud service (e.g., Amazon Web Services).

[0206] Output: Data waiting for validation

[0207] Step 3:

[0208] The server validates the uploaded data using generated AI.

[0209] Input: Data waiting for validation

[0210] Specific operation: Generative AI models are used to check the authenticity, morality, and privacy of data. Specifically, image analysis technology is used to perform facial recognition and blur personal information.

[0211] Output: Verification completed data

[0212] Step 4:

[0213] The server categorizes the verified data and assigns appropriate tags.

[0214] Input: Verification completed data

[0215] Specific operation: Analyzes the data content and automatically generates and assigns tags such as "XX Castle" and "2023 Summer Festival."

[0216] Output: Classified and tagged data

[0217] Step 5:

[0218] The server integrates the classified data and generates a spatial experience using XR technology.

[0219] Input: Classified and tagged data

[0220] How it works: Using an XR development platform (e.g., Unity, Unreal Engine), a virtual space is created, which aggregates photos, videos, audio, and text information.

[0221] Output: Generated XR spatial experience data

[0222] Step 6:

[0223] The server provides users with XR content hosted on the cloud.

[0224] Input: Generated XR spatial experience data

[0225] Specific operation: Data is streamed in real time to the user's smartphone or smart glasses via a cloud service. The user operates the application and starts the XR experience.

[0226] Output: XR content played on the user's device

[0227] Step 7:

[0228] Users can access the XR experience in real time using a smartphone or smart glasses.

[0229] Input: XR content deployed on the user's device

[0230] Specific operation: The user wears the device and uses the application to control the XR experience and virtually experience tourist attractions and events.

[0231] Output: Rich user experience feedback

[0232] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0233] This invention is a system in which users upload data about landscapes and events to the cloud, which is then verified and organized by a generative AI, and tagged and classified based on the results of an emotion engine's analysis of the user's emotions, and then uses XR technology to provide a realistic spatial experience.

[0234] System program processing

[0235] 1. Upload your data

[0236] Users: Upload photos, videos, and audio data related to cityscapes and events to the cloud using their smartphones or PCs.

[0237] Example: A user uploads photos and videos of a local festival from their smartphone to the cloud.

[0238] 2. Data Verification

[0239] Server: Uses generative AI to verify the uploaded data, checking for authenticity, inappropriate content, and personal information.

[0240] Example: The server automatically detects and blurs the faces of individuals in uploaded festival photos.

[0241] 3. User Emotion Recognition

[0242] Server: Uses an emotion engine to recognize the user's emotion related to the uploaded data. Emotions are identified from the user's facial expressions and voice analysis.

[0243] Example: The server analyzes the user's emotions from the audio of the video and assigns emotion tags such as "joy" or "surprise."

[0244] 4. Data Classification and Tagging

[0245] Server: Based on the verified data and recognized emotion data, the data is classified into categories and assigned appropriate tags.

[0246] For example, in addition to event tags such as "2023 Autumn Festival" and "Fireworks Display," emotion tags such as "Fun" and "Emotional" are added.

[0247] 5. Building an XR spatial experience

[0248] Server: Integrates the classified data and emotional information, and generates a realistic spatial experience using XR technology. The content and presentation method are adjusted according to the user's emotions.

[0249] Example: A user uses a VR headset to experience a virtual festival venue, with content customized based on sentiment analysis.

[0250] 6. Content Provision

[0251] Server: Provides users with data and XR content hosted on the cloud. Users can access this data via the Internet.

[0252] Example: Tourists browse information and experiences on the cloud using their smartphones or PCs and enjoy emotion-based experiences.

[0253] Specific examples

[0254] Example of data upload

[0255] Users upload photos and videos of local autumn festivals from their smartphones to the cloud, where the cloud server receives and temporarily stores them.

[0256] Specific examples of verification

[0257] The server uses generative AI to analyze uploaded photos of the autumn festival, identify faces in the images, and apply blurring to them. It also compares the event details with an existing database to confirm accuracy.

[0258] Specific examples of emotion recognition

[0259] The emotion engine analyzes the audio of videos uploaded by users and detects their "happiness" or "excitement." Based on this, emotion tags are assigned to the data.

[0260] Examples of classification and tagging

[0261] Once verification and emotion recognition are complete, the server automatically assigns event tags such as "2023 Autumn Festival" and "Fireworks Display" and emotion tags such as "Fun" and "Emotional," making it easier for users to search for data.

[0262] Specific examples of XR spatial experiences

[0263] The server integrates the classified photos, videos, audio data, and emotional information to generate a virtual space using XR technology. Users can then use a VR headset to experience the festival venue in a realistic way. Based on the emotional analysis, the user's experience is customized, highlighting fun and moving scenes.

[0264] Examples of content provision

[0265] Users access the cloud-based platform as tourists and enjoy information about the autumn festival and XR experiences. The content provided by the server is customized based on emotional information, providing a more personalized experience.

[0266] The system effectively preserves and shares the city's history and culture, promoting tourism and strengthening ties with the local community. The integration of generative AI and emotion engine validation with XR technology enables the provision of safe, high-quality content.

[0267] The processing flow will be explained below.

[0268] Step 1:

[0269] The user opens the application on their device (smartphone or PC) and selects photos, videos, and audio data related to cityscapes and events.

[0270] Specific operation: The user selects multiple image or video files through the file selection dialog on the device and clicks the upload button.

[0271] Step 2:

[0272] The terminal transmits the selected data to the cloud server.

[0273] What happens: The device compresses the selected photo or video data and sends it to the server using a secure protocol (e.g., HTTPS).

[0274] Step 3:

[0275] The server receives the uploaded data and stores it temporarily.

[0276] Specific operation: The server stores the uploaded data in a temporary storage area and records the data's meta information (upload date and time, user ID, etc.) in a database.

[0277] Step 4:

[0278] The server begins validating the uploaded data using the generated AI.

[0279] How it works: The server applies image analysis algorithms to detect personal information and inappropriate content in the data, automatically blurring out faces and displaying a warning if the data contains content that violates public order and morals.

[0280] Step 5:

[0281] The server verifies the authenticity of the uploaded data.

[0282] Specific operation: The server checks the event information in the data against the existing database to determine the reliability of the data. If the data is unreliable, the user is asked to reconfirm it.

[0283] Step 6:

[0284] The server categorizes the verified data and assigns appropriate tags.

[0285] How it works: The server analyzes the data and automatically categorizes it into categories such as landscape photos, event videos, audio guides, etc. It also assigns tags based on the event name and era.

[0286] Step 7:

[0287] The server uses an emotion engine to recognize the user's emotion.

[0288] Specific operation: The server analyzes the user's facial expressions and voice in uploaded photos and videos to identify emotions such as "happiness" or "surprise." Based on this, an emotion tag is assigned.

[0289] Step 8:

[0290] The server adjusts the content and presentation of the data based on the emotional data.

[0291] Specific operation: The server optimizes the order of data placement and emphasis points by taking into account the emotion data. For example, scenes in which the user expressed the emotion of "surprise" are displayed preferentially.

[0292] Step 9:

[0293] The server integrates the classified data and emotional information and generates a realistic spatial experience using XR technology.

[0294] How it works: The server uses 3D modeling software to create a virtual space, then places photos and videos into the 3D space. Audio data is synchronized appropriately, enhancing the sense of realism.

[0295] Step 10:

[0296] The server provides users with data and XR content hosted on the cloud.

[0297] Specific operation: The server makes the XR content publicly available through a hosting service, allowing users to access it through a VR headset or browser, and generates a link or QR code to access the data.

[0298] Step 11:

[0299] The user accesses the cloud-based platform and begins the experience by using the provided resources.

[0300] Specific operation: The user launches a browser or VR headset, accesses the URL, logs in to the XR content, begins the experience in the virtual space, and uses the navigation function to explore various content.

[0301] This series of processing steps enables a system that allows users to easily upload information about a city's history and culture to the cloud and share it with other users as a realistic spatial experience. Verification by generative AI and emotion engines, combined with XR technology, enables the provision of safe, high-quality content.

[0302] Example 2

[0303] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0304] In modern society, users generate enormous amounts of photos, videos, and audio data. It is challenging to efficiently manage this data, appropriately classify it based on user emotions, and provide rich experiences in real time. In particular, technologies that consistently protect data privacy, filter inappropriate content, and recognize emotions are needed.

[0305] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0306] In this invention, the server includes an uploading means for allowing a user to upload data related to scenery or events onto a network, a verification means using a generation AI to check the authenticity, morality, and privacy of the uploaded data, an emotion recognition means for analyzing the verified data and identifying the user's emotion, a classification means for classifying the verified and emotion-recognized data into categories and tagging them, an integration means for integrating the classified data and generating a spatial experience using augmented reality technology, and a provision means for hosting the integrated data and spatial experience on a network and providing it to users. This enables efficient management of data based on user emotions and the provision of a rich spatial experience in real time.

[0307] 1. "Uploading means" refers to a method or device that allows a user to transmit data about scenery or events onto a network.

[0308] 2. "Verification Measures" refers to methods or devices that use Generative AI to verify the authenticity, morality, and privacy of uploaded data.

[0309] 3. "Emotion Recognition Means" refers to a method or device for analyzing verified data and identifying a user's emotion.

[0310] 4. "Classification means" refers to a method or device that classifies and tags verified and emotion-recognized data into categories.

[0311] 5. "Integration means" refers to a method or device that integrates classified data and generates a spatial experience using augmented reality technology.

[0312] 6. "Delivery Means" means a method or device for hosting integrated data and spatial experiences on a network and delivering them to users.

[0313] 7. "Network" refers to the infrastructure for transmitting and receiving data, including the Internet and other information and communications networks.

[0314] 8. “Generative AI” refers to algorithms that use artificial intelligence techniques to verify data authenticity, morality, and privacy.

[0315] 9. “Emotion recognition” refers to the process of identifying a user’s emotions through analysis of data.

[0316] 10. "Augmented reality technology" refers to technology that uses computer graphics and sensor technology to overlay additional information or content onto real-world visual information.

[0317] 11. "Hosting" refers to the process of placing data or content on a server and making it available to users over a network.

[0318] This invention is a system in which users upload data about scenery and events to the network, which is then verified and organized by a generation AI, which then performs emotion recognition, and finally uses augmented reality technology to provide a realistic spatial experience. This system performs the following specific processes:

[0319] Users use devices such as smartphones and PCs to upload photos, videos, and audio data related to landscapes and events to the cloud. This requires a device with a network connection and a cloud storage service. For example, a cloud storage service such as Amazon S3 is used.

[0320] The server uses generative AI to verify the uploaded data. Specifically, it checks the authenticity, morality, and privacy of the data. Image recognition technology is used for image analysis, specifically, general image analysis software and libraries are applied. For example, a generative AI model can be used to analyze an image, identify an individual's face, and apply blurring. An example of a prompt is written as follows:

[0321] "Please explain how the server uses image analysis software to verify uploaded photos."

[0322] The server further uses emotion recognition means to identify the user's emotion associated with the uploaded data. Specifically, facial expression analysis and voice analysis are performed to identify emotions such as "happiness" and "surprise." Emotion analysis software is used for emotion recognition. For example, the user's "happiness" or "excitement" can be detected through voice analysis. An example of a prompt sentence is described as follows:

[0323] "Explain how a server can perform audio analysis of a video to identify emotions."

[0324] The verified and emotion-recognized data is then classified by the server and tagged by category. This makes it easier to search and manage the data. A database-based search and tagging system is used for classification. For example, in addition to event tags such as "2023 Autumn Festival" and "Fireworks Display," emotion tags such as "Fun" and "Emotional" are added.

[0325] The server then integrates the classified data and emotional information to generate a realistic spatial experience using augmented reality technology. This process uses an augmented reality development environment, such as Unity or Unreal Engine. This allows users to experience a virtual reproduction of the festival venue using a VR headset (e.g., Oculus Rift).

[0326] Finally, the server hosts the integrated data and spatial experience on the cloud and provides it to users. Users can access this data via the Internet. A cloud hosting service is used for providing the data. For example, the generated XR content can be hosted in cloud storage and the URL can be notified to the user. An example of a prompt sentence is written as follows:

[0327] "Please explain how the server can host the generated XR content in the cloud and make it accessible to the user."

[0328] This system enables efficient management of data based on user emotions and provides rich spatial experiences in real time.

[0329] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0330] Step 1:

[0331] Users upload photos, videos, and audio data related to landscapes and events to the cloud using their smartphones or PCs.

[0332] Input: Photos, videos, and audio data taken by the user

[0333] Specific operation: The user launches the dedicated app, selects the data, and presses the "Upload" button. The data is then sent over the network to cloud storage, typically Amazon S3.

[0334] Output: Multimedia data stored on cloud storage

[0335] Step 2:

[0336] The server uses generative AI to verify the authenticity, morality, and privacy of uploaded data.

[0337] Input: Photos, videos, and audio data stored on the cloud

[0338] What it does: The server calls Amazon Rekognition or a similar image analysis service via API to verify the data, perform facial recognition in images and videos, and blur images if necessary. It also checks the database to confirm the accuracy of the event.

[0339] Output: Verified multimedia data (mosaic processed, accuracy checked)

[0340] Step 3:

[0341] The server uses an emotion recognition means to identify the user's emotion associated with the verified data.

[0342] Input: Verified multimedia data

[0343] How it works: The server extracts the audio portion of the video and uses emotion analysis software such as Microsoft Azure's Emotion API to identify the emotion. For example, it sends a prompt to the API saying, "Analyze the audio of the video and identify the user's emotion."

[0344] Output: Emotion-analyzed data (with emotion tags such as "happiness" and "surprise")

[0345] Step 4:

[0346] The server categorizes the verified and emotion-recognized data and assigns appropriate tags.

[0347] Input: Emotionally tagged multimedia data

[0348] Specific operation: The server uses a data classification system such as Elasticsearch to automatically assign event tags such as "2023 Autumn Festival" and "Fireworks Display" and emotion tags such as "Fun" and "Emotional" to the data.

[0349] Output: Classified and tagged data

[0350] Step 5:

[0351] The server integrates the classified data and emotional information and generates a spatial experience using augmented reality technology.

[0352] Input: Classified and tagged data

[0353] How it works: The server uses development environments such as Unity and Unreal Engine to import the data needed to design the XR space. This data is then used to customize the spatial experience based on the user's emotional tags. For example, it can recreate a virtual venue for the Oculus Rift.

[0354] Output: Customized spatial experience using XR technology

[0355] Step 6:

[0356] The server hosts the integrated data and spatial experiences on the network and provides them to users.

[0357] Input: Customized XR spatial experience

[0358] Specific operation: The server uploads the generated XR content to a cloud hosting service such as AWS and generates an access URL for the user. The user can enjoy the XR experience via this URL. For example, they can access it using a smartphone or a web browser.

[0359] Output: User-accessible XR content URL and experience

[0360] (Application example 2)

[0361] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0362] In recent years, there has been a demand for systems that digitize experiences at tourist spots and events and allow users to share and experience them in real time. However, these systems often lack the ability to analyze user sentiment and customize content accordingly. Another issue is insufficient measures to ensure data authenticity and privacy. Furthermore, it is difficult to balance the quality and security of content provided in real time. This can lead to a decline in the quality of the user experience and a loss of trust in the system.

[0363] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0364] In this invention, the server includes an uploading unit that allows users to upload data related to scenery and events to the cloud, a verification unit that uses a generation AI to check the authenticity, morality, and privacy of the uploaded data, an emotion analysis unit that uses image analysis technology and voice analysis technology to recognize emotions in the uploaded data and tag the data based on the results, a classification unit that classifies the data based on the emotion tags and category information assigned by the emotion analysis unit, an integration unit that integrates the classified data and emotion information and generates a realistic spatial experience using XR technology, and a provision unit that provides the data and XR content hosted on the cloud to users, thereby enabling the provision of a high-quality XR spatial experience personalized based on the user's emotions in real time.

[0365] A "user" is an individual or legal entity that uses the system to upload data about scenery or events to the cloud.

[0366] "Data related to scenery and events" is a general term for digital data such as photographs, videos, and audio related to scenery and events.

[0367] "On the cloud" refers to a group of remote servers accessible via the Internet.

[0368] An "uploading means" is a function or device that allows a user to transfer digital data to the cloud.

[0369] "Generative AI" is a system that uses artificial intelligence technology to analyze and verify uploaded digital data.

[0370] A "verification method" is a mechanism or algorithm that uses generative AI to check the authenticity, morality, and privacy of data.

[0371] "Image analysis technology" is a technology that analyzes digital images and extracts specific information and features.

[0372] "Voice analysis technology" is a technology that analyzes digital voice data and extracts specific information and characteristics.

[0373] "Emotion recognition" is the process of identifying a user's emotional state from digital data.

[0374] The "emotion analysis means" is a function or device for analyzing the user's emotions using image analysis technology and voice analysis technology and tagging the results to data.

[0375] A "tag" is a label or keyword added to digital data to indicate its characteristics or content.

[0376] "Category information" refers to information on the criteria and categories for classifying digital data.

[0377] A "classifier" is a function or device for classifying digital data based on tag and category information.

[0378] "XR technology" is a general term for extended reality technologies, including virtual reality (VR), augmented reality (AR), and mixed reality (MR).

[0379] "Integration means" refers to a function or device that integrates classified digital data and emotional information and generates a realistic spatial experience using XR technology.

[0380] "Providing means" refers to a function or device for providing users with data and XR content hosted on the cloud.

[0381] "Hosting" refers to the storage and management of digital data and content on a server in order to provide it over the Internet.

[0382] "Real-time" refers to an environment where users can access and use data and content with minimal delay or latency.

[0383] This invention is a system that allows users to upload data about landscapes and events to the cloud, which is then verified and organized by a generative AI, tagged and classified based on the results of an analysis of the user's emotions using an emotion engine, and uses XR technology to provide a realistic spatial experience.

[0384] 1. Upload your data

[0385] Users use devices such as smartphones and personal computers to upload photos, videos, and audio data related to scenery and events to the cloud. A specific example is when a user visits a tourist spot and uploads photos and videos of the scenery and events they have taken to the cloud from their smartphone.

[0386] 2. Data Verification

[0387] The server uses generative AI to verify the uploaded data, checking its authenticity, morality, and privacy, and checking for inappropriate content and personal information. For example, the server automatically detects and blurs the faces of individuals in uploaded event photos.

[0388] 3. User Emotion Recognition

[0389] The server uses an emotion engine to recognize the user's emotion associated with the uploaded data. Image analysis and audio analysis technologies are used to identify emotions from the user's facial expressions and voice. For example, the server analyzes the user's emotions, such as "joy" or "surprise," from the audio of a video and assigns an emotion tag based on the results.

[0390] 4. Data Classification and Tagging

[0391] The server categorizes the data based on the verified data and the recognized emotion data and assigns appropriate tags. For example, by adding emotion tags such as "fun" and "moving" to tags such as "tourist spot name" and "event name," the data can be more easily searched.

[0392] 5. Building an XR spatial experience

[0393] The server integrates the classified data and emotional information and generates a realistic spatial experience using XR technology. The content and presentation method are adjusted according to the user's emotions. A specific example is a scene where a user uses a VR headset to experience a virtual recreation of a tourist attraction. Based on emotional analysis, fun and moving scenes are emphasized.

[0394] 6. Content Provision

[0395] The server provides users with cloud-hosted data and XR content, which they can access through their smartphones, personal computers, or VR headsets to enjoy emotion-based experiences.

[0396] Specific examples

[0397] A user visits a local autumn festival, takes photos of the festival with their smartphone, and uploads the data to the cloud. The server receives the data, uses a generation AI to blur personal information, and uses an emotion engine to recognize the user's emotion as "fun." Based on this, the data is tagged with tags such as "fun," "autumn festival," and "fireworks," and an XR spatial experience is generated based on this. Finally, the user can use a VR headset to experience the atmosphere of the autumn festival in the comfort of their own home.

[0398] Prompt Sentence Examples

[0399] Create an XR experience that allows people to virtually experience the moving scenes of the Autumn Festival.

[0400] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0401] Step 1:

[0402] Users use their smartphones or personal computers to upload photos, videos, and audio data related to landscapes and events to the cloud. To do this, users use the cloud server's upload interface, select the data file, and click the upload button. The input is digital data stored on the user's device, and the output is data securely stored on the cloud.

[0403] Step 2:

[0404] The server verifies the uploaded data using generative AI. Here, the generative AI model analyzes the data to verify its authenticity, morality, and privacy. Specific operations include detecting faces in images and filtering inappropriate content. The input is digital data stored on the cloud, and the output is the verified, corrected data that has been pixelated or removed.

[0405] Step 3:

[0406] The server uses an emotion engine to recognize the user's emotion associated with the uploaded data. At this stage, image and audio analysis technologies are used to identify emotions from the user's facial expressions and voice. For example, facial muscle movements are analyzed to extract emotions such as "happiness" or "surprise" as tags. The input is the corrected digital data, and the output is data with emotion tags.

[0407] Step 4:

[0408] The server classifies the data into categories based on the verified data and the recognized emotion data, and assigns appropriate tags. Here, emotion tags such as "fun" and "moving" are assigned in addition to category tags such as "tourist spot name" and "event name." The input is data with emotion tags assigned, and the output is data classified by category.

[0409] Step 5:

[0410] The server integrates the categorized data and emotional information and generates a realistic spatial experience using XR technology. Specific operations include 3D rendering of the data and customizing the scene based on emotional tags. The input is categorized data and emotional information, and the output is a spatial experience generated in XR format.

[0411] Step 6:

[0412] The server provides users with data and XR content hosted on the cloud. Users access and view this data through internet-connected smartphones, personal computers, or VR headsets. Specific operations include real-time distribution in response to user requests and high-speed data transfer. The input is spatial experience data in XR format, and the output is real-time experience content on the user's device.

[0413] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0414] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0415] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0416] [Second embodiment]

[0417] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0418] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0419] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0420] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0421] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0422] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0423] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0424] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0425] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0426] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0427] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0428] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0429] This invention is a system that allows users to upload data about cityscapes and events to the cloud, which is then verified and organized by generative AI, and uses XR technology to provide a realistic spatial experience.

[0430] System program processing

[0431] 1. Upload your data

[0432] Users: Upload photos, videos, and audio data related to cityscapes and events to the cloud using their smartphones or PCs.

[0433] Example: A user uploads photos and videos of a local festival from their smartphone to the cloud.

[0434] 2. Data Verification

[0435] Server: Uses generative AI to verify the uploaded data, checking for authenticity, inappropriate content, and personal information.

[0436] Example: The server automatically detects and blurs the faces of individuals in uploaded festival photos.

[0437] 3. Data Classification and Tagging

[0438] Server: Categorizes the validated data and assigns appropriate tags, making it easier to search and organize.

[0439] For example, add tags such as "2023 Autumn Festival" and "Fireworks Display" to make it easier for users to find what they're looking for.

[0440] 4. Building an XR spatial experience

[0441] Server: Integrates the classified data and generates a realistic spatial experience using XR technology, including various data such as photos, videos, audio, and text information.

[0442] Example: A user uses a VR headset to experience a virtual recreation of a festival venue.

[0443] 5. Content Provision

[0444] Server: Provides users with data and XR content hosted on the cloud. Users can access this data via the Internet.

[0445] Example: Tourists can view and experience information and experiences on the cloud using their smartphones or PCs.

[0446] Specific examples

[0447] Example of data upload

[0448] Users upload photos and videos of local autumn festivals from their smartphones to the cloud, where the cloud server receives and temporarily stores them.

[0449] Specific examples of verification

[0450] The server uses generative AI to analyze uploaded photos of the autumn festival, identify faces in the images, and apply blurring to them. It also compares the event details with an existing database to confirm accuracy.

[0451] Examples of classification and tagging

[0452] Once the data has been verified, the server automatically assigns tags such as "2023 Autumn Festival" or "Fireworks Display." This classification makes it easy for users to search for data.

[0453] Specific examples of XR spatial experiences

[0454] The server integrates the classified photos, videos, and audio data and uses XR technology to generate a virtual space, allowing users to experience the festival venue in a realistic way using a VR headset.

[0455] Examples of content provision

[0456] Users access the cloud-based platform as tourists and enjoy information about the autumn festival and XR experiences. Using content provided by the server, users can take a virtual tour before actually going sightseeing.

[0457] This system effectively preserves and shares the city's history and culture, promoting tourism and strengthening ties with the local community. The integration of generative AI verification and XR technology enables the provision of high-quality content.

[0458] The processing flow will be explained below.

[0459] Step 1:

[0460] The user opens the application on their device (smartphone or PC) and selects photos, videos, and audio data related to cityscapes and events.

[0461] Specific operation: The user selects multiple image or video files through the file selection dialog on the device and clicks the upload button.

[0462] Step 2:

[0463] The terminal transmits the selected data to the cloud server.

[0464] What happens: The device compresses the selected photo or video data and sends it to the server using a secure protocol (e.g., HTTPS).

[0465] Step 3:

[0466] The server receives the uploaded data and stores it temporarily.

[0467] Specific operation: The server stores the uploaded data in a temporary storage area and records the data's meta information (upload date and time, user ID, etc.) in a database.

[0468] Step 4:

[0469] The server begins validating the uploaded data using the generated AI.

[0470] How it works: The server applies image analysis algorithms to detect personal information and inappropriate content in the data, automatically blurring out faces and displaying a warning if the data contains content that violates public order and morals.

[0471] Step 5:

[0472] The server verifies the authenticity of the uploaded data.

[0473] Specific operation: The server checks the event information in the data against the existing database to determine the reliability of the data. If the data is unreliable, the user is asked to reconfirm it.

[0474] Step 6:

[0475] The server categorizes the verified data and assigns appropriate tags.

[0476] How it works: The server analyzes the data and automatically categorizes it into categories such as landscape photos, event videos, audio guides, etc. It also assigns tags based on the event name and era.

[0477] Step 7:

[0478] The server integrates the classified data and generates a realistic spatial experience using XR technology.

[0479] How it works: The server uses 3D modeling software to create a virtual space, then places photos and videos into the 3D space. Audio data is synchronized appropriately, enhancing the sense of realism.

[0480] Step 8:

[0481] The server provides users with data and XR content hosted on the cloud.

[0482] Specific operation: The server makes the XR content publicly available through a hosting service, allowing users to access it through a VR headset or browser, and generates a link or QR code to access the data.

[0483] Step 9:

[0484] The user accesses the cloud-based platform and begins the experience by using the provided resources.

[0485] Specific operation: The user launches a browser or VR headset, accesses the URL, logs in to the XR content, begins the experience in the virtual space, and uses the navigation function to explore various content.

[0486] This series of processing steps will enable a system that allows users to easily upload information about a city's history and culture to the cloud and share it with other users as a realistic spatial experience. Verification by generative AI and the integration of XR technology will enable the provision of safe, high-quality content.

[0487] Example 1

[0488] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0489] In today's world, it has become common to upload data related to cityscapes and events to the cloud and share it with many people. However, the sheer volume of this data makes managing and organizing it difficult. Furthermore, uploaded data may contain privacy violations or inappropriate content, and manually reviewing it takes time and effort. Furthermore, generating high-quality XR spatial experiences requires appropriate data integration and real-time content delivery. The present invention aims to solve these problems and realize efficient, high-quality data management and the provision of XR spatial experiences.

[0490] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0491] In this invention, the server includes a means for users to upload data to the cloud server using a dedicated application or web browser and temporarily store the uploaded data, an image analysis means for automatically detecting personal information using generation AI and modifying or deleting it, and an XR space generation means for integrating photos, videos, and audio data and generating a virtual space using XR technology. This makes it possible to efficiently manage data related to cityscapes and events, and provide users with a high-quality XR space experience while protecting their privacy.

[0492] The "uploading means" is a means by which a user uploads data about scenery or events onto the cloud.

[0493] "Verification measures" are measures using generative AI to check the authenticity, morality, and privacy of uploaded data.

[0494] The "classification means" is a means for classifying and tagging verified data into categories.

[0495] "Integration means" refers to a means for integrating classified data and generating a spatial experience using XR technology.

[0496] "Provision means" refers to the means for providing data and XR content hosted on the cloud to users.

[0497] The "temporary storage means" is a means by which a user uploads data to a cloud server using a dedicated application or a web browser, and temporarily stores the uploaded data.

[0498] "Image analysis means" refers to a means of automatically detecting personal information using generative AI and correcting or deleting it.

[0499] An "XR space generation means" is a means for integrating photographs, videos, and audio data to generate a virtual space using XR technology.

[0500] "Authentication means" means a means for managing user access to a platform on the cloud.

[0501] This invention is a system that allows users to upload data related to cityscapes and events to the cloud, which is then verified and organized by a generative AI, and uses XR technology to provide a realistic spatial experience. Below we will explain how this system is implemented in concrete terms.

[0502] 1. Upload your data

[0503] Users use devices such as smartphones or PCs to collect photos, videos, and audio data related to cityscapes and events. This data is then uploaded to the system's cloud server via a dedicated application or web browser. Commonly used cloud platforms (e.g., Amazon S3 or Google Cloud Storage) are used for cloud storage.

[0504] Example: A user uploads photos and videos of a local autumn festival to the cloud using a smartphone app.

[0505] 2. Data Verification

[0506] The cloud server temporarily stores the received data and then verifies it using a generative AI model. This verification process includes verifying the authenticity of the data, checking morality, and detecting and correcting personal information to protect privacy. Image analysis technology is used to detect inappropriate content and personal information (e.g., faces and names) and to correct them as necessary (e.g., by mosaic processing).

[0507] Example: The server analyzes photos from an autumn festival, automatically detects individuals' faces in the images, and applies mosaic processing.

[0508] 3. Data Classification and Tagging

[0509] Once verified, the data is categorized by a generative AI model. The model analyzes the content of photos, videos, audio, and other data, and assigns appropriate tags (e.g., "2023 Autumn Festival" or "Fireworks Display"). This categorization and tagging makes it easier to search and organize the data.

[0510] Example: The server assigns tags such as "Autumn Festival 2023" and "Fireworks Display" to autumn festival data.

[0511] 4. Building an XR spatial experience

[0512] The classified data is integrated using XR technology to generate a realistic spatial experience. This includes photos, videos, audio data, and more, each of which is appropriately positioned within the virtual space. Development platforms such as Unity and Unreal Engine are used to generate the XR space.

[0513] Example: The server virtually recreates the venue of an autumn festival and provides a virtual space that users can experience using a VR headset.

[0514] 5. Content Provision

[0515] The generated XR content is hosted on the cloud and provided to users via the Internet. Users can access this data and spatial experiences using dedicated applications or web browsers. User access is managed through authentication methods.

[0516] Example: A user accesses the cloud-based platform and enjoys the provided XR experience of the Autumn Festival.

[0517] This system will efficiently preserve and share the city's history and culture, promote tourism, and strengthen ties with the local community. Furthermore, the integration of generative AI data verification and XR technology will enable the provision of high-quality content.

[0518] An example prompt might be, "Upload a photo of the autumn festival, perform facial recognition and mosaic processing, and generate a virtual space using XR technology."

[0519] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0520] Step 1: Collect and upload data

[0521] Users use devices such as smartphones or PCs to take photos, videos, and audio data of cityscapes and events.Then, they upload the data to the cloud using a dedicated application or web browser.The input includes raw data such as the photos and videos taken, and the output is the uploaded data stored in the cloud.

[0522] Specific operation: The user takes photos or videos of the autumn festival using the smartphone camera, opens the dedicated app and presses the "upload" button.

[0523] Step 2: Receiving and temporarily storing data

[0524] The server receives data uploaded by users and temporarily stores it in cloud storage. The input is photos, videos, and audio data uploaded by users, and the output is the data temporarily stored in cloud storage.

[0525] Specific operation: The server receives the upload request, receives the data, and stores it in cloud storage such as Amazon S3.

[0526] Step 3: Validate the data

[0527] The server is a process that verifies the uploaded data using a generative AI model. Here, data authenticity is verified, morality is checked, and privacy protection (detection and correction of personal information) is performed. The input is temporarily stored data, and the output is verified and corrected data.

[0528] Specific operation: The server uses generated AI to analyze photos of the autumn festival, automatically detect individual faces, and apply mosaic processing.

[0529] Step 4: Classify and tag your data

[0530] Once verified, the data is classified by category and appropriately tagged by a generative AI model. This process uses image recognition and natural language processing techniques. The input is verified data, and the output is classified and tagged data.

[0531] Specific operation: The server classifies photos and videos of autumn festivals into categories such as "Autumn Festival 2023" and "Fireworks Display," and automatically assigns corresponding tags.

[0532] Step 5: Build an XR spatial experience

[0533] The classified data is integrated to generate a realistic spatial experience using XR technology. This process involves integrating photo, video, and audio data with 3D modeling and CG. The input is classified and tagged data, and the output is the XR spatial experience content.

[0534] Specific operation: The server integrates various data and uses Unity and Unreal Engine to virtually recreate the autumn festival venue.

[0535] Step 6: Providing content

[0536] The generated XR content is hosted on the cloud and provided to users via the Internet. Users can access this data and spatial experiences through dedicated applications or web browsers. The input is the generated XR content, and the output is content hosted on the cloud that is accessible to users.

[0537] How it works: The server hosts the XR spatial experience on the cloud, and the user accesses it using a dedicated app to begin the experience.

[0538] These steps will enable efficient management of cityscape and event data, enabling users to enjoy high-quality XR spatial experiences.

[0539] (Application example 1)

[0540] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0541] Conventional tourist guide systems have problems such as improper classification of information provided by tourists and local residents, and a high risk of inappropriate content or personal information being leaked. Furthermore, there was a lack of technology for providing real-time tourist information, making it difficult to provide tourists with appropriate information immediately. Furthermore, there were limited methods for integrating a large amount of data to provide a virtual space experience, making it difficult to provide users with a satisfactory virtual experience. To solve these problems, there was a need to build a highly functional system incorporating new technologies.

[0542] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0543] In this invention, the server includes an uploading unit that allows users to upload data related to scenery and events to the cloud, a verification unit using a generation AI to check the authenticity, morality, and privacy of the uploaded data, a classification unit that classifies the verified data into categories and tags them, an integration unit that integrates the classified data and generates a spatial experience using XR technology, a provision unit that provides users with the data and XR content hosted on the cloud, a utilization unit that allows users to enjoy the XR experience in real time using a smartphone or smart glasses, and a tagging assistant unit that automatically tags tourist spots and events when uploading data. This enables information provided by tourists and local residents to be quickly and accurately classified and safely provided, and further enables tourists to enjoy high-quality XR experiences in real time.

[0544] The "uploading means" is a device or program that allows a user to upload data related to scenery or events onto the cloud.

[0545] "Verification means" is a device or program that uses generative AI to check the authenticity, morality, and privacy of uploaded data.

[0546] The "classification means" is a device or program for classifying and tagging verified data into categories.

[0547] "Integration means" refers to a device or program for integrating classified data and generating a spatial experience using XR technology.

[0548] "Providing means" refers to a device or program for providing users with data and XR content hosted on the cloud.

[0549] "Means of use" refers to a device or program that allows a user to experience an XR experience in real time using a smartphone or smart glasses.

[0550] The "tagging assistant" is a device or program that automatically tags tourist spots and events when uploading data.

[0551] "Generative AI" refers to artificial intelligence technology used to verify uploaded data.

[0552] The "cloud" is a network-based platform for providing computing resources and storage over the Internet.

[0553] "XR technology" is a comprehensive extended reality technology that includes virtual reality (VR), augmented reality (AR), and mixed reality (MR).

[0554] "Data" refers to digital information such as photos, videos, and audio related to landscapes and events.

[0555] "Tourist spots" refer to places and famous places that tourists visit.

[0556] An "event" refers to a festival or event that is held on a specific date and time.

[0557] "Personal information" is information that can identify a specific individual.

[0558] "Means of use" refers to a device or program that allows a user to experience an XR experience in real time using a smartphone or smart glasses.

[0559] These definitions provide a clear understanding of key terms in the claims and provide a concrete explanation of how the invention works.

[0560] In the embodiment of the present invention, a system is constructed that quickly and accurately classifies and safely provides tourist information provided by tourists and local residents. The specific contents of the system program and its processing are described below.

[0561] This system uses the following equipment and software:

[0562] Hardware

[0563] Smartphone: Users upload data about scenery and events and use the XR experience.

[0564] Smart glasses: A device that allows users to enjoy XR experiences in real time.

[0565] Server: The main processing unit that validates, categorizes, consolidates, and serves data.

[0566] software

[0567] Generative AI: Artificial intelligence technology that checks data for authenticity, morality, and privacy.

[0568] Cloud services (e.g., Amazon Web Services (AWS)): Data storage and high-performance computing.

[0569] XR development platform (e.g. Unity, Unreal Engine): Building virtual spaces.

[0570] Program processing explanation

[0571] Uploading data

[0572] Users use their smartphones to upload photos, videos, and audio data related to landscapes and events to the cloud, where the cloud service receives and temporarily stores the data.

[0573] Data validation

[0574] The server uses generative AI to verify the uploaded data, confirming its authenticity and detecting inappropriate content or personal information, and then correcting or deleting it. For example, it can automatically detect faces in images and apply blurring to them.

[0575] Data classification and tagging

[0576] The server categorizes the verified data and assigns appropriate tags. For example, tags such as "XX Castle" and "2023 Summer Festival" are automatically assigned. This makes it easier for users to search for data.

[0577] Building an XR spatial experience

[0578] The server integrates the classified data and generates a realistic spatial experience using XR technology. This includes various data such as photos, videos, audio, and text information. For example, users can use a VR headset to experience a virtual reproduction of a festival venue.

[0579] Content provision

[0580] The server provides users with data and XR content hosted on the cloud. Users can experience information about tourist spots and events in real time using their smartphones or smart glasses. Data and XR content can also be delivered in real time upon user request.

[0581] Specific examples

[0582] For example, if a user uploads photos or videos of a local festival from their smartphone to the cloud, the server receives and verifies them. Generative AI then blurs out private information (such as faces) in the images and automatically adds tags such as "XX Festival" or "2023." Unity is then used to build a virtual space, allowing users to experience the festival venue in real time through smart glasses.

[0583] Prompt Sentence Examples

[0584] Data uploaded by tourists and local residents must include the following: The data content must identify which tourist spot or event it relates to and be tagged appropriately.

[0585] Data includes images, videos, and audio. Please review each data and detect any inappropriate content or personal information and delete or blur it. Furthermore, please classify the data and use XR technology to generate a virtual space.

[0586] The above is an embodiment of the present invention. This system makes it possible to quickly and accurately process and safely provide information provided by tourists and local residents. Furthermore, the virtual space experience using XR technology can provide users with a new way to enjoy sightseeing.

[0587] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0588] Step 1:

[0589] Users upload data about scenery and events from their smartphones to the cloud.

[0590] Input: Photos, videos, and audio data of scenery and events

[0591] Specific operation: Use the smartphone application to select data and press the button to upload to cloud storage.

[0592] Output: Data stored in the cloud

[0593] Step 2:

[0594] The server receives the uploaded data and stores it temporarily.

[0595] Input: Data uploaded to the cloud

[0596] Specific behavior: Receives and securely stores data using a cloud service (e.g., Amazon Web Services).

[0597] Output: Data waiting for validation

[0598] Step 3:

[0599] The server validates the uploaded data using generated AI.

[0600] Input: Data waiting for validation

[0601] Specific operation: Generative AI models are used to check the authenticity, morality, and privacy of data. Specifically, image analysis technology is used to perform facial recognition and blur personal information.

[0602] Output: Verification completed data

[0603] Step 4:

[0604] The server categorizes the verified data and assigns appropriate tags.

[0605] Input: Verification completed data

[0606] Specific operation: Analyzes the data content and automatically generates and assigns tags such as "XX Castle" and "2023 Summer Festival."

[0607] Output: Classified and tagged data

[0608] Step 5:

[0609] The server integrates the classified data and generates a spatial experience using XR technology.

[0610] Input: Classified and tagged data

[0611] How it works: Using an XR development platform (e.g., Unity, Unreal Engine), a virtual space is created, which aggregates photos, videos, audio, and text information.

[0612] Output: Generated XR spatial experience data

[0613] Step 6:

[0614] The server provides users with XR content hosted on the cloud.

[0615] Input: Generated XR spatial experience data

[0616] Specific operation: Data is streamed in real time to the user's smartphone or smart glasses via a cloud service. The user operates the application and starts the XR experience.

[0617] Output: XR content played on the user's device

[0618] Step 7:

[0619] Users can access the XR experience in real time using a smartphone or smart glasses.

[0620] Input: XR content deployed on the user's device

[0621] Specific operation: The user wears the device and uses the application to control the XR experience and virtually experience tourist attractions and events.

[0622] Output: Rich user experience feedback

[0623] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0624] This invention is a system in which users upload data about landscapes and events to the cloud, which is then verified and organized by a generative AI, and tagged and classified based on the results of an emotion engine's analysis of the user's emotions, and then uses XR technology to provide a realistic spatial experience.

[0625] System program processing

[0626] 1. Upload your data

[0627] Users: Upload photos, videos, and audio data related to cityscapes and events to the cloud using their smartphones or PCs.

[0628] Example: A user uploads photos and videos of a local festival from their smartphone to the cloud.

[0629] 2. Data Verification

[0630] Server: Uses generative AI to verify the uploaded data, checking for authenticity, inappropriate content, and personal information.

[0631] Example: The server automatically detects and blurs the faces of individuals in uploaded festival photos.

[0632] 3. User Emotion Recognition

[0633] Server: Uses an emotion engine to recognize the user's emotion related to the uploaded data. Emotions are identified from the user's facial expressions and voice analysis.

[0634] Example: The server analyzes the user's emotions from the audio of the video and assigns emotion tags such as "joy" or "surprise."

[0635] 4. Data Classification and Tagging

[0636] Server: Based on the verified data and recognized emotion data, the data is classified into categories and assigned appropriate tags.

[0637] For example, in addition to event tags such as "2023 Autumn Festival" and "Fireworks Display," emotion tags such as "Fun" and "Emotional" are added.

[0638] 5. Building an XR spatial experience

[0639] Server: Integrates the classified data and emotional information, and generates a realistic spatial experience using XR technology. The content and presentation method are adjusted according to the user's emotions.

[0640] Example: A user uses a VR headset to experience a virtual festival venue, with content customized based on sentiment analysis.

[0641] 6. Content Provision

[0642] Server: Provides users with data and XR content hosted on the cloud. Users can access this data via the Internet.

[0643] Example: Tourists browse information and experiences on the cloud using their smartphones or PCs and enjoy emotion-based experiences.

[0644] Specific examples

[0645] Example of data upload

[0646] Users upload photos and videos of local autumn festivals from their smartphones to the cloud, where the cloud server receives and temporarily stores them.

[0647] Specific examples of verification

[0648] The server uses generative AI to analyze uploaded photos of the autumn festival, identify faces in the images, and apply blurring to them. It also compares the event details with an existing database to confirm accuracy.

[0649] Specific examples of emotion recognition

[0650] The emotion engine analyzes the audio of videos uploaded by users and detects their "happiness" or "excitement." Based on this, emotion tags are assigned to the data.

[0651] Examples of classification and tagging

[0652] Once verification and emotion recognition are complete, the server automatically assigns event tags such as "2023 Autumn Festival" and "Fireworks Display" and emotion tags such as "Fun" and "Emotional," making it easier for users to search for data.

[0653] Specific examples of XR spatial experiences

[0654] The server integrates the classified photos, videos, audio data, and emotional information to generate a virtual space using XR technology. Users can then use a VR headset to experience the festival venue in a realistic way. Based on the emotional analysis, the user's experience is customized, highlighting fun and moving scenes.

[0655] Examples of content provision

[0656] Users access the cloud-based platform as tourists and enjoy information about the autumn festival and XR experiences. The content provided by the server is customized based on emotional information, providing a more personalized experience.

[0657] The system effectively preserves and shares the city's history and culture, promoting tourism and strengthening ties with the local community. The integration of generative AI and emotion engine validation with XR technology enables the provision of safe, high-quality content.

[0658] The processing flow will be explained below.

[0659] Step 1:

[0660] The user opens the application on their device (smartphone or PC) and selects photos, videos, and audio data related to cityscapes and events.

[0661] Specific operation: The user selects multiple image or video files through the file selection dialog on the device and clicks the upload button.

[0662] Step 2:

[0663] The terminal transmits the selected data to the cloud server.

[0664] What happens: The device compresses the selected photo or video data and sends it to the server using a secure protocol (e.g., HTTPS).

[0665] Step 3:

[0666] The server receives the uploaded data and stores it temporarily.

[0667] Specific operation: The server stores the uploaded data in a temporary storage area and records the data's meta information (upload date and time, user ID, etc.) in a database.

[0668] Step 4:

[0669] The server begins validating the uploaded data using the generated AI.

[0670] How it works: The server applies image analysis algorithms to detect personal information and inappropriate content in the data, automatically blurring out faces and displaying a warning if the data contains content that violates public order and morals.

[0671] Step 5:

[0672] The server verifies the authenticity of the uploaded data.

[0673] Specific operation: The server checks the event information in the data against the existing database to determine the reliability of the data. If the data is unreliable, the user is asked to reconfirm it.

[0674] Step 6:

[0675] The server categorizes the verified data and assigns appropriate tags.

[0676] How it works: The server analyzes the data and automatically categorizes it into categories such as landscape photos, event videos, audio guides, etc. It also assigns tags based on the event name and era.

[0677] Step 7:

[0678] The server uses an emotion engine to recognize the user's emotion.

[0679] Specific operation: The server analyzes the user's facial expressions and voice in uploaded photos and videos to identify emotions such as "happiness" or "surprise." Based on this, an emotion tag is assigned.

[0680] Step 8:

[0681] The server adjusts the content and presentation of the data based on the emotional data.

[0682] Specific operation: The server optimizes the order of data placement and emphasis points by taking into account the emotion data. For example, scenes in which the user expressed the emotion of "surprise" are displayed preferentially.

[0683] Step 9:

[0684] The server integrates the classified data and emotional information and generates a realistic spatial experience using XR technology.

[0685] How it works: The server uses 3D modeling software to create a virtual space, then places photos and videos into the 3D space. Audio data is synchronized appropriately, enhancing the sense of realism.

[0686] Step 10:

[0687] The server provides users with data and XR content hosted on the cloud.

[0688] Specific operation: The server makes the XR content publicly available through a hosting service, allowing users to access it through a VR headset or browser, and generates a link or QR code to access the data.

[0689] Step 11:

[0690] The user accesses the cloud-based platform and begins the experience by using the provided resources.

[0691] Specific operation: The user launches a browser or VR headset, accesses the URL, logs in to the XR content, begins the experience in the virtual space, and uses the navigation function to explore various content.

[0692] This series of processing steps enables a system that allows users to easily upload information about a city's history and culture to the cloud and share it with other users as a realistic spatial experience. Verification by generative AI and emotion engines, combined with XR technology, enables the provision of safe, high-quality content.

[0693] Example 2

[0694] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0695] In modern society, users generate enormous amounts of photos, videos, and audio data. It is challenging to efficiently manage this data, appropriately classify it based on user emotions, and provide rich experiences in real time. In particular, technologies that consistently protect data privacy, filter inappropriate content, and recognize emotions are needed.

[0696] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0697] In this invention, the server includes an uploading means for allowing a user to upload data related to scenery or events onto a network, a verification means using a generation AI to check the authenticity, morality, and privacy of the uploaded data, an emotion recognition means for analyzing the verified data and identifying the user's emotion, a classification means for classifying the verified and emotion-recognized data into categories and tagging them, an integration means for integrating the classified data and generating a spatial experience using augmented reality technology, and a provision means for hosting the integrated data and spatial experience on a network and providing it to users. This enables efficient management of data based on user emotions and the provision of a rich spatial experience in real time.

[0698] 1. "Uploading means" refers to a method or device that allows a user to transmit data about scenery or events onto a network.

[0699] 2. "Verification Measures" refers to methods or devices that use Generative AI to verify the authenticity, morality, and privacy of uploaded data.

[0700] 3. "Emotion Recognition Means" refers to a method or device for analyzing verified data and identifying a user's emotion.

[0701] 4. "Classification means" refers to a method or device that classifies and tags verified and emotion-recognized data into categories.

[0702] 5. "Integration means" refers to a method or device that integrates classified data and generates a spatial experience using augmented reality technology.

[0703] 6. "Delivery Means" means a method or device for hosting integrated data and spatial experiences on a network and delivering them to users.

[0704] 7. "Network" refers to the infrastructure for transmitting and receiving data, including the Internet and other information and communications networks.

[0705] 8. “Generative AI” refers to algorithms that use artificial intelligence techniques to verify data authenticity, morality, and privacy.

[0706] 9. “Emotion recognition” refers to the process of identifying a user’s emotions through analysis of data.

[0707] 10. "Augmented reality technology" refers to technology that uses computer graphics and sensor technology to overlay additional information or content onto real-world visual information.

[0708] 11. "Hosting" refers to the process of placing data or content on a server and making it available to users over a network.

[0709] This invention is a system in which users upload data about scenery and events to the network, which is then verified and organized by a generation AI, which then performs emotion recognition, and finally uses augmented reality technology to provide a realistic spatial experience. This system performs the following specific processes:

[0710] Users use devices such as smartphones and PCs to upload photos, videos, and audio data related to landscapes and events to the cloud. This requires a device with a network connection and a cloud storage service. For example, a cloud storage service such as Amazon S3 is used.

[0711] The server uses generative AI to verify the uploaded data. Specifically, it checks the authenticity, morality, and privacy of the data. Image recognition technology is used for image analysis, specifically, general image analysis software and libraries are applied. For example, a generative AI model can be used to analyze an image, identify an individual's face, and apply blurring. An example of a prompt is written as follows:

[0712] "Please explain how the server uses image analysis software to verify uploaded photos."

[0713] The server further uses emotion recognition means to identify the user's emotion associated with the uploaded data. Specifically, facial expression analysis and voice analysis are performed to identify emotions such as "happiness" and "surprise." Emotion analysis software is used for emotion recognition. For example, the user's "happiness" or "excitement" can be detected through voice analysis. An example of a prompt sentence is described as follows:

[0714] "Explain how a server can perform audio analysis of a video to identify emotions."

[0715] The verified and emotion-recognized data is then classified by the server and tagged by category. This makes it easier to search and manage the data. A database-based search and tagging system is used for classification. For example, in addition to event tags such as "2023 Autumn Festival" and "Fireworks Display," emotion tags such as "Fun" and "Emotional" are added.

[0716] The server then integrates the classified data and emotional information to generate a realistic spatial experience using augmented reality technology. This process uses an augmented reality development environment, such as Unity or Unreal Engine. This allows users to experience a virtual reproduction of the festival venue using a VR headset (e.g., Oculus Rift).

[0717] Finally, the server hosts the integrated data and spatial experience on the cloud and provides it to users. Users can access this data via the Internet. A cloud hosting service is used for providing the data. For example, the generated XR content can be hosted in cloud storage and the URL can be notified to the user. An example of a prompt sentence is written as follows:

[0718] "Please explain how the server can host the generated XR content in the cloud and make it accessible to the user."

[0719] This system enables efficient management of data based on user emotions and provides rich spatial experiences in real time.

[0720] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0721] Step 1:

[0722] Users upload photos, videos, and audio data related to landscapes and events to the cloud using their smartphones or PCs.

[0723] Input: Photos, videos, and audio data taken by the user

[0724] Specific operation: The user launches the dedicated app, selects the data, and presses the "Upload" button. The data is then sent over the network to cloud storage, typically Amazon S3.

[0725] Output: Multimedia data stored on cloud storage

[0726] Step 2:

[0727] The server uses generative AI to verify the authenticity, morality, and privacy of uploaded data.

[0728] Input: Photos, videos, and audio data stored on the cloud

[0729] What it does: The server calls Amazon Rekognition or a similar image analysis service via API to verify the data, perform facial recognition in images and videos, and blur images if necessary. It also checks the database to confirm the accuracy of the event.

[0730] Output: Verified multimedia data (mosaic processed, accuracy checked)

[0731] Step 3:

[0732] The server uses an emotion recognition means to identify the user's emotion associated with the verified data.

[0733] Input: Verified multimedia data

[0734] How it works: The server extracts the audio portion of the video and uses emotion analysis software such as Microsoft Azure's Emotion API to identify the emotion. For example, it sends a prompt to the API saying, "Analyze the audio of the video and identify the user's emotion."

[0735] Output: Emotion-analyzed data (with emotion tags such as "happiness" and "surprise")

[0736] Step 4:

[0737] The server categorizes the verified and emotion-recognized data and assigns appropriate tags.

[0738] Input: Emotionally tagged multimedia data

[0739] Specific operation: The server uses a data classification system such as Elasticsearch to automatically assign event tags such as "2023 Autumn Festival" and "Fireworks Display" and emotion tags such as "Fun" and "Emotional" to the data.

[0740] Output: Classified and tagged data

[0741] Step 5:

[0742] The server integrates the classified data and emotional information and generates a spatial experience using augmented reality technology.

[0743] Input: Classified and tagged data

[0744] How it works: The server uses development environments such as Unity and Unreal Engine to import the data needed to design the XR space. This data is then used to customize the spatial experience based on the user's emotional tags. For example, it can recreate a virtual venue for the Oculus Rift.

[0745] Output: Customized spatial experience using XR technology

[0746] Step 6:

[0747] The server hosts the integrated data and spatial experiences on the network and provides them to users.

[0748] Input: Customized XR spatial experience

[0749] Specific operation: The server uploads the generated XR content to a cloud hosting service such as AWS and generates an access URL for the user. The user can enjoy the XR experience via this URL. For example, they can access it using a smartphone or a web browser.

[0750] Output: User-accessible XR content URL and experience

[0751] (Application example 2)

[0752] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0753] In recent years, there has been a demand for systems that digitize experiences at tourist spots and events and allow users to share and experience them in real time. However, these systems often lack the ability to analyze user sentiment and customize content accordingly. Another issue is insufficient measures to ensure data authenticity and privacy. Furthermore, it is difficult to balance the quality and security of content provided in real time. This can lead to a decline in the quality of the user experience and a loss of trust in the system.

[0754] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0755] In this invention, the server includes an uploading unit that allows users to upload data related to scenery and events to the cloud, a verification unit that uses a generation AI to check the authenticity, morality, and privacy of the uploaded data, an emotion analysis unit that uses image analysis technology and voice analysis technology to recognize emotions in the uploaded data and tag the data based on the results, a classification unit that classifies the data based on the emotion tags and category information assigned by the emotion analysis unit, an integration unit that integrates the classified data and emotion information and generates a realistic spatial experience using XR technology, and a provision unit that provides the data and XR content hosted on the cloud to users, thereby enabling the provision of a high-quality XR spatial experience personalized based on the user's emotions in real time.

[0756] A "user" is an individual or legal entity that uses the system to upload data about scenery or events to the cloud.

[0757] "Data related to scenery and events" is a general term for digital data such as photographs, videos, and audio related to scenery and events.

[0758] "On the cloud" refers to a group of remote servers accessible via the Internet.

[0759] An "uploading means" is a function or device that allows a user to transfer digital data to the cloud.

[0760] "Generative AI" is a system that uses artificial intelligence technology to analyze and verify uploaded digital data.

[0761] A "verification method" is a mechanism or algorithm that uses generative AI to check the authenticity, morality, and privacy of data.

[0762] "Image analysis technology" is a technology that analyzes digital images and extracts specific information and features.

[0763] "Voice analysis technology" is a technology that analyzes digital voice data and extracts specific information and characteristics.

[0764] "Emotion recognition" is the process of identifying a user's emotional state from digital data.

[0765] The "emotion analysis means" is a function or device for analyzing the user's emotions using image analysis technology and voice analysis technology and tagging the results to data.

[0766] A "tag" is a label or keyword added to digital data to indicate its characteristics or content.

[0767] "Category information" refers to information on the criteria and categories for classifying digital data.

[0768] A "classifier" is a function or device for classifying digital data based on tag and category information.

[0769] "XR technology" is a general term for extended reality technologies, including virtual reality (VR), augmented reality (AR), and mixed reality (MR).

[0770] "Integration means" refers to a function or device that integrates classified digital data and emotional information and generates a realistic spatial experience using XR technology.

[0771] "Providing means" refers to a function or device for providing users with data and XR content hosted on the cloud.

[0772] "Hosting" refers to the storage and management of digital data and content on a server in order to provide it over the Internet.

[0773] "Real-time" refers to an environment where users can access and use data and content with minimal delay or latency.

[0774] This invention is a system that allows users to upload data about landscapes and events to the cloud, which is then verified and organized by a generative AI, tagged and classified based on the results of an analysis of the user's emotions using an emotion engine, and uses XR technology to provide a realistic spatial experience.

[0775] 1. Upload your data

[0776] Users use devices such as smartphones and personal computers to upload photos, videos, and audio data related to scenery and events to the cloud. A specific example is when a user visits a tourist spot and uploads photos and videos of the scenery and events they have taken to the cloud from their smartphone.

[0777] 2. Data Verification

[0778] The server uses generative AI to verify the uploaded data, checking its authenticity, morality, and privacy, and checking for inappropriate content and personal information. For example, the server automatically detects and blurs the faces of individuals in uploaded event photos.

[0779] 3. User Emotion Recognition

[0780] The server uses an emotion engine to recognize the user's emotion associated with the uploaded data. Image analysis and audio analysis technologies are used to identify emotions from the user's facial expressions and voice. For example, the server analyzes the user's emotions, such as "joy" or "surprise," from the audio of a video and assigns an emotion tag based on the results.

[0781] 4. Data Classification and Tagging

[0782] The server categorizes the data based on the verified data and the recognized emotion data and assigns appropriate tags. For example, by adding emotion tags such as "fun" and "moving" to tags such as "tourist spot name" and "event name," the data can be more easily searched.

[0783] 5. Building an XR spatial experience

[0784] The server integrates the classified data and emotional information and generates a realistic spatial experience using XR technology. The content and presentation method are adjusted according to the user's emotions. A specific example is a scene where a user uses a VR headset to experience a virtual recreation of a tourist attraction. Based on emotional analysis, fun and moving scenes are emphasized.

[0785] 6. Content Provision

[0786] The server provides users with cloud-hosted data and XR content, which they can access through their smartphones, personal computers, or VR headsets to enjoy emotion-based experiences.

[0787] Specific examples

[0788] A user visits a local autumn festival, takes photos of the festival with their smartphone, and uploads the data to the cloud. The server receives the data, uses a generation AI to blur personal information, and uses an emotion engine to recognize the user's emotion as "fun." Based on this, the data is tagged with tags such as "fun," "autumn festival," and "fireworks," and an XR spatial experience is generated based on this. Finally, the user can use a VR headset to experience the atmosphere of the autumn festival in the comfort of their own home.

[0789] Prompt Sentence Examples

[0790] Create an XR experience that allows people to virtually experience the moving scenes of the Autumn Festival.

[0791] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0792] Step 1:

[0793] Users use their smartphones or personal computers to upload photos, videos, and audio data related to landscapes and events to the cloud. To do this, users use the cloud server's upload interface, select the data file, and click the upload button. The input is digital data stored on the user's device, and the output is data securely stored on the cloud.

[0794] Step 2:

[0795] The server verifies the uploaded data using generative AI. Here, the generative AI model analyzes the data to verify its authenticity, morality, and privacy. Specific operations include detecting faces in images and filtering inappropriate content. The input is digital data stored on the cloud, and the output is the verified, corrected data that has been pixelated or removed.

[0796] Step 3:

[0797] The server uses an emotion engine to recognize the user's emotion associated with the uploaded data. At this stage, image and audio analysis technologies are used to identify emotions from the user's facial expressions and voice. For example, facial muscle movements are analyzed to extract emotions such as "happiness" or "surprise" as tags. The input is the corrected digital data, and the output is data with emotion tags.

[0798] Step 4:

[0799] The server classifies the data into categories based on the verified data and the recognized emotion data, and assigns appropriate tags. Here, emotion tags such as "fun" and "moving" are assigned in addition to category tags such as "tourist spot name" and "event name." The input is data with emotion tags assigned, and the output is data classified by category.

[0800] Step 5:

[0801] The server integrates the categorized data and emotional information and generates a realistic spatial experience using XR technology. Specific operations include 3D rendering of the data and customizing the scene based on emotional tags. The input is categorized data and emotional information, and the output is a spatial experience generated in XR format.

[0802] Step 6:

[0803] The server provides users with data and XR content hosted on the cloud. Users access and view this data through internet-connected smartphones, personal computers, or VR headsets. Specific operations include real-time distribution in response to user requests and high-speed data transfer. The input is spatial experience data in XR format, and the output is real-time experience content on the user's device.

[0804] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0805] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0806] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0807] [Third embodiment]

[0808] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0809] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0810] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0811] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0812] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0813] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0814] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0815] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0816] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0817] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0818] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0819] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0820] This invention is a system that allows users to upload data about cityscapes and events to the cloud, which is then verified and organized by generative AI, and uses XR technology to provide a realistic spatial experience.

[0821] System program processing

[0822] 1. Upload your data

[0823] Users: Upload photos, videos, and audio data related to cityscapes and events to the cloud using their smartphones or PCs.

[0824] Example: A user uploads photos and videos of a local festival from their smartphone to the cloud.

[0825] 2. Data Verification

[0826] Server: Uses generative AI to verify the uploaded data, checking for authenticity, inappropriate content, and personal information.

[0827] Example: The server automatically detects and blurs the faces of individuals in uploaded festival photos.

[0828] 3. Data Classification and Tagging

[0829] Server: Categorizes the validated data and assigns appropriate tags, making it easier to search and organize.

[0830] For example, add tags such as "2023 Autumn Festival" and "Fireworks Display" to make it easier for users to find what they're looking for.

[0831] 4. Building an XR spatial experience

[0832] Server: Integrates the classified data and generates a realistic spatial experience using XR technology, including various data such as photos, videos, audio, and text information.

[0833] Example: A user uses a VR headset to experience a virtual recreation of a festival venue.

[0834] 5. Content Provision

[0835] Server: Provides users with data and XR content hosted on the cloud. Users can access this data via the Internet.

[0836] Example: Tourists can view and experience information and experiences on the cloud using their smartphones or PCs.

[0837] Specific examples

[0838] Example of data upload

[0839] Users upload photos and videos of local autumn festivals from their smartphones to the cloud, where the cloud server receives and temporarily stores them.

[0840] Specific examples of verification

[0841] The server uses generative AI to analyze uploaded photos of the autumn festival, identify faces in the images, and apply blurring to them. It also compares the event details with an existing database to confirm accuracy.

[0842] Examples of classification and tagging

[0843] Once the data has been verified, the server automatically assigns tags such as "2023 Autumn Festival" or "Fireworks Display." This classification makes it easy for users to search for data.

[0844] Specific examples of XR spatial experiences

[0845] The server integrates the classified photos, videos, and audio data and uses XR technology to generate a virtual space, allowing users to experience the festival venue in a realistic way using a VR headset.

[0846] Examples of content provision

[0847] Users access the cloud-based platform as tourists and enjoy information about the autumn festival and XR experiences. Using content provided by the server, users can take a virtual tour before actually going sightseeing.

[0848] This system effectively preserves and shares the city's history and culture, promoting tourism and strengthening ties with the local community. The integration of generative AI verification and XR technology enables the provision of high-quality content.

[0849] The processing flow will be explained below.

[0850] Step 1:

[0851] The user opens the application on their device (smartphone or PC) and selects photos, videos, and audio data related to cityscapes and events.

[0852] Specific operation: The user selects multiple image or video files through the file selection dialog on the device and clicks the upload button.

[0853] Step 2:

[0854] The terminal transmits the selected data to the cloud server.

[0855] What happens: The device compresses the selected photo or video data and sends it to the server using a secure protocol (e.g., HTTPS).

[0856] Step 3:

[0857] The server receives the uploaded data and stores it temporarily.

[0858] Specific operation: The server stores the uploaded data in a temporary storage area and records the data's meta information (upload date and time, user ID, etc.) in a database.

[0859] Step 4:

[0860] The server begins validating the uploaded data using the generated AI.

[0861] How it works: The server applies image analysis algorithms to detect personal information and inappropriate content in the data, automatically blurring out faces and displaying a warning if the data contains content that violates public order and morals.

[0862] Step 5:

[0863] The server verifies the authenticity of the uploaded data.

[0864] Specific operation: The server checks the event information in the data against the existing database to determine the reliability of the data. If the data is unreliable, the user is asked to reconfirm it.

[0865] Step 6:

[0866] The server categorizes the verified data and assigns appropriate tags.

[0867] How it works: The server analyzes the data and automatically categorizes it into categories such as landscape photos, event videos, audio guides, etc. It also assigns tags based on the event name and era.

[0868] Step 7:

[0869] The server integrates the classified data and generates a realistic spatial experience using XR technology.

[0870] How it works: The server uses 3D modeling software to create a virtual space, then places photos and videos into the 3D space. Audio data is synchronized appropriately, enhancing the sense of realism.

[0871] Step 8:

[0872] The server provides users with data and XR content hosted on the cloud.

[0873] Specific operation: The server makes the XR content publicly available through a hosting service, allowing users to access it through a VR headset or browser, and generates a link or QR code to access the data.

[0874] Step 9:

[0875] The user accesses the cloud-based platform and begins the experience by using the provided resources.

[0876] Specific operation: The user launches a browser or VR headset, accesses the URL, logs in to the XR content, begins the experience in the virtual space, and uses the navigation function to explore various content.

[0877] This series of processing steps will enable a system that allows users to easily upload information about a city's history and culture to the cloud and share it with other users as a realistic spatial experience. Verification by generative AI and the integration of XR technology will enable the provision of safe, high-quality content.

[0878] Example 1

[0879] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0880] In today's world, it has become common to upload data related to cityscapes and events to the cloud and share it with many people. However, the sheer volume of this data makes managing and organizing it difficult. Furthermore, uploaded data may contain privacy violations or inappropriate content, and manually reviewing it takes time and effort. Furthermore, generating high-quality XR spatial experiences requires appropriate data integration and real-time content delivery. The present invention aims to solve these problems and realize efficient, high-quality data management and the provision of XR spatial experiences.

[0881] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0882] In this invention, the server includes a means for users to upload data to the cloud server using a dedicated application or web browser and temporarily store the uploaded data, an image analysis means for automatically detecting personal information using generation AI and modifying or deleting it, and an XR space generation means for integrating photos, videos, and audio data and generating a virtual space using XR technology. This makes it possible to efficiently manage data related to cityscapes and events, and provide users with a high-quality XR space experience while protecting their privacy.

[0883] The "uploading means" is a means by which a user uploads data about scenery or events onto the cloud.

[0884] "Verification measures" are measures using generative AI to check the authenticity, morality, and privacy of uploaded data.

[0885] The "classification means" is a means for classifying and tagging verified data into categories.

[0886] "Integration means" refers to a means for integrating classified data and generating a spatial experience using XR technology.

[0887] "Provision means" refers to the means for providing data and XR content hosted on the cloud to users.

[0888] The "temporary storage means" is a means by which a user uploads data to a cloud server using a dedicated application or a web browser, and temporarily stores the uploaded data.

[0889] "Image analysis means" refers to a means of automatically detecting personal information using generative AI and correcting or deleting it.

[0890] An "XR space generation means" is a means for integrating photographs, videos, and audio data to generate a virtual space using XR technology.

[0891] "Authentication means" means a means for managing user access to a platform on the cloud.

[0892] This invention is a system that allows users to upload data related to cityscapes and events to the cloud, which is then verified and organized by a generative AI, and uses XR technology to provide a realistic spatial experience. Below we will explain how this system is implemented in concrete terms.

[0893] 1. Upload your data

[0894] Users use devices such as smartphones or PCs to collect photos, videos, and audio data related to cityscapes and events. This data is then uploaded to the system's cloud server via a dedicated application or web browser. Commonly used cloud platforms (e.g., Amazon S3 or Google Cloud Storage) are used for cloud storage.

[0895] Example: A user uploads photos and videos of a local autumn festival to the cloud using a smartphone app.

[0896] 2. Data Verification

[0897] The cloud server temporarily stores the received data and then verifies it using a generative AI model. This verification process includes verifying the authenticity of the data, checking morality, and detecting and correcting personal information to protect privacy. Image analysis technology is used to detect inappropriate content and personal information (e.g., faces and names) and to correct them as necessary (e.g., by mosaic processing).

[0898] Example: The server analyzes photos from an autumn festival, automatically detects individuals' faces in the images, and applies mosaic processing.

[0899] 3. Data Classification and Tagging

[0900] Once verified, the data is categorized by a generative AI model. The model analyzes the content of photos, videos, audio, and other data, and assigns appropriate tags (e.g., "2023 Autumn Festival" or "Fireworks Display"). This categorization and tagging makes it easier to search and organize the data.

[0901] Example: The server assigns tags such as "Autumn Festival 2023" and "Fireworks Display" to autumn festival data.

[0902] 4. Building an XR spatial experience

[0903] The classified data is integrated using XR technology to generate a realistic spatial experience. This includes photos, videos, audio data, and more, each of which is appropriately positioned within the virtual space. Development platforms such as Unity and Unreal Engine are used to generate the XR space.

[0904] Example: The server virtually recreates the venue of an autumn festival and provides a virtual space that users can experience using a VR headset.

[0905] 5. Content Provision

[0906] The generated XR content is hosted on the cloud and provided to users via the Internet. Users can access this data and spatial experiences using dedicated applications or web browsers. User access is managed through authentication methods.

[0907] Example: A user accesses the cloud-based platform and enjoys the provided XR experience of the Autumn Festival.

[0908] This system will efficiently preserve and share the city's history and culture, promote tourism, and strengthen ties with the local community. Furthermore, the integration of generative AI data verification and XR technology will enable the provision of high-quality content.

[0909] An example prompt might be, "Upload a photo of the autumn festival, perform facial recognition and mosaic processing, and generate a virtual space using XR technology."

[0910] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0911] Step 1: Collect and upload data

[0912] Users use devices such as smartphones or PCs to take photos, videos, and audio data of cityscapes and events.Then, they upload the data to the cloud using a dedicated application or web browser.The input includes raw data such as the photos and videos taken, and the output is the uploaded data stored in the cloud.

[0913] Specific operation: The user takes photos or videos of the autumn festival using the smartphone camera, opens the dedicated app and presses the "upload" button.

[0914] Step 2: Receiving and temporarily storing data

[0915] The server receives data uploaded by users and temporarily stores it in cloud storage. The input is photos, videos, and audio data uploaded by users, and the output is the data temporarily stored in cloud storage.

[0916] Specific operation: The server receives the upload request, receives the data, and stores it in cloud storage such as Amazon S3.

[0917] Step 3: Validate the data

[0918] The server is a process that verifies the uploaded data using a generative AI model. Here, data authenticity is verified, morality is checked, and privacy protection (detection and correction of personal information) is performed. The input is temporarily stored data, and the output is verified and corrected data.

[0919] Specific operation: The server uses generated AI to analyze photos of the autumn festival, automatically detect individual faces, and apply mosaic processing.

[0920] Step 4: Classify and tag your data

[0921] Once verified, the data is classified by category and appropriately tagged by a generative AI model. This process uses image recognition and natural language processing techniques. The input is verified data, and the output is classified and tagged data.

[0922] Specific operation: The server classifies photos and videos of autumn festivals into categories such as "Autumn Festival 2023" and "Fireworks Display," and automatically assigns corresponding tags.

[0923] Step 5: Build an XR spatial experience

[0924] The classified data is integrated to generate a realistic spatial experience using XR technology. This process involves integrating photo, video, and audio data with 3D modeling and CG. The input is classified and tagged data, and the output is the XR spatial experience content.

[0925] Specific operation: The server integrates various data and uses Unity and Unreal Engine to virtually recreate the autumn festival venue.

[0926] Step 6: Providing content

[0927] The generated XR content is hosted on the cloud and provided to users via the Internet. Users can access this data and spatial experiences through dedicated applications or web browsers. The input is the generated XR content, and the output is content hosted on the cloud that is accessible to users.

[0928] How it works: The server hosts the XR spatial experience on the cloud, and the user accesses it using a dedicated app to begin the experience.

[0929] These steps will enable efficient management of cityscape and event data, enabling users to enjoy high-quality XR spatial experiences.

[0930] (Application example 1)

[0931] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0932] Conventional tourist guide systems have problems such as improper classification of information provided by tourists and local residents, and a high risk of inappropriate content or personal information being leaked. Furthermore, there was a lack of technology for providing real-time tourist information, making it difficult to provide tourists with appropriate information immediately. Furthermore, there were limited methods for integrating a large amount of data to provide a virtual space experience, making it difficult to provide users with a satisfactory virtual experience. To solve these problems, there was a need to build a highly functional system incorporating new technologies.

[0933] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0934] In this invention, the server includes an uploading unit that allows users to upload data related to scenery and events to the cloud, a verification unit using a generation AI to check the authenticity, morality, and privacy of the uploaded data, a classification unit that classifies the verified data into categories and tags them, an integration unit that integrates the classified data and generates a spatial experience using XR technology, a provision unit that provides users with the data and XR content hosted on the cloud, a utilization unit that allows users to enjoy the XR experience in real time using a smartphone or smart glasses, and a tagging assistant unit that automatically tags tourist spots and events when uploading data. This enables information provided by tourists and local residents to be quickly and accurately classified and safely provided, and further enables tourists to enjoy high-quality XR experiences in real time.

[0935] The "uploading means" is a device or program that allows a user to upload data related to scenery or events onto the cloud.

[0936] "Verification means" is a device or program that uses generative AI to check the authenticity, morality, and privacy of uploaded data.

[0937] The "classification means" is a device or program for classifying and tagging verified data into categories.

[0938] "Integration means" refers to a device or program for integrating classified data and generating a spatial experience using XR technology.

[0939] "Providing means" refers to a device or program for providing users with data and XR content hosted on the cloud.

[0940] "Means of use" refers to a device or program that allows a user to experience an XR experience in real time using a smartphone or smart glasses.

[0941] The "tagging assistant" is a device or program that automatically tags tourist spots and events when uploading data.

[0942] "Generative AI" refers to artificial intelligence technology used to verify uploaded data.

[0943] The "cloud" is a network-based platform for providing computing resources and storage over the Internet.

[0944] "XR technology" is a comprehensive extended reality technology that includes virtual reality (VR), augmented reality (AR), and mixed reality (MR).

[0945] "Data" refers to digital information such as photos, videos, and audio related to landscapes and events.

[0946] "Tourist spots" refer to places and famous places that tourists visit.

[0947] An "event" refers to a festival or event that is held on a specific date and time.

[0948] "Personal information" is information that can identify a specific individual.

[0949] "Means of use" refers to a device or program that allows a user to experience an XR experience in real time using a smartphone or smart glasses.

[0950] These definitions provide a clear understanding of key terms in the claims and provide a concrete explanation of how the invention works.

[0951] In the embodiment of the present invention, a system is constructed that quickly and accurately classifies and safely provides tourist information provided by tourists and local residents. The specific contents of the system program and its processing are described below.

[0952] This system uses the following equipment and software:

[0953] Hardware

[0954] Smartphone: Users upload data about scenery and events and use the XR experience.

[0955] Smart glasses: A device that allows users to enjoy XR experiences in real time.

[0956] Server: The main processing unit that validates, categorizes, consolidates, and serves data.

[0957] software

[0958] Generative AI: Artificial intelligence technology that checks data for authenticity, morality, and privacy.

[0959] Cloud services (e.g., Amazon Web Services (AWS)): Data storage and high-performance computing.

[0960] XR development platform (e.g. Unity, Unreal Engine): Building virtual spaces.

[0961] Program processing explanation

[0962] Uploading data

[0963] Users use their smartphones to upload photos, videos, and audio data related to landscapes and events to the cloud, where the cloud service receives and temporarily stores the data.

[0964] Data validation

[0965] The server uses generative AI to verify the uploaded data, confirming its authenticity and detecting inappropriate content or personal information, and then correcting or deleting it. For example, it can automatically detect faces in images and apply blurring to them.

[0966] Data classification and tagging

[0967] The server categorizes the verified data and assigns appropriate tags. For example, tags such as "XX Castle" and "2023 Summer Festival" are automatically assigned. This makes it easier for users to search for data.

[0968] Building an XR spatial experience

[0969] The server integrates the classified data and generates a realistic spatial experience using XR technology. This includes various data such as photos, videos, audio, and text information. For example, users can use a VR headset to experience a virtual reproduction of a festival venue.

[0970] Content provision

[0971] The server provides users with data and XR content hosted on the cloud. Users can experience information about tourist spots and events in real time using their smartphones or smart glasses. Data and XR content can also be delivered in real time upon user request.

[0972] Specific examples

[0973] For example, if a user uploads photos or videos of a local festival from their smartphone to the cloud, the server receives and verifies them. Generative AI then blurs out private information (such as faces) in the images and automatically adds tags such as "XX Festival" or "2023." Unity is then used to build a virtual space, allowing users to experience the festival venue in real time through smart glasses.

[0974] Prompt Sentence Examples

[0975] Data uploaded by tourists and local residents must include the following: The data content must identify which tourist spot or event it relates to and be tagged appropriately.

[0976] Data includes images, videos, and audio. Please review each data and detect any inappropriate content or personal information and delete or blur it. Furthermore, please classify the data and use XR technology to generate a virtual space.

[0977] The above is an embodiment of the present invention. This system makes it possible to quickly and accurately process and safely provide information provided by tourists and local residents. Furthermore, the virtual space experience using XR technology can provide users with a new way to enjoy sightseeing.

[0978] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0979] Step 1:

[0980] Users upload data about scenery and events from their smartphones to the cloud.

[0981] Input: Photos, videos, and audio data of scenery and events

[0982] Specific operation: Use the smartphone application to select data and press the button to upload to cloud storage.

[0983] Output: Data stored in the cloud

[0984] Step 2:

[0985] The server receives the uploaded data and stores it temporarily.

[0986] Input: Data uploaded to the cloud

[0987] Specific behavior: Receives and securely stores data using a cloud service (e.g., Amazon Web Services).

[0988] Output: Data waiting for validation

[0989] Step 3:

[0990] The server validates the uploaded data using generated AI.

[0991] Input: Data waiting for validation

[0992] Specific operation: Generative AI models are used to check the authenticity, morality, and privacy of data. Specifically, image analysis technology is used to perform facial recognition and blur personal information.

[0993] Output: Verification completed data

[0994] Step 4:

[0995] The server categorizes the verified data and assigns appropriate tags.

[0996] Input: Verification completed data

[0997] Specific operation: Analyzes the data content and automatically generates and assigns tags such as "XX Castle" and "2023 Summer Festival."

[0998] Output: Classified and tagged data

[0999] Step 5:

[1000] The server integrates the classified data and generates a spatial experience using XR technology.

[1001] Input: Classified and tagged data

[1002] How it works: Using an XR development platform (e.g., Unity, Unreal Engine), a virtual space is created, which aggregates photos, videos, audio, and text information.

[1003] Output: Generated XR spatial experience data

[1004] Step 6:

[1005] The server provides users with XR content hosted on the cloud.

[1006] Input: Generated XR spatial experience data

[1007] Specific operation: Data is streamed in real time to the user's smartphone or smart glasses via a cloud service. The user operates the application and starts the XR experience.

[1008] Output: XR content played on the user's device

[1009] Step 7:

[1010] Users can access the XR experience in real time using a smartphone or smart glasses.

[1011] Input: XR content deployed on the user's device

[1012] Specific operation: The user wears the device and uses the application to control the XR experience and virtually experience tourist attractions and events.

[1013] Output: Rich user experience feedback

[1014] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1015] This invention is a system in which users upload data about landscapes and events to the cloud, which is then verified and organized by a generative AI, and tagged and classified based on the results of an emotion engine's analysis of the user's emotions, and then uses XR technology to provide a realistic spatial experience.

[1016] System program processing

[1017] 1. Upload your data

[1018] Users: Upload photos, videos, and audio data related to cityscapes and events to the cloud using their smartphones or PCs.

[1019] Example: A user uploads photos and videos of a local festival from their smartphone to the cloud.

[1020] 2. Data Verification

[1021] Server: Uses generative AI to verify the uploaded data, checking for authenticity, inappropriate content, and personal information.

[1022] Example: The server automatically detects and blurs the faces of individuals in uploaded festival photos.

[1023] 3. User Emotion Recognition

[1024] Server: Uses an emotion engine to recognize the user's emotion related to the uploaded data. Emotions are identified from the user's facial expressions and voice analysis.

[1025] Example: The server analyzes the user's emotions from the audio of the video and assigns emotion tags such as "joy" or "surprise."

[1026] 4. Data Classification and Tagging

[1027] Server: Based on the verified data and recognized emotion data, the data is classified into categories and assigned appropriate tags.

[1028] For example, in addition to event tags such as "2023 Autumn Festival" and "Fireworks Display," emotion tags such as "Fun" and "Emotional" are added.

[1029] 5. Building an XR spatial experience

[1030] Server: Integrates the classified data and emotional information, and generates a realistic spatial experience using XR technology. The content and presentation method are adjusted according to the user's emotions.

[1031] Example: A user uses a VR headset to experience a virtual festival venue, with content customized based on sentiment analysis.

[1032] 6. Content Provision

[1033] Server: Provides users with data and XR content hosted on the cloud. Users can access this data via the Internet.

[1034] Example: Tourists browse information and experiences on the cloud using their smartphones or PCs and enjoy emotion-based experiences.

[1035] Specific examples

[1036] Example of data upload

[1037] Users upload photos and videos of local autumn festivals from their smartphones to the cloud, where the cloud server receives and temporarily stores them.

[1038] Specific examples of verification

[1039] The server uses generative AI to analyze uploaded photos of the autumn festival, identify faces in the images, and apply blurring to them. It also compares the event details with an existing database to confirm accuracy.

[1040] Specific examples of emotion recognition

[1041] The emotion engine analyzes the audio of videos uploaded by users and detects their "happiness" or "excitement." Based on this, emotion tags are assigned to the data.

[1042] Examples of classification and tagging

[1043] Once verification and emotion recognition are complete, the server automatically assigns event tags such as "2023 Autumn Festival" and "Fireworks Display" and emotion tags such as "Fun" and "Emotional," making it easier for users to search for data.

[1044] Specific examples of XR spatial experiences

[1045] The server integrates the classified photos, videos, audio data, and emotional information to generate a virtual space using XR technology. Users can then use a VR headset to experience the festival venue in a realistic way. Based on the emotional analysis, the user's experience is customized, highlighting fun and moving scenes.

[1046] Examples of content provision

[1047] Users access the cloud-based platform as tourists and enjoy information about the autumn festival and XR experiences. The content provided by the server is customized based on emotional information, providing a more personalized experience.

[1048] The system effectively preserves and shares the city's history and culture, promoting tourism and strengthening ties with the local community. The integration of generative AI and emotion engine validation with XR technology enables the provision of safe, high-quality content.

[1049] The processing flow will be explained below.

[1050] Step 1:

[1051] The user opens the application on their device (smartphone or PC) and selects photos, videos, and audio data related to cityscapes and events.

[1052] Specific operation: The user selects multiple image or video files through the file selection dialog on the device and clicks the upload button.

[1053] Step 2:

[1054] The terminal transmits the selected data to the cloud server.

[1055] What happens: The device compresses the selected photo or video data and sends it to the server using a secure protocol (e.g., HTTPS).

[1056] Step 3:

[1057] The server receives the uploaded data and stores it temporarily.

[1058] Specific operation: The server stores the uploaded data in a temporary storage area and records the data's meta information (upload date and time, user ID, etc.) in a database.

[1059] Step 4:

[1060] The server begins validating the uploaded data using the generated AI.

[1061] How it works: The server applies image analysis algorithms to detect personal information and inappropriate content in the data, automatically blurring out faces and displaying a warning if the data contains content that violates public order and morals.

[1062] Step 5:

[1063] The server verifies the authenticity of the uploaded data.

[1064] Specific operation: The server checks the event information in the data against the existing database to determine the reliability of the data. If the data is unreliable, the user is asked to reconfirm it.

[1065] Step 6:

[1066] The server categorizes the verified data and assigns appropriate tags.

[1067] How it works: The server analyzes the data and automatically categorizes it into categories such as landscape photos, event videos, audio guides, etc. It also assigns tags based on the event name and era.

[1068] Step 7:

[1069] The server uses an emotion engine to recognize the user's emotion.

[1070] Specific operation: The server analyzes the user's facial expressions and voice in uploaded photos and videos to identify emotions such as "happiness" or "surprise." Based on this, an emotion tag is assigned.

[1071] Step 8:

[1072] The server adjusts the content and presentation of the data based on the emotional data.

[1073] Specific operation: The server optimizes the order of data placement and emphasis points by taking into account the emotion data. For example, scenes in which the user expressed the emotion of "surprise" are displayed preferentially.

[1074] Step 9:

[1075] The server integrates the classified data and emotional information and generates a realistic spatial experience using XR technology.

[1076] How it works: The server uses 3D modeling software to create a virtual space, then places photos and videos into the 3D space. Audio data is synchronized appropriately, enhancing the sense of realism.

[1077] Step 10:

[1078] The server provides users with data and XR content hosted on the cloud.

[1079] Specific operation: The server makes the XR content publicly available through a hosting service, allowing users to access it through a VR headset or browser, and generates a link or QR code to access the data.

[1080] Step 11:

[1081] The user accesses the cloud-based platform and begins the experience by using the provided resources.

[1082] Specific operation: The user launches a browser or VR headset, accesses the URL, logs in to the XR content, begins the experience in the virtual space, and uses the navigation function to explore various content.

[1083] This series of processing steps enables a system that allows users to easily upload information about a city's history and culture to the cloud and share it with other users as a realistic spatial experience. Verification by generative AI and emotion engines, combined with XR technology, enables the provision of safe, high-quality content.

[1084] Example 2

[1085] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1086] In modern society, users generate enormous amounts of photos, videos, and audio data. It is challenging to efficiently manage this data, appropriately classify it based on user emotions, and provide rich experiences in real time. In particular, technologies that consistently protect data privacy, filter inappropriate content, and recognize emotions are needed.

[1087] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1088] In this invention, the server includes an uploading means for allowing a user to upload data related to scenery or events onto a network, a verification means using a generation AI to check the authenticity, morality, and privacy of the uploaded data, an emotion recognition means for analyzing the verified data and identifying the user's emotion, a classification means for classifying the verified and emotion-recognized data into categories and tagging them, an integration means for integrating the classified data and generating a spatial experience using augmented reality technology, and a provision means for hosting the integrated data and spatial experience on a network and providing it to users. This enables efficient management of data based on user emotions and the provision of a rich spatial experience in real time.

[1089] 1. "Uploading means" refers to a method or device that allows a user to transmit data about scenery or events onto a network.

[1090] 2. "Verification Measures" refers to methods or devices that use Generative AI to verify the authenticity, morality, and privacy of uploaded data.

[1091] 3. "Emotion Recognition Means" refers to a method or device for analyzing verified data and identifying a user's emotion.

[1092] 4. "Classification means" refers to a method or device that classifies and tags verified and emotion-recognized data into categories.

[1093] 5. "Integration means" refers to a method or device that integrates classified data and generates a spatial experience using augmented reality technology.

[1094] 6. "Delivery Means" means a method or device for hosting integrated data and spatial experiences on a network and delivering them to users.

[1095] 7. "Network" refers to the infrastructure for transmitting and receiving data, including the Internet and other information and communications networks.

[1096] 8. “Generative AI” refers to algorithms that use artificial intelligence techniques to verify data authenticity, morality, and privacy.

[1097] 9. “Emotion recognition” refers to the process of identifying a user’s emotions through analysis of data.

[1098] 10. "Augmented reality technology" refers to technology that uses computer graphics and sensor technology to overlay additional information or content onto real-world visual information.

[1099] 11. "Hosting" refers to the process of placing data or content on a server and making it available to users over a network.

[1100] This invention is a system in which users upload data about scenery and events to the network, which is then verified and organized by a generation AI, which then performs emotion recognition, and finally uses augmented reality technology to provide a realistic spatial experience. This system performs the following specific processes:

[1101] Users use devices such as smartphones and PCs to upload photos, videos, and audio data related to landscapes and events to the cloud. This requires a device with a network connection and a cloud storage service. For example, a cloud storage service such as Amazon S3 is used.

[1102] The server uses generative AI to verify the uploaded data. Specifically, it checks the authenticity, morality, and privacy of the data. Image recognition technology is used for image analysis, specifically, general image analysis software and libraries are applied. For example, a generative AI model can be used to analyze an image, identify an individual's face, and apply blurring. An example of a prompt is written as follows:

[1103] "Please explain how the server uses image analysis software to verify uploaded photos."

[1104] The server further uses emotion recognition means to identify the user's emotion associated with the uploaded data. Specifically, facial expression analysis and voice analysis are performed to identify emotions such as "happiness" and "surprise." Emotion analysis software is used for emotion recognition. For example, the user's "happiness" or "excitement" can be detected through voice analysis. An example of a prompt sentence is described as follows:

[1105] "Explain how a server can perform audio analysis of a video to identify emotions."

[1106] The verified and emotion-recognized data is then classified by the server and tagged by category. This makes it easier to search and manage the data. A database-based search and tagging system is used for classification. For example, in addition to event tags such as "2023 Autumn Festival" and "Fireworks Display," emotion tags such as "Fun" and "Emotional" are added.

[1107] The server then integrates the classified data and emotional information to generate a realistic spatial experience using augmented reality technology. This process uses an augmented reality development environment, such as Unity or Unreal Engine. This allows users to experience a virtual reproduction of the festival venue using a VR headset (e.g., Oculus Rift).

[1108] Finally, the server hosts the integrated data and spatial experience on the cloud and provides it to users. Users can access this data via the Internet. A cloud hosting service is used for providing the data. For example, the generated XR content can be hosted in cloud storage and the URL can be notified to the user. An example of a prompt sentence is written as follows:

[1109] "Please explain how the server can host the generated XR content in the cloud and make it accessible to the user."

[1110] This system enables efficient management of data based on user emotions and provides rich spatial experiences in real time.

[1111] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1112] Step 1:

[1113] Users upload photos, videos, and audio data related to landscapes and events to the cloud using their smartphones or PCs.

[1114] Input: Photos, videos, and audio data taken by the user

[1115] Specific operation: The user launches the dedicated app, selects the data, and presses the "Upload" button. The data is then sent over the network to cloud storage, typically Amazon S3.

[1116] Output: Multimedia data stored on cloud storage

[1117] Step 2:

[1118] The server uses generative AI to verify the authenticity, morality, and privacy of uploaded data.

[1119] Input: Photos, videos, and audio data stored on the cloud

[1120] What it does: The server calls Amazon Rekognition or a similar image analysis service via API to verify the data, perform facial recognition in images and videos, and blur images if necessary. It also checks the database to confirm the accuracy of the event.

[1121] Output: Verified multimedia data (mosaic processed, accuracy checked)

[1122] Step 3:

[1123] The server uses an emotion recognition means to identify the user's emotion associated with the verified data.

[1124] Input: Verified multimedia data

[1125] How it works: The server extracts the audio portion of the video and uses emotion analysis software such as Microsoft Azure's Emotion API to identify the emotion. For example, it sends a prompt to the API saying, "Analyze the audio of the video and identify the user's emotion."

[1126] Output: Emotion-analyzed data (with emotion tags such as "happiness" and "surprise")

[1127] Step 4:

[1128] The server categorizes the verified and emotion-recognized data and assigns appropriate tags.

[1129] Input: Emotionally tagged multimedia data

[1130] Specific operation: The server uses a data classification system such as Elasticsearch to automatically assign event tags such as "2023 Autumn Festival" and "Fireworks Display" and emotion tags such as "Fun" and "Emotional" to the data.

[1131] Output: Classified and tagged data

[1132] Step 5:

[1133] The server integrates the classified data and emotional information and generates a spatial experience using augmented reality technology.

[1134] Input: Classified and tagged data

[1135] How it works: The server uses development environments such as Unity and Unreal Engine to import the data needed to design the XR space. This data is then used to customize the spatial experience based on the user's emotional tags. For example, it can recreate a virtual venue for the Oculus Rift.

[1136] Output: Customized spatial experience using XR technology

[1137] Step 6:

[1138] The server hosts the integrated data and spatial experiences on the network and provides them to users.

[1139] Input: Customized XR spatial experience

[1140] Specific operation: The server uploads the generated XR content to a cloud hosting service such as AWS and generates an access URL for the user. The user can enjoy the XR experience via this URL. For example, they can access it using a smartphone or a web browser.

[1141] Output: User-accessible XR content URL and experience

[1142] (Application example 2)

[1143] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1144] In recent years, there has been a demand for systems that digitize experiences at tourist spots and events and allow users to share and experience them in real time. However, these systems often lack the ability to analyze user sentiment and customize content accordingly. Another issue is insufficient measures to ensure data authenticity and privacy. Furthermore, it is difficult to balance the quality and security of content provided in real time. This can lead to a decline in the quality of the user experience and a loss of trust in the system.

[1145] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1146] In this invention, the server includes an uploading unit that allows users to upload data related to scenery and events to the cloud, a verification unit that uses a generation AI to check the authenticity, morality, and privacy of the uploaded data, an emotion analysis unit that uses image analysis technology and voice analysis technology to recognize emotions in the uploaded data and tag the data based on the results, a classification unit that classifies the data based on the emotion tags and category information assigned by the emotion analysis unit, an integration unit that integrates the classified data and emotion information and generates a realistic spatial experience using XR technology, and a provision unit that provides the data and XR content hosted on the cloud to users, thereby enabling the provision of a high-quality XR spatial experience personalized based on the user's emotions in real time.

[1147] A "user" is an individual or legal entity that uses the system to upload data about scenery or events to the cloud.

[1148] "Data related to scenery and events" is a general term for digital data such as photographs, videos, and audio related to scenery and events.

[1149] "On the cloud" refers to a group of remote servers accessible via the Internet.

[1150] An "uploading means" is a function or device that allows a user to transfer digital data to the cloud.

[1151] "Generative AI" is a system that uses artificial intelligence technology to analyze and verify uploaded digital data.

[1152] A "verification method" is a mechanism or algorithm that uses generative AI to check the authenticity, morality, and privacy of data.

[1153] "Image analysis technology" is a technology that analyzes digital images and extracts specific information and features.

[1154] "Voice analysis technology" is a technology that analyzes digital voice data and extracts specific information and characteristics.

[1155] "Emotion recognition" is the process of identifying a user's emotional state from digital data.

[1156] The "emotion analysis means" is a function or device for analyzing the user's emotions using image analysis technology and voice analysis technology and tagging the results to data.

[1157] A "tag" is a label or keyword added to digital data to indicate its characteristics or content.

[1158] "Category information" refers to information on the criteria and categories for classifying digital data.

[1159] A "classifier" is a function or device for classifying digital data based on tag and category information.

[1160] "XR technology" is a general term for extended reality technologies, including virtual reality (VR), augmented reality (AR), and mixed reality (MR).

[1161] "Integration means" refers to a function or device that integrates classified digital data and emotional information and generates a realistic spatial experience using XR technology.

[1162] "Providing means" refers to a function or device for providing users with data and XR content hosted on the cloud.

[1163] "Hosting" refers to the storage and management of digital data and content on a server in order to provide it over the Internet.

[1164] "Real-time" refers to an environment where users can access and use data and content with minimal delay or latency.

[1165] This invention is a system that allows users to upload data about landscapes and events to the cloud, which is then verified and organized by a generative AI, tagged and classified based on the results of an analysis of the user's emotions using an emotion engine, and uses XR technology to provide a realistic spatial experience.

[1166] 1. Upload your data

[1167] Users use devices such as smartphones and personal computers to upload photos, videos, and audio data related to scenery and events to the cloud. A specific example is when a user visits a tourist spot and uploads photos and videos of the scenery and events they have taken to the cloud from their smartphone.

[1168] 2. Data Verification

[1169] The server uses generative AI to verify the uploaded data, checking its authenticity, morality, and privacy, and checking for inappropriate content and personal information. For example, the server automatically detects and blurs the faces of individuals in uploaded event photos.

[1170] 3. User Emotion Recognition

[1171] The server uses an emotion engine to recognize the user's emotion associated with the uploaded data. Image analysis and audio analysis technologies are used to identify emotions from the user's facial expressions and voice. For example, the server analyzes the user's emotions, such as "joy" or "surprise," from the audio of a video and assigns an emotion tag based on the results.

[1172] 4. Data Classification and Tagging

[1173] The server categorizes the data based on the verified data and the recognized emotion data and assigns appropriate tags. For example, by adding emotion tags such as "fun" and "moving" to tags such as "tourist spot name" and "event name," the data can be more easily searched.

[1174] 5. Building an XR spatial experience

[1175] The server integrates the classified data and emotional information and generates a realistic spatial experience using XR technology. The content and presentation method are adjusted according to the user's emotions. A specific example is a scene where a user uses a VR headset to experience a virtual recreation of a tourist attraction. Based on emotional analysis, fun and moving scenes are emphasized.

[1176] 6. Content Provision

[1177] The server provides users with cloud-hosted data and XR content, which they can access through their smartphones, personal computers, or VR headsets to enjoy emotion-based experiences.

[1178] Specific examples

[1179] A user visits a local autumn festival, takes photos of the festival with their smartphone, and uploads the data to the cloud. The server receives the data, uses a generation AI to blur personal information, and uses an emotion engine to recognize the user's emotion as "fun." Based on this, the data is tagged with tags such as "fun," "autumn festival," and "fireworks," and an XR spatial experience is generated based on this. Finally, the user can use a VR headset to experience the atmosphere of the autumn festival in the comfort of their own home.

[1180] Prompt Sentence Examples

[1181] Create an XR experience that allows people to virtually experience the moving scenes of the Autumn Festival.

[1182] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1183] Step 1:

[1184] Users use their smartphones or personal computers to upload photos, videos, and audio data related to landscapes and events to the cloud. To do this, users use the cloud server's upload interface, select the data file, and click the upload button. The input is digital data stored on the user's device, and the output is data securely stored on the cloud.

[1185] Step 2:

[1186] The server verifies the uploaded data using generative AI. Here, the generative AI model analyzes the data to verify its authenticity, morality, and privacy. Specific operations include detecting faces in images and filtering inappropriate content. The input is digital data stored on the cloud, and the output is the verified, corrected data that has been pixelated or removed.

[1187] Step 3:

[1188] The server uses an emotion engine to recognize the user's emotion associated with the uploaded data. At this stage, image and audio analysis technologies are used to identify emotions from the user's facial expressions and voice. For example, facial muscle movements are analyzed to extract emotions such as "happiness" or "surprise" as tags. The input is the corrected digital data, and the output is data with emotion tags.

[1189] Step 4:

[1190] The server classifies the data into categories based on the verified data and the recognized emotion data, and assigns appropriate tags. Here, emotion tags such as "fun" and "moving" are assigned in addition to category tags such as "tourist spot name" and "event name." The input is data with emotion tags assigned, and the output is data classified by category.

[1191] Step 5:

[1192] The server integrates the categorized data and emotional information and generates a realistic spatial experience using XR technology. Specific operations include 3D rendering of the data and customizing the scene based on emotional tags. The input is categorized data and emotional information, and the output is a spatial experience generated in XR format.

[1193] Step 6:

[1194] The server provides users with data and XR content hosted on the cloud. Users access and view this data through internet-connected smartphones, personal computers, or VR headsets. Specific operations include real-time distribution in response to user requests and high-speed data transfer. The input is spatial experience data in XR format, and the output is real-time experience content on the user's device.

[1195] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1196] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1197] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1198] [Fourth embodiment]

[1199] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1200] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1201] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1202] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1203] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1204] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1205] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1206] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1207] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1208] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1209] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1210] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1211] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1212] This invention is a system that allows users to upload data about cityscapes and events to the cloud, which is then verified and organized by generative AI, and uses XR technology to provide a realistic spatial experience.

[1213] System program processing

[1214] 1. Upload your data

[1215] Users: Upload photos, videos, and audio data related to cityscapes and events to the cloud using their smartphones or PCs.

[1216] Example: A user uploads photos and videos of a local festival from their smartphone to the cloud.

[1217] 2. Data Verification

[1218] Server: Uses generative AI to verify the uploaded data, checking for authenticity, inappropriate content, and personal information.

[1219] Example: The server automatically detects and blurs the faces of individuals in uploaded festival photos.

[1220] 3. Data Classification and Tagging

[1221] Server: Categorizes the validated data and assigns appropriate tags, making it easier to search and organize.

[1222] For example, add tags such as "2023 Autumn Festival" and "Fireworks Display" to make it easier for users to find what they're looking for.

[1223] 4. Building an XR spatial experience

[1224] Server: Integrates the classified data and generates a realistic spatial experience using XR technology, including various data such as photos, videos, audio, and text information.

[1225] Example: A user uses a VR headset to experience a virtual recreation of a festival venue.

[1226] 5. Content Provision

[1227] Server: Provides users with data and XR content hosted on the cloud. Users can access this data via the Internet.

[1228] Example: Tourists can view and experience information and experiences on the cloud using their smartphones or PCs.

[1229] Specific examples

[1230] Example of data upload

[1231] Users upload photos and videos of local autumn festivals from their smartphones to the cloud, where the cloud server receives and temporarily stores them.

[1232] Specific examples of verification

[1233] The server uses generative AI to analyze uploaded photos of the autumn festival, identify faces in the images, and apply blurring to them. It also compares the event details with an existing database to confirm accuracy.

[1234] Examples of classification and tagging

[1235] Once the data has been verified, the server automatically assigns tags such as "2023 Autumn Festival" or "Fireworks Display." This classification makes it easy for users to search for data.

[1236] Specific examples of XR spatial experiences

[1237] The server integrates the classified photos, videos, and audio data and uses XR technology to generate a virtual space, allowing users to experience the festival venue in a realistic way using a VR headset.

[1238] Examples of content provision

[1239] Users access the cloud-based platform as tourists and enjoy information about the autumn festival and XR experiences. Using content provided by the server, users can take a virtual tour before actually going sightseeing.

[1240] This system effectively preserves and shares the city's history and culture, promoting tourism and strengthening ties with the local community. The integration of generative AI verification and XR technology enables the provision of high-quality content.

[1241] The processing flow will be explained below.

[1242] Step 1:

[1243] The user opens the application on their device (smartphone or PC) and selects photos, videos, and audio data related to cityscapes and events.

[1244] Specific operation: The user selects multiple image or video files through the file selection dialog on the device and clicks the upload button.

[1245] Step 2:

[1246] The terminal transmits the selected data to the cloud server.

[1247] What happens: The device compresses the selected photo or video data and sends it to the server using a secure protocol (e.g., HTTPS).

[1248] Step 3:

[1249] The server receives the uploaded data and stores it temporarily.

[1250] Specific operation: The server stores the uploaded data in a temporary storage area and records the data's meta information (upload date and time, user ID, etc.) in a database.

[1251] Step 4:

[1252] The server begins validating the uploaded data using the generated AI.

[1253] How it works: The server applies image analysis algorithms to detect personal information and inappropriate content in the data, automatically blurring out faces and displaying a warning if the data contains content that violates public order and morals.

[1254] Step 5:

[1255] The server verifies the authenticity of the uploaded data.

[1256] Specific operation: The server checks the event information in the data against the existing database to determine the reliability of the data. If the data is unreliable, the user is asked to reconfirm it.

[1257] Step 6:

[1258] The server categorizes the verified data and assigns appropriate tags.

[1259] How it works: The server analyzes the data and automatically categorizes it into categories such as landscape photos, event videos, audio guides, etc. It also assigns tags based on the event name and era.

[1260] Step 7:

[1261] The server integrates the classified data and generates a realistic spatial experience using XR technology.

[1262] How it works: The server uses 3D modeling software to create a virtual space, then places photos and videos into the 3D space. Audio data is synchronized appropriately, enhancing the sense of realism.

[1263] Step 8:

[1264] The server provides users with data and XR content hosted on the cloud.

[1265] Specific operation: The server makes the XR content publicly available through a hosting service, allowing users to access it through a VR headset or browser, and generates a link or QR code to access the data.

[1266] Step 9:

[1267] The user accesses the cloud-based platform and begins the experience by using the provided resources.

[1268] Specific operation: The user launches a browser or VR headset, accesses the URL, logs in to the XR content, begins the experience in the virtual space, and uses the navigation function to explore various content.

[1269] This series of processing steps will enable a system that allows users to easily upload information about a city's history and culture to the cloud and share it with other users as a realistic spatial experience. Verification by generative AI and the integration of XR technology will enable the provision of safe, high-quality content.

[1270] Example 1

[1271] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1272] In today's world, it has become common to upload data related to cityscapes and events to the cloud and share it with many people. However, the sheer volume of this data makes managing and organizing it difficult. Furthermore, uploaded data may contain privacy violations or inappropriate content, and manually reviewing it takes time and effort. Furthermore, generating high-quality XR spatial experiences requires appropriate data integration and real-time content delivery. The present invention aims to solve these problems and realize efficient, high-quality data management and the provision of XR spatial experiences.

[1273] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1274] In this invention, the server includes a means for users to upload data to the cloud server using a dedicated application or web browser and temporarily store the uploaded data, an image analysis means for automatically detecting personal information using generation AI and modifying or deleting it, and an XR space generation means for integrating photos, videos, and audio data and generating a virtual space using XR technology. This makes it possible to efficiently manage data related to cityscapes and events, and provide users with a high-quality XR space experience while protecting their privacy.

[1275] The "uploading means" is a means by which a user uploads data about scenery or events onto the cloud.

[1276] "Verification measures" are measures using generative AI to check the authenticity, morality, and privacy of uploaded data.

[1277] The "classification means" is a means for classifying and tagging verified data into categories.

[1278] "Integration means" refers to a means for integrating classified data and generating a spatial experience using XR technology.

[1279] "Provision means" refers to the means for providing data and XR content hosted on the cloud to users.

[1280] The "temporary storage means" is a means by which a user uploads data to a cloud server using a dedicated application or a web browser, and temporarily stores the uploaded data.

[1281] "Image analysis means" refers to a means of automatically detecting personal information using generative AI and correcting or deleting it.

[1282] An "XR space generation means" is a means for integrating photographs, videos, and audio data to generate a virtual space using XR technology.

[1283] "Authentication means" means a means for managing user access to a platform on the cloud.

[1284] This invention is a system that allows users to upload data related to cityscapes and events to the cloud, which is then verified and organized by a generative AI, and uses XR technology to provide a realistic spatial experience. Below we will explain how this system is implemented in concrete terms.

[1285] 1. Upload your data

[1286] Users use devices such as smartphones or PCs to collect photos, videos, and audio data related to cityscapes and events. This data is then uploaded to the system's cloud server via a dedicated application or web browser. Commonly used cloud platforms (e.g., Amazon S3 or Google Cloud Storage) are used for cloud storage.

[1287] Example: A user uploads photos and videos of a local autumn festival to the cloud using a smartphone app.

[1288] 2. Data Verification

[1289] The cloud server temporarily stores the received data and then verifies it using a generative AI model. This verification process includes verifying the authenticity of the data, checking morality, and detecting and correcting personal information to protect privacy. Image analysis technology is used to detect inappropriate content and personal information (e.g., faces and names) and to correct them as necessary (e.g., by mosaic processing).

[1290] Example: The server analyzes photos from an autumn festival, automatically detects individuals' faces in the images, and applies mosaic processing.

[1291] 3. Data Classification and Tagging

[1292] Once verified, the data is categorized by a generative AI model. The model analyzes the content of photos, videos, audio, and other data, and assigns appropriate tags (e.g., "2023 Autumn Festival" or "Fireworks Display"). This categorization and tagging makes it easier to search and organize the data.

[1293] Example: The server assigns tags such as "Autumn Festival 2023" and "Fireworks Display" to autumn festival data.

[1294] 4. Building an XR spatial experience

[1295] The classified data is integrated using XR technology to generate a realistic spatial experience. This includes photos, videos, audio data, and more, each of which is appropriately positioned within the virtual space. Development platforms such as Unity and Unreal Engine are used to generate the XR space.

[1296] Example: The server virtually recreates the venue of an autumn festival and provides a virtual space that users can experience using a VR headset.

[1297] 5. Content Provision

[1298] The generated XR content is hosted on the cloud and provided to users via the Internet. Users can access this data and spatial experiences using dedicated applications or web browsers. User access is managed through authentication methods.

[1299] Example: A user accesses the cloud-based platform and enjoys the provided XR experience of the Autumn Festival.

[1300] This system will efficiently preserve and share the city's history and culture, promote tourism, and strengthen ties with the local community. Furthermore, the integration of generative AI data verification and XR technology will enable the provision of high-quality content.

[1301] An example prompt might be, "Upload a photo of the autumn festival, perform facial recognition and mosaic processing, and generate a virtual space using XR technology."

[1302] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1303] Step 1: Collect and upload data

[1304] Users use devices such as smartphones or PCs to take photos, videos, and audio data of cityscapes and events.Then, they upload the data to the cloud using a dedicated application or web browser.The input includes raw data such as the photos and videos taken, and the output is the uploaded data stored in the cloud.

[1305] Specific operation: The user takes photos or videos of the autumn festival using the smartphone camera, opens the dedicated app and presses the "upload" button.

[1306] Step 2: Receiving and temporarily storing data

[1307] The server receives data uploaded by users and temporarily stores it in cloud storage. The input is photos, videos, and audio data uploaded by users, and the output is the data temporarily stored in cloud storage.

[1308] Specific operation: The server receives the upload request, receives the data, and stores it in cloud storage such as Amazon S3.

[1309] Step 3: Validate the data

[1310] The server is a process that verifies the uploaded data using a generative AI model. Here, data authenticity is verified, morality is checked, and privacy protection (detection and correction of personal information) is performed. The input is temporarily stored data, and the output is verified and corrected data.

[1311] Specific operation: The server uses generated AI to analyze photos of the autumn festival, automatically detect individual faces, and apply mosaic processing.

[1312] Step 4: Classify and tag your data

[1313] Once verified, the data is classified by category and appropriately tagged by a generative AI model. This process uses image recognition and natural language processing techniques. The input is verified data, and the output is classified and tagged data.

[1314] Specific operation: The server classifies photos and videos of autumn festivals into categories such as "Autumn Festival 2023" and "Fireworks Display," and automatically assigns corresponding tags.

[1315] Step 5: Build an XR spatial experience

[1316] The classified data is integrated to generate a realistic spatial experience using XR technology. This process involves integrating photo, video, and audio data with 3D modeling and CG. The input is classified and tagged data, and the output is the XR spatial experience content.

[1317] Specific operation: The server integrates various data and uses Unity and Unreal Engine to virtually recreate the autumn festival venue.

[1318] Step 6: Providing content

[1319] The generated XR content is hosted on the cloud and provided to users via the Internet. Users can access this data and spatial experiences through dedicated applications or web browsers. The input is the generated XR content, and the output is content hosted on the cloud that is accessible to users.

[1320] How it works: The server hosts the XR spatial experience on the cloud, and the user accesses it using a dedicated app to begin the experience.

[1321] These steps will enable efficient management of cityscape and event data, enabling users to enjoy high-quality XR spatial experiences.

[1322] (Application example 1)

[1323] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1324] Conventional tourist guide systems have problems such as improper classification of information provided by tourists and local residents, and a high risk of inappropriate content or personal information being leaked. Furthermore, there was a lack of technology for providing real-time tourist information, making it difficult to provide tourists with appropriate information immediately. Furthermore, there were limited methods for integrating a large amount of data to provide a virtual space experience, making it difficult to provide users with a satisfactory virtual experience. To solve these problems, there was a need to build a highly functional system incorporating new technologies.

[1325] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1326] In this invention, the server includes an uploading unit that allows users to upload data related to scenery and events to the cloud, a verification unit using a generation AI to check the authenticity, morality, and privacy of the uploaded data, a classification unit that classifies the verified data into categories and tags them, an integration unit that integrates the classified data and generates a spatial experience using XR technology, a provision unit that provides users with the data and XR content hosted on the cloud, a utilization unit that allows users to enjoy the XR experience in real time using a smartphone or smart glasses, and a tagging assistant unit that automatically tags tourist spots and events when uploading data. This enables information provided by tourists and local residents to be quickly and accurately classified and safely provided, and further enables tourists to enjoy high-quality XR experiences in real time.

[1327] The "uploading means" is a device or program that allows a user to upload data related to scenery or events onto the cloud.

[1328] "Verification means" is a device or program that uses generative AI to check the authenticity, morality, and privacy of uploaded data.

[1329] The "classification means" is a device or program for classifying and tagging verified data into categories.

[1330] "Integration means" refers to a device or program for integrating classified data and generating a spatial experience using XR technology.

[1331] "Providing means" refers to a device or program for providing users with data and XR content hosted on the cloud.

[1332] "Means of use" refers to a device or program that allows a user to experience an XR experience in real time using a smartphone or smart glasses.

[1333] The "tagging assistant" is a device or program that automatically tags tourist spots and events when uploading data.

[1334] "Generative AI" refers to artificial intelligence technology used to verify uploaded data.

[1335] The "cloud" is a network-based platform for providing computing resources and storage over the Internet.

[1336] "XR technology" is a comprehensive extended reality technology that includes virtual reality (VR), augmented reality (AR), and mixed reality (MR).

[1337] "Data" refers to digital information such as photos, videos, and audio related to landscapes and events.

[1338] "Tourist spots" refer to places and famous places that tourists visit.

[1339] An "event" refers to a festival or event that is held on a specific date and time.

[1340] "Personal information" is information that can identify a specific individual.

[1341] "Means of use" refers to a device or program that allows a user to experience an XR experience in real time using a smartphone or smart glasses.

[1342] These definitions provide a clear understanding of key terms in the claims and provide a concrete explanation of how the invention works.

[1343] In the embodiment of the present invention, a system is constructed that quickly and accurately classifies and safely provides tourist information provided by tourists and local residents. The specific contents of the system program and its processing are described below.

[1344] This system uses the following equipment and software:

[1345] Hardware

[1346] Smartphone: Users upload data about scenery and events and use the XR experience.

[1347] Smart glasses: A device that allows users to enjoy XR experiences in real time.

[1348] Server: The main processing unit that validates, categorizes, consolidates, and serves data.

[1349] software

[1350] Generative AI: Artificial intelligence technology that checks data for authenticity, morality, and privacy.

[1351] Cloud services (e.g., Amazon Web Services (AWS)): Data storage and high-performance computing.

[1352] XR development platform (e.g. Unity, Unreal Engine): Building virtual spaces.

[1353] Program processing explanation

[1354] Uploading data

[1355] Users use their smartphones to upload photos, videos, and audio data related to landscapes and events to the cloud, where the cloud service receives and temporarily stores the data.

[1356] Data validation

[1357] The server uses generative AI to verify the uploaded data, confirming its authenticity and detecting inappropriate content or personal information, and then correcting or deleting it. For example, it can automatically detect faces in images and apply blurring to them.

[1358] Data classification and tagging

[1359] The server categorizes the verified data and assigns appropriate tags. For example, tags such as "XX Castle" and "2023 Summer Festival" are automatically assigned. This makes it easier for users to search for data.

[1360] Building an XR spatial experience

[1361] The server integrates the classified data and generates a realistic spatial experience using XR technology. This includes various data such as photos, videos, audio, and text information. For example, users can use a VR headset to experience a virtual reproduction of a festival venue.

[1362] Content provision

[1363] The server provides users with data and XR content hosted on the cloud. Users can experience information about tourist spots and events in real time using their smartphones or smart glasses. Data and XR content can also be delivered in real time upon user request.

[1364] Specific examples

[1365] For example, if a user uploads photos or videos of a local festival from their smartphone to the cloud, the server receives and verifies them. Generative AI then blurs out private information (such as faces) in the images and automatically adds tags such as "XX Festival" or "2023." Unity is then used to build a virtual space, allowing users to experience the festival venue in real time through smart glasses.

[1366] Prompt Sentence Examples

[1367] Data uploaded by tourists and local residents must include the following: The data content must identify which tourist spot or event it relates to and be tagged appropriately.

[1368] Data includes images, videos, and audio. Please review each data and detect any inappropriate content or personal information and delete or blur it. Furthermore, please classify the data and use XR technology to generate a virtual space.

[1369] The above is an embodiment of the present invention. This system makes it possible to quickly and accurately process and safely provide information provided by tourists and local residents. Furthermore, the virtual space experience using XR technology can provide users with a new way to enjoy sightseeing.

[1370] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1371] Step 1:

[1372] Users upload data about scenery and events from their smartphones to the cloud.

[1373] Input: Photos, videos, and audio data of scenery and events

[1374] Specific operation: Use the smartphone application to select data and press the button to upload to cloud storage.

[1375] Output: Data stored in the cloud

[1376] Step 2:

[1377] The server receives the uploaded data and stores it temporarily.

[1378] Input: Data uploaded to the cloud

[1379] Specific behavior: Receives and securely stores data using a cloud service (e.g., Amazon Web Services).

[1380] Output: Data waiting for validation

[1381] Step 3:

[1382] The server validates the uploaded data using generated AI.

[1383] Input: Data waiting for validation

[1384] Specific operation: Generative AI models are used to check the authenticity, morality, and privacy of data. Specifically, image analysis technology is used to perform facial recognition and blur personal information.

[1385] Output: Verification completed data

[1386] Step 4:

[1387] The server categorizes the verified data and assigns appropriate tags.

[1388] Input: Verification completed data

[1389] Specific operation: Analyzes the data content and automatically generates and assigns tags such as "XX Castle" and "2023 Summer Festival."

[1390] Output: Classified and tagged data

[1391] Step 5:

[1392] The server integrates the classified data and generates a spatial experience using XR technology.

[1393] Input: Classified and tagged data

[1394] How it works: Using an XR development platform (e.g., Unity, Unreal Engine), a virtual space is created, which aggregates photos, videos, audio, and text information.

[1395] Output: Generated XR spatial experience data

[1396] Step 6:

[1397] The server provides users with XR content hosted on the cloud.

[1398] Input: Generated XR spatial experience data

[1399] Specific operation: Data is streamed in real time to the user's smartphone or smart glasses via a cloud service. The user operates the application and starts the XR experience.

[1400] Output: XR content played on the user's device

[1401] Step 7:

[1402] Users can access the XR experience in real time using a smartphone or smart glasses.

[1403] Input: XR content deployed on the user's device

[1404] Specific operation: The user wears the device and uses the application to control the XR experience and virtually experience tourist attractions and events.

[1405] Output: Rich user experience feedback

[1406] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1407] This invention is a system in which users upload data about landscapes and events to the cloud, which is then verified and organized by a generative AI, and tagged and classified based on the results of an emotion engine's analysis of the user's emotions, and then uses XR technology to provide a realistic spatial experience.

[1408] System program processing

[1409] 1. Upload your data

[1410] Users: Upload photos, videos, and audio data related to cityscapes and events to the cloud using their smartphones or PCs.

[1411] Example: A user uploads photos and videos of a local festival from their smartphone to the cloud.

[1412] 2. Data Verification

[1413] Server: Uses generative AI to verify the uploaded data, checking for authenticity, inappropriate content, and personal information.

[1414] Example: The server automatically detects and blurs the faces of individuals in uploaded festival photos.

[1415] 3. User Emotion Recognition

[1416] Server: Uses an emotion engine to recognize the user's emotion related to the uploaded data. Emotions are identified from the user's facial expressions and voice analysis.

[1417] Example: The server analyzes the user's emotions from the audio of the video and assigns emotion tags such as "joy" or "surprise."

[1418] 4. Data Classification and Tagging

[1419] Server: Based on the verified data and recognized emotion data, the data is classified into categories and assigned appropriate tags.

[1420] For example, in addition to event tags such as "2023 Autumn Festival" and "Fireworks Display," emotion tags such as "Fun" and "Emotional" are added.

[1421] 5. Building an XR spatial experience

[1422] Server: Integrates the classified data and emotional information, and generates a realistic spatial experience using XR technology. The content and presentation method are adjusted according to the user's emotions.

[1423] Example: A user uses a VR headset to experience a virtual festival venue, with content customized based on sentiment analysis.

[1424] 6. Content Provision

[1425] Server: Provides users with data and XR content hosted on the cloud. Users can access this data via the Internet.

[1426] Example: Tourists browse information and experiences on the cloud using their smartphones or PCs and enjoy emotion-based experiences.

[1427] Specific examples

[1428] Example of data upload

[1429] Users upload photos and videos of local autumn festivals from their smartphones to the cloud, where the cloud server receives and temporarily stores them.

[1430] Specific examples of verification

[1431] The server uses generative AI to analyze uploaded photos of the autumn festival, identify faces in the images, and apply blurring to them. It also compares the event details with an existing database to confirm accuracy.

[1432] Specific examples of emotion recognition

[1433] The emotion engine analyzes the audio of videos uploaded by users and detects their "happiness" or "excitement." Based on this, emotion tags are assigned to the data.

[1434] Examples of classification and tagging

[1435] Once verification and emotion recognition are complete, the server automatically assigns event tags such as "2023 Autumn Festival" and "Fireworks Display" and emotion tags such as "Fun" and "Emotional," making it easier for users to search for data.

[1436] Specific examples of XR spatial experiences

[1437] The server integrates the classified photos, videos, audio data, and emotional information to generate a virtual space using XR technology. Users can then use a VR headset to experience the festival venue in a realistic way. Based on the emotional analysis, the user's experience is customized, highlighting fun and moving scenes.

[1438] Examples of content provision

[1439] Users access the cloud-based platform as tourists and enjoy information about the autumn festival and XR experiences. The content provided by the server is customized based on emotional information, providing a more personalized experience.

[1440] The system effectively preserves and shares the city's history and culture, promoting tourism and strengthening ties with the local community. The integration of generative AI and emotion engine validation with XR technology enables the provision of safe, high-quality content.

[1441] The processing flow will be explained below.

[1442] Step 1:

[1443] The user opens the application on their device (smartphone or PC) and selects photos, videos, and audio data related to cityscapes and events.

[1444] Specific operation: The user selects multiple image or video files through the file selection dialog on the device and clicks the upload button.

[1445] Step 2:

[1446] The terminal transmits the selected data to the cloud server.

[1447] What happens: The device compresses the selected photo or video data and sends it to the server using a secure protocol (e.g., HTTPS).

[1448] Step 3:

[1449] The server receives the uploaded data and stores it temporarily.

[1450] Specific operation: The server stores the uploaded data in a temporary storage area and records the data's meta information (upload date and time, user ID, etc.) in a database.

[1451] Step 4:

[1452] The server begins validating the uploaded data using the generated AI.

[1453] How it works: The server applies image analysis algorithms to detect personal information and inappropriate content in the data, automatically blurring out faces and displaying a warning if the data contains content that violates public order and morals.

[1454] Step 5:

[1455] The server verifies the authenticity of the uploaded data.

[1456] Specific operation: The server checks the event information in the data against the existing database to determine the reliability of the data. If the data is unreliable, the user is asked to reconfirm it.

[1457] Step 6:

[1458] The server categorizes the verified data and assigns appropriate tags.

[1459] How it works: The server analyzes the data and automatically categorizes it into categories such as landscape photos, event videos, audio guides, etc. It also assigns tags based on the event name and era.

[1460] Step 7:

[1461] The server uses an emotion engine to recognize the user's emotion.

[1462] Specific operation: The server analyzes the user's facial expressions and voice in uploaded photos and videos to identify emotions such as "happiness" or "surprise." Based on this, an emotion tag is assigned.

[1463] Step 8:

[1464] The server adjusts the content and presentation of the data based on the emotional data.

[1465] Specific operation: The server optimizes the order of data placement and emphasis points by taking into account the emotion data. For example, scenes in which the user expressed the emotion of "surprise" are displayed preferentially.

[1466] Step 9:

[1467] The server integrates the classified data and emotional information and generates a realistic spatial experience using XR technology.

[1468] How it works: The server uses 3D modeling software to create a virtual space, then places photos and videos into the 3D space. Audio data is synchronized appropriately, enhancing the sense of realism.

[1469] Step 10:

[1470] The server provides users with data and XR content hosted on the cloud.

[1471] Specific operation: The server makes the XR content publicly available through a hosting service, allowing users to access it through a VR headset or browser, and generates a link or QR code to access the data.

[1472] Step 11:

[1473] The user accesses the cloud-based platform and begins the experience by using the provided resources.

[1474] Specific operation: The user launches a browser or VR headset, accesses the URL, logs in to the XR content, begins the experience in the virtual space, and uses the navigation function to explore various content.

[1475] This series of processing steps enables a system that allows users to easily upload information about a city's history and culture to the cloud and share it with other users as a realistic spatial experience. Verification by generative AI and emotion engines, combined with XR technology, enables the provision of safe, high-quality content.

[1476] Example 2

[1477] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1478] In modern society, users generate enormous amounts of photos, videos, and audio data. It is challenging to efficiently manage this data, appropriately classify it based on user emotions, and provide rich experiences in real time. In particular, technologies that consistently protect data privacy, filter inappropriate content, and recognize emotions are needed.

[1479] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1480] In this invention, the server includes an uploading means for allowing a user to upload data related to scenery or events onto a network, a verification means using a generation AI to check the authenticity, morality, and privacy of the uploaded data, an emotion recognition means for analyzing the verified data and identifying the user's emotion, a classification means for classifying the verified and emotion-recognized data into categories and tagging them, an integration means for integrating the classified data and generating a spatial experience using augmented reality technology, and a provision means for hosting the integrated data and spatial experience on a network and providing it to users. This enables efficient management of data based on user emotions and the provision of a rich spatial experience in real time.

[1481] 1. "Uploading means" refers to a method or device that allows a user to transmit data about scenery or events onto a network.

[1482] 2. "Verification Measures" refers to methods or devices that use Generative AI to verify the authenticity, morality, and privacy of uploaded data.

[1483] 3. "Emotion Recognition Means" refers to a method or device for analyzing verified data and identifying a user's emotion.

[1484] 4. "Classification means" refers to a method or device that classifies and tags verified and emotion-recognized data into categories.

[1485] 5. "Integration means" refers to a method or device that integrates classified data and generates a spatial experience using augmented reality technology.

[1486] 6. "Delivery Means" means a method or device for hosting integrated data and spatial experiences on a network and delivering them to users.

[1487] 7. "Network" refers to the infrastructure for transmitting and receiving data, including the Internet and other information and communications networks.

[1488] 8. “Generative AI” refers to algorithms that use artificial intelligence techniques to verify data authenticity, morality, and privacy.

[1489] 9. “Emotion recognition” refers to the process of identifying a user’s emotions through analysis of data.

[1490] 10. "Augmented reality technology" refers to technology that uses computer graphics and sensor technology to overlay additional information or content onto real-world visual information.

[1491] 11. "Hosting" refers to the process of placing data or content on a server and making it available to users over a network.

[1492] This invention is a system in which users upload data about scenery and events to the network, which is then verified and organized by a generation AI, which then performs emotion recognition, and finally uses augmented reality technology to provide a realistic spatial experience. This system performs the following specific processes:

[1493] Users use devices such as smartphones and PCs to upload photos, videos, and audio data related to landscapes and events to the cloud. This requires a device with a network connection and a cloud storage service. For example, a cloud storage service such as Amazon S3 is used.

[1494] The server uses generative AI to verify the uploaded data. Specifically, it checks the authenticity, morality, and privacy of the data. Image recognition technology is used for image analysis, specifically, general image analysis software and libraries are applied. For example, a generative AI model can be used to analyze an image, identify an individual's face, and apply blurring. An example of a prompt is written as follows:

[1495] "Please explain how the server uses image analysis software to verify uploaded photos."

[1496] The server further uses emotion recognition means to identify the user's emotion associated with the uploaded data. Specifically, facial expression analysis and voice analysis are performed to identify emotions such as "happiness" and "surprise." Emotion analysis software is used for emotion recognition. For example, the user's "happiness" or "excitement" can be detected through voice analysis. An example of a prompt sentence is described as follows:

[1497] "Explain how a server can perform audio analysis of a video to identify emotions."

[1498] The verified and emotion-recognized data is then classified by the server and tagged by category. This makes it easier to search and manage the data. A database-based search and tagging system is used for classification. For example, in addition to event tags such as "2023 Autumn Festival" and "Fireworks Display," emotion tags such as "Fun" and "Emotional" are added.

[1499] The server then integrates the classified data and emotional information to generate a realistic spatial experience using augmented reality technology. This process uses an augmented reality development environment, such as Unity or Unreal Engine. This allows users to experience a virtual reproduction of the festival venue using a VR headset (e.g., Oculus Rift).

[1500] Finally, the server hosts the integrated data and spatial experience on the cloud and provides it to users. Users can access this data via the Internet. A cloud hosting service is used for providing the data. For example, the generated XR content can be hosted in cloud storage and the URL can be notified to the user. An example of a prompt sentence is written as follows:

[1501] "Please explain how the server can host the generated XR content in the cloud and make it accessible to the user."

[1502] This system enables efficient management of data based on user emotions and provides rich spatial experiences in real time.

[1503] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1504] Step 1:

[1505] Users upload photos, videos, and audio data related to landscapes and events to the cloud using their smartphones or PCs.

[1506] Input: Photos, videos, and audio data taken by the user

[1507] Specific operation: The user launches the dedicated app, selects the data, and presses the "Upload" button. The data is then sent over the network to cloud storage, typically Amazon S3.

[1508] Output: Multimedia data stored on cloud storage

[1509] Step 2:

[1510] The server uses generative AI to verify the authenticity, morality, and privacy of uploaded data.

[1511] Input: Photos, videos, and audio data stored on the cloud

[1512] What it does: The server calls Amazon Rekognition or a similar image analysis service via API to verify the data, perform facial recognition in images and videos, and blur images if necessary. It also checks the database to confirm the accuracy of the event.

[1513] Output: Verified multimedia data (mosaic processed, accuracy checked)

[1514] Step 3:

[1515] The server uses an emotion recognition means to identify the user's emotion associated with the verified data.

[1516] Input: Verified multimedia data

[1517] How it works: The server extracts the audio portion of the video and uses emotion analysis software such as Microsoft Azure's Emotion API to identify the emotion. For example, it sends a prompt to the API saying, "Analyze the audio of the video and identify the user's emotion."

[1518] Output: Emotion-analyzed data (with emotion tags such as "happiness" and "surprise")

[1519] Step 4:

[1520] The server categorizes the verified and emotion-recognized data and assigns appropriate tags.

[1521] Input: Emotionally tagged multimedia data

[1522] Specific operation: The server uses a data classification system such as Elasticsearch to automatically assign event tags such as "2023 Autumn Festival" and "Fireworks Display" and emotion tags such as "Fun" and "Emotional" to the data.

[1523] Output: Classified and tagged data

[1524] Step 5:

[1525] The server integrates the classified data and emotional information and generates a spatial experience using augmented reality technology.

[1526] Input: Classified and tagged data

[1527] How it works: The server uses development environments such as Unity and Unreal Engine to import the data needed to design the XR space. This data is then used to customize the spatial experience based on the user's emotional tags. For example, it can recreate a virtual venue for the Oculus Rift.

[1528] Output: Customized spatial experience using XR technology

[1529] Step 6:

[1530] The server hosts the integrated data and spatial experiences on the network and provides them to users.

[1531] Input: Customized XR spatial experience

[1532] Specific operation: The server uploads the generated XR content to a cloud hosting service such as AWS and generates an access URL for the user. The user can enjoy the XR experience via this URL. For example, they can access it using a smartphone or a web browser.

[1533] Output: User-accessible XR content URL and experience

[1534] (Application example 2)

[1535] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1536] In recent years, there has been a demand for systems that digitize experiences at tourist spots and events and allow users to share and experience them in real time. However, these systems often lack the ability to analyze user sentiment and customize content accordingly. Another issue is insufficient measures to ensure data authenticity and privacy. Furthermore, it is difficult to balance the quality and security of content provided in real time. This can lead to a decline in the quality of the user experience and a loss of trust in the system.

[1537] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1538] In this invention, the server includes an uploading unit that allows users to upload data related to scenery and events to the cloud, a verification unit that uses a generation AI to check the authenticity, morality, and privacy of the uploaded data, an emotion analysis unit that uses image analysis technology and voice analysis technology to recognize emotions in the uploaded data and tag the data based on the results, a classification unit that classifies the data based on the emotion tags and category information assigned by the emotion analysis unit, an integration unit that integrates the classified data and emotion information and generates a realistic spatial experience using XR technology, and a provision unit that provides the data and XR content hosted on the cloud to users, thereby enabling the provision of a high-quality XR spatial experience personalized based on the user's emotions in real time.

[1539] A "user" is an individual or legal entity that uses the system to upload data about scenery or events to the cloud.

[1540] "Data related to scenery and events" is a general term for digital data such as photographs, videos, and audio related to scenery and events.

[1541] "On the cloud" refers to a group of remote servers accessible via the Internet.

[1542] An "uploading means" is a function or device that allows a user to transfer digital data to the cloud.

[1543] "Generative AI" is a system that uses artificial intelligence technology to analyze and verify uploaded digital data.

[1544] A "verification method" is a mechanism or algorithm that uses generative AI to check the authenticity, morality, and privacy of data.

[1545] "Image analysis technology" is a technology that analyzes digital images and extracts specific information and features.

[1546] "Voice analysis technology" is a technology that analyzes digital voice data and extracts specific information and characteristics.

[1547] "Emotion recognition" is the process of identifying a user's emotional state from digital data.

[1548] The "emotion analysis means" is a function or device for analyzing the user's emotions using image analysis technology and voice analysis technology and tagging the results to data.

[1549] A "tag" is a label or keyword added to digital data to indicate its characteristics or content.

[1550] "Category information" refers to information on the criteria and categories for classifying digital data.

[1551] A "classifier" is a function or device for classifying digital data based on tag and category information.

[1552] "XR technology" is a general term for extended reality technologies, including virtual reality (VR), augmented reality (AR), and mixed reality (MR).

[1553] "Integration means" refers to a function or device that integrates classified digital data and emotional information and generates a realistic spatial experience using XR technology.

[1554] "Providing means" refers to a function or device for providing users with data and XR content hosted on the cloud.

[1555] "Hosting" refers to the storage and management of digital data and content on a server in order to provide it over the Internet.

[1556] "Real-time" refers to an environment where users can access and use data and content with minimal delay or latency.

[1557] This invention is a system that allows users to upload data about landscapes and events to the cloud, which is then verified and organized by a generative AI, tagged and classified based on the results of an analysis of the user's emotions using an emotion engine, and uses XR technology to provide a realistic spatial experience.

[1558] 1. Upload your data

[1559] Users use devices such as smartphones and personal computers to upload photos, videos, and audio data related to scenery and events to the cloud. A specific example is when a user visits a tourist spot and uploads photos and videos of the scenery and events they have taken to the cloud from their smartphone.

[1560] 2. Data Verification

[1561] The server uses generative AI to verify the uploaded data, checking its authenticity, morality, and privacy, and checking for inappropriate content and personal information. For example, the server automatically detects and blurs the faces of individuals in uploaded event photos.

[1562] 3. User Emotion Recognition

[1563] The server uses an emotion engine to recognize the user's emotion associated with the uploaded data. Image analysis and audio analysis technologies are used to identify emotions from the user's facial expressions and voice. For example, the server analyzes the user's emotions, such as "joy" or "surprise," from the audio of a video and assigns an emotion tag based on the results.

[1564] 4. Data Classification and Tagging

[1565] The server categorizes the data based on the verified data and the recognized emotion data and assigns appropriate tags. For example, by adding emotion tags such as "fun" and "moving" to tags such as "tourist spot name" and "event name," the data can be more easily searched.

[1566] 5. Building an XR spatial experience

[1567] The server integrates the classified data and emotional information and generates a realistic spatial experience using XR technology. The content and presentation method are adjusted according to the user's emotions. A specific example is a scene where a user uses a VR headset to experience a virtual recreation of a tourist attraction. Based on emotional analysis, fun and moving scenes are emphasized.

[1568] 6. Content Provision

[1569] The server provides users with cloud-hosted data and XR content, which they can access through their smartphones, personal computers, or VR headsets to enjoy emotion-based experiences.

[1570] Specific examples

[1571] A user visits a local autumn festival, takes photos of the festival with their smartphone, and uploads the data to the cloud. The server receives the data, uses a generation AI to blur personal information, and uses an emotion engine to recognize the user's emotion as "fun." Based on this, the data is tagged with tags such as "fun," "autumn festival," and "fireworks," and an XR spatial experience is generated based on this. Finally, the user can use a VR headset to experience the atmosphere of the autumn festival in the comfort of their own home.

[1572] Prompt Sentence Examples

[1573] Create an XR experience that allows people to virtually experience the moving scenes of the Autumn Festival.

[1574] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1575] Step 1:

[1576] Users use their smartphones or personal computers to upload photos, videos, and audio data related to landscapes and events to the cloud. To do this, users use the cloud server's upload interface, select the data file, and click the upload button. The input is digital data stored on the user's device, and the output is data securely stored on the cloud.

[1577] Step 2:

[1578] The server verifies the uploaded data using generative AI. Here, the generative AI model analyzes the data to verify its authenticity, morality, and privacy. Specific operations include detecting faces in images and filtering inappropriate content. The input is digital data stored on the cloud, and the output is the verified, corrected data that has been pixelated or removed.

[1579] Step 3:

[1580] The server uses an emotion engine to recognize the user's emotion associated with the uploaded data. At this stage, image and audio analysis technologies are used to identify emotions from the user's facial expressions and voice. For example, facial muscle movements are analyzed to extract emotions such as "happiness" or "surprise" as tags. The input is the corrected digital data, and the output is data with emotion tags.

[1581] Step 4:

[1582] The server classifies the data into categories based on the verified data and the recognized emotion data, and assigns appropriate tags. Here, emotion tags such as "fun" and "moving" are assigned in addition to category tags such as "tourist spot name" and "event name." The input is data with emotion tags assigned, and the output is data classified by category.

[1583] Step 5:

[1584] The server integrates the categorized data and emotional information and generates a realistic spatial experience using XR technology. Specific operations include 3D rendering of the data and customizing the scene based on emotional tags. The input is categorized data and emotional information, and the output is a spatial experience generated in XR format.

[1585] Step 6:

[1586] The server provides users with data and XR content hosted on the cloud. Users access and view this data through internet-connected smartphones, personal computers, or VR headsets. Specific operations include real-time distribution in response to user requests and high-speed data transfer. The input is spatial experience data in XR format, and the output is real-time experience content on the user's device.

[1587] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1588] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1589] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1590] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1591] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1592] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1593] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1594] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1595] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1596] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1597] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1598] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1599] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1600] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1601] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1602] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1603] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1604] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1605] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1606] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1607] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1608] The following is further disclosed regarding the above embodiment.

[1609] (Claim 1)

[1610] An uploading means for a user to upload data relating to scenery and events to the cloud;

[1611] Verification measures using generative AI to check the authenticity, morality, and privacy of uploaded data;

[1612] a classification means for categorizing and tagging the verified data;

[1613] an integration means for integrating the classified data and generating a spatial experience using XR technology;

[1614] A system including a providing means for providing data and XR content hosted on the cloud to a user.

[1615] (Claim 2)

[1616] 2. The system of claim 1, wherein the data verification means uses image analysis technology in the uploaded data to detect inappropriate content or personal information and correct or delete it.

[1617] (Claim 3)

[1618] The system according to claim 1, wherein the data providing means delivers data and XR content on the cloud in real time in response to a user request.

[1619] "Example 1"

[1620] (Claim 1)

[1621] An uploading means for a user to upload data relating to scenery and events to the cloud;

[1622] Verification measures using generative AI to check the authenticity, morality, and privacy of uploaded data;

[1623] a classification means for categorizing and tagging the verified data;

[1624] an integration means for integrating the classified data and generating a spatial experience using XR technology;

[1625] A means for providing the data and XR content hosted on the cloud to users;

[1626] A means for a user to upload data to a cloud server using a dedicated application or a web browser and temporarily store the uploaded data;

[1627] Image analysis means that automatically detects personal information using generative AI and then corrects or deletes it;

[1628] An XR space generation means that integrates photo, video, and audio data and generates a virtual space using XR technology;

[1629] A system that includes authentication measures to manage user access to a cloud-based platform.

[1630] (Claim 2)

[1631] 2. The system of claim 1, wherein the data verification means uses image analysis technology in the uploaded data to detect inappropriate content or personal information and correct or delete it.

[1632] (Claim 3)

[1633] The system according to claim 1, wherein the data providing means delivers data and XR content on the cloud in real time in response to a user request.

[1634] "Application Example 1"

[1635] (Claim 1)

[1636] An uploading means for a user to upload data relating to scenery and events to the cloud;

[1637] Verification measures using generative AI to check the authenticity, morality, and privacy of uploaded data;

[1638] a classification means for categorizing and tagging the verified data;

[1639] an integration means for integrating the classified data and generating a spatial experience using XR technology;

[1640] A means for providing the data and XR content hosted on the cloud to users;

[1641] A means for users to experience the XR experience in real time using a smartphone or smart glasses;

[1642] The system includes a tagging assistant to automatically tag tourist attractions and events when uploading data.

[1643] (Claim 2)

[1644] 2. The system of claim 1, wherein the data verification means uses image analysis technology in the uploaded data to detect inappropriate content or personal information and correct or delete it.

[1645] (Claim 3)

[1646] The system according to claim 1, wherein the data providing means delivers data and XR content on the cloud in real time in response to a user request.

[1647] "Example 2: Combining Emotion Engines"

[1648] (Claim 1)

[1649] uploading means for allowing a user to upload data relating to scenery or events onto the network;

[1650] Verification measures using generative AI to check the authenticity, morality, and privacy of uploaded data;

[1651] emotion recognition means for analyzing the verified data and identifying the user's emotion;

[1652] a classification means for classifying and tagging the verified and emotion-recognized data by category;

[1653] an integration means for integrating the classified data and generating a spatial experience using augmented reality technology;

[1654] A system including a delivery means for hosting and delivering integrated data and spatial experiences to users on a network.

[1655] (Claim 2)

[1656] The system of claim 1, wherein the verification means uses image analysis technology of the uploaded data to detect inappropriate content or personal information and correct or delete it.

[1657] (Claim 3)

[1658] 2. The system according to claim 1, wherein the data providing means delivers data and spatial experiences over the network in real time in response to a user's request.

[1659] "Application example 2 when combining emotion engines"

[1660] (Claim 1)

[1661] An uploading means for a user to upload data relating to scenery and events to the cloud;

[1662] Verification measures to check the authenticity, morality, and privacy of uploaded data using generative AI;

[1663] an emotion analysis means for recognizing emotions in the uploaded data using image analysis technology and voice analysis technology and tagging the data based on the results of the recognition;

[1664] a classification means for classifying data based on the emotion tag and category information assigned by the emotion analysis means;

[1665] an integration means for integrating the classified data and emotional information and generating a realistic spatial experience using XR technology;

[1666] A system including a providing means for providing data and XR content hosted on the cloud to a user.

[1667] (Claim 2)

[1668] 2. The system of claim 1, wherein the data verification means uses image analysis technology in the uploaded data to detect inappropriate content or personal information and correct or delete it.

[1669] (Claim 3)

[1670] The system according to claim 1, wherein the data providing means delivers data and XR content on the cloud in real time in response to a user request. [Explanation of symbols]

[1671] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. An uploading means for a user to upload data relating to scenery and events to the cloud; Verification measures using generative AI to check the authenticity, morality, and privacy of uploaded data; a classification means for categorizing and tagging the verified data; an integration means for integrating the classified data and generating a spatial experience using XR technology; A system including a providing means for providing data and XR content hosted on the cloud to a user.

2. 2. The system according to claim 1, wherein the data verification means uses image analysis technology in the uploaded data to detect inappropriate content or personal information and correct or delete it.

3. The system according to claim 1, wherein the data providing means delivers data and XR content on the cloud in real time in response to a user request.

Images