system

Abstract

A system is provided.SOLUTION: A system, comprising: means for receiving user posts from terminals and storing the user posts in a database; means for analyzing the database using a generation AI to organize and provide relevant information; means for analyzing user queries using an interactive AI to suggest travel plans; and means for dynamically generating and providing AR / VR content based on user requests.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] The purpose of this invention is to solve the difficulties faced by users with niche hobbies when gathering information and interacting with communities. Specifically, the lack of information specialized for individual hobbies has made it difficult to obtain information on areas of interest. Furthermore, travel restrictions and financial constraints make it difficult to participate in events and trips. The objective of this invention is to provide technology that will resolve these problems and facilitate information sharing and interaction between users. [Means for solving the problem]

[0005] The present invention solves the above problems with a system that includes a means for collecting user posts and storing them in a database, a means for analyzing the data stored in the database using generation AI to organize and provide related information, a means for analyzing user inquiries using conversational AI to propose travel plans, and a means for dynamically generating and providing AR / VR content based on user requests. This allows users to efficiently obtain information related to specific hobbies, and further enables fulfilling hobby activities without physical constraints through travel planning support and virtual experiences.

[0006] "Terminal" means a device used by a user to input information, including a smartphone, tablet, computer, etc.

[0007] "User posts" are content such as text, images, and videos that users provide to the system.

[0008] A "database" is an information system for systematically storing and managing posted data collected from users.

[0009] "Generative AI" is an artificial intelligence technology that analyzes and organizes data collected from users to generate summaries of information and new content.

[0010] "Related information" refers to useful data and knowledge about areas that interest the user.

[0011] "Conversational AI" is an artificial intelligence technology that understands users' inquiries in natural language and generates appropriate answers and suggestions.

[0012] A "travel itinerary" is a specific itinerary or activity suggestion that a user associates with a specific travel destination or event attendance.

[0013] "AR / VR content" refers to digital experiences provided using augmented reality (AR) and virtual reality (VR) technologies, and is content that allows users to virtually experience the experience without being physically present.

[0014] "Dynamically generated" refers to generating the necessary data or content in real time in response to a request. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] This invention is a system for users with particularly niche interests to collect information and interact with the community. This system has four main functions: collecting user posts, organizing information using generative AI, supporting travel planning using conversational AI, and providing AR / VR content. The details of this system are explained below.

[0037] Collecting user posts

[0038] Users post information about their hobbies and experiences through applications and websites. The device receives this posted data and sends it to a server. The server stores the received data in a database. This data can be in the form of text, images, videos, etc., and is centrally managed in the database.

[0039] Information organization using generative AI

[0040] The server periodically retrieves user posts from the database and passes them to the generation AI, which analyzes this data, organizes and summarizes the relevant information, allowing other users to efficiently obtain the information they need. The summarized information is then provided to users in an accessible format.

[0041] Travel planning support using conversational AI

[0042] When a user enters a travel planning question into the application, the device sends the query data to the server, which uses conversational AI to generate and return a travel itinerary that best suits the user's query, including specific travel destinations, places to visit, and recommended activities.

[0043] Providing AR / VR content

[0044] For users who find it difficult to travel or attend events, the server uses AR / VR technology to provide rich virtual experiences. When a user requests a virtual experience of a specific location or event, the device sends the request to the server. The server generates AR / VR content in real time based on the necessary data and provides the virtual experience. This allows users with financial or physical limitations to enjoy activities that interest them.

[0045] Specific examples

[0046] For example, if a user interested in model trains posts, "I want to go to a new model train exhibition," this information is stored in a database. Then, if another user uses the conversational AI to ask, "What train museum should I visit this weekend?", the conversational AI will suggest the best travel plan to answer that question. Furthermore, users who cannot physically visit museums can use AR / VR technology to virtually experience the inside of the museum.

[0047] This invention enables users with particularly niche hobbies to enrich their hobby activities by collecting and sharing information, planning trips, and engaging in virtual experiences, which is expected to promote interaction between users and improve engagement across the community.

[0048] The processing flow will be explained below.

[0049] Collecting and organizing user posts

[0050] Collecting user posts

[0051] Step 1:

[0052] User: Enters submissions into application or website forms.

[0053] Step 2:

[0054] Terminal: Sends the entered post content to the server.

[0055] Step 3:

[0056] Server: Stores the received submission data in a database.

[0057] Information organization

[0058] Step 4:

[0059] Server: Periodically retrieves user posts from the database.

[0060] Step 5:

[0061] Server: Passes the acquired data to the generation AI, which analyzes and summarizes the information.

[0062] Step 6:

[0063] Server: Provides summarized information in a form that users can access.

[0064] Travel planning support using conversational AI

[0065] Step 1:

[0066] User: Enters a travel planning question into a text box in the application.

[0067] Step 2:

[0068] Terminal: Sends the entered question to the server.

[0069] Step 3:

[0070] Server: Analyzes user questions using conversational AI and generates travel plans.

[0071] Step 4:

[0072] Server: Returns the generated itinerary to the user.

[0073] Step 5:

[0074] Terminal: Shows suggested travel itineraries to the user.

[0075] Providing AR / VR content

[0076] Step 1:

[0077] User: Requests a virtual experience of a specific location or event through a form in the app.

[0078] Step 2:

[0079] Terminal: Sends the input request to the server.

[0080] Step 3:

[0081] Server: Retrieves relevant data from a database based on the requested location or event.

[0082] Step 4:

[0083] Server: Generates the required AR / VR content in real time.

[0084] Step 5:

[0085] Server: Sends the generated AR / VR content to the user.

[0086] Step 6:

[0087] Device: Displays the received AR / VR content on the user's device.

[0088] Specific examples

[0089] Step 1:

[0090] User: Posts information about a model railroad show.

[0091] Step 2:

[0092] Terminal: Sends the post to the server.

[0093] Step 3:

[0094] Server: Stores the received posts in a database.

[0095] Step 4:

[0096] Server: Periodically retrieves posted data and summarizes it using a generation AI.

[0097] Step 5:

[0098] User: Finds museums to visit next weekend and asks the conversational AI a question.

[0099] Step 6:

[0100] Terminal: Sends the question to the server.

[0101] Step 7:

[0102] Server: Conversational AI generates optimal travel plans and suggests them to users.

[0103] Step 8:

[0104] User: If you can't make it to the museum, request a virtual experience.

[0105] Step 9:

[0106] Terminal: Sends the request to the server.

[0107] Step 10:

[0108] Server: Generates AR / VR content and provides it to users.

[0109] Step 11:

[0110] Device: Displays the received AR / VR content on the user's device.

[0111] This allows users to efficiently gather relevant information and enjoy travel and virtual experiences through a dedicated system, particularly for those with physical limitations, allowing them to enrich their hobbies.

[0112] Example 1

[0113] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0114] Conventional information gathering and community systems make it difficult for users with niche interests to efficiently collect and share information in an organized manner. They also lack the means for users to receive interactive assistance when planning trips, or for users with physical limitations to enjoy rich virtual experiences. As a result, it is difficult to provide comprehensive services to users with specific interests.

[0115] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0116] In this invention, the server includes means for receiving user posts from terminals and storing them in a database, means for analyzing the data stored in the database using a natural language processing model and organizing and providing related information, means for analyzing user inquiries using interactive artificial intelligence and proposing travel plans, means for dynamically generating and providing virtual reality technology content based on user requests, and means for retrieving user posts from the database and summarizing them using a natural language processing model. This allows users to efficiently collect information, receive travel plan suggestions, and enjoy rich experiences through virtual reality technology.

[0117] A "terminal" is a device that allows a user to input or receive information, and is often a smartphone, tablet, or computer.

[0118] "User Posts" refers to hobby-related information and experiences that users share through applications and websites, often in the form of text, images, videos, etc.

[0119] "Database" refers to a system for efficiently managing, storing, and retrieving data in a structured format.

[0120] A "natural language processing model" refers to an artificial intelligence technology that analyzes natural language data entered by a user and understands, generates, and summarizes human language.

[0121] "Conversational artificial intelligence" refers to a conversational artificial intelligence system that can provide natural responses to user questions and commands.

[0122] "Virtual reality technology" refers to technology that allows users to visually and aurally experience places and experiences that do not actually exist in the physical world.

[0123] "Means for organizing and providing information" refers to a method for using generative AI to analyze information stored in a database and provide related information to users in an easy-to-understand format.

[0124] "Means for proposing travel plans" refers to a method for using conversational AI to generate and propose optimal travel plans in response to user inquiries.

[0125] "Dynamic generation and serving means" refers to a method for creating and serving content in real time based on user requests.

[0126] "Means for summarizing" refers to a method for concisely summarizing multiple user-posted information using a natural language processing model.

[0127] This invention is a system for users with particularly niche interests to collect information and interact with the community. The system has four main functions: collecting user posts, organizing information using natural language processing models, assisting with travel planning using conversational artificial intelligence, and providing content using virtual reality technology.

[0128] Collecting user posts

[0129] Users post information about their hobbies and experiences through applications and websites. This data can be in the form of text, images, videos, etc.

[0130] The device receives the user's posted data and sends it to the server. For example, if a user posts "I want to go to the new model train exhibition," the device sends this to the server.

[0131] The server stores the received data in a database, which is a means of efficiently managing and centralizing this data.

[0132] Information organization using natural language processing models

[0133] The server periodically retrieves user posts from the database and passes them to a natural language processing model, such as OpenAI's GPT-3.

[0134] Natural language processing models analyze user posts and organize and summarize relevant information. The generated summaries are a means for other users to efficiently access information. For example, they can concisely summarize reviews of a model train exhibition posted by multiple users.

[0135] The server stores the summarized information in a database and makes it accessible to users.

[0136] Travel planning support using conversational AI

[0137] A user enters a travel planning question into an application, for example, "What train museums should I visit this weekend?"

[0138] The terminal transmits this inquiry data to the server.

[0139] The server uses conversational artificial intelligence (e.g., Dialogflow) to generate an optimal travel plan based on the user's questions, including destinations, places to visit, and recommended activities.

[0140] The server returns the generated itinerary to the user, who can view it within the app.

[0141] Virtual reality technology content provision

[0142] A user requests a virtual experience. For example, a request to "virtually experience the Railway Museum" is made within the app.

[0143] The terminal transmits this request data to the server.

[0144] The server collects relevant data and generates content in real time using virtual reality technology, which transcends physical limitations to provide users with a rich experience.

[0145] The server sends the generated content to the user's device, and the user experiences it using a dedicated app or VR headset.

[0146] Examples and prompts

[0147] For example, if a user who is interested in model trains posts that they want to go to a new model train exhibition, this information is stored in the database. Then, if another user uses the conversational AI to ask, "What train museum should I visit this weekend?", the conversational AI will suggest the best travel plan to answer that question. Additionally, users who cannot physically visit museums can use virtual reality technology to virtually experience the inside of the museum.

[0148] An example of a prompt to input to a generative AI model is as follows:

[0149] "I'm interested in model trains. Which railway museum should I visit this weekend? Also, what should I look out for when I visit?"

[0150] This invention is expected to enable users with particularly niche interests to efficiently gather information, receive travel plan suggestions, and enjoy rich virtual experiences, thereby increasing engagement across the community.

[0151] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0152] Processing Steps

[0153] Step 1: Collect user posts

[0154] 1. Users post information and experiences about their hobbies through the application or website.

[0155] Specific behavior:

[0156] A user posts, "I want to go to a model train exhibition."

[0157] Input and Output:

[0158] Input: Text data entered by the user (e.g., "I want to go to a model train exhibition")

[0159] Output: Text data is sent to the terminal.

[0160] 2. The terminal receives the posted data from the user and sends it to the server.

[0161] Specific behavior:

[0162] The app converts user input into JSON format and sends it to the server via an HTTP POST request.

[0163] Input and Output:

[0164] Input: JSON format text data (e.g., {"post": "I want to go to a model train exhibition"})

[0165] Output: The server receives the HTTP request.

[0166] 3. The server stores the received data in a database.

[0167] Specific behavior:

[0168] The server parses the data from the received POST request and stores it as a new entry in the database.

[0169] Input and Output:

[0170] Input: HTTP request content data

[0171] Output: A new entry is created in the database.

[0172] Step 2: Organizing information using natural language processing models

[0173] 1. The server periodically retrieves user posts from the database.

[0174] Specific behavior:

[0175] A scheduler on the server runs every day at 2 AM and retrieves all recently added user posts.

[0176] Input and Output:

[0177] Input: User posted data from database

[0178] Output: A list of retrieved user post data

[0179] 2. The server passes the acquired data to the natural language processing model.

[0180] Specific behavior:

[0181] The server formats user posts into API requests and sends them to a natural language processing model.

[0182] Input and Output:

[0183] Input: User post data in API request format

[0184] Output: Data sent to the natural language processing model

[0185] 3. A natural language processing model analyzes the data and generates summary information.

[0186] Specific behavior:

[0187] A natural language processing model analyzes multiple posts and summarizes relevant information.

[0188] Input and Output:

[0189] Input: User Post Data

[0190] Output: Summary information (e.g., "Summary of reviews of recent model railroad shows")

[0191] 4. The server stores the generated summary information in a database and provides it in a user-accessible format.

[0192] Specific behavior:

[0193] The server stores the summary information in a database and displays it in specific sections of the website or app.

[0194] Input and Output:

[0195] Input: Summary information

[0196] Output: Summary information stored in a database, summary information displayed on a website or app

[0197] Step 3: Travel planning assistance with conversational AI

[0198] 1. The user enters a travel planning question into the application.

[0199] Specific behavior:

[0200] A user types, "What train museum should I visit this weekend?"

[0201] Input and Output:

[0202] Input: User question text

[0203] Output: Text data is sent to the terminal.

[0204] 2. The device sends the query data to the server.

[0205] Specific behavior:

[0206] The app converts the question into JSON format and sends it to the server via an HTTP POST request.

[0207] Input and Output:

[0208] Input: JSON-formatted question data (e.g., {"query": "What railway museum should I visit this weekend?"})

[0209] Output: The server receives the HTTP request.

[0210] 3. The server uses conversational artificial intelligence to generate the best travel plan for the query.

[0211] Specific behavior:

[0212] The server analyzes the question data and sends it to a conversational artificial intelligence, which then generates travel destination and activity suggestions.

[0213] Input and Output:

[0214] Input: User question data

[0215] Output: Generated itinerary

[0216] 4. The server returns the generated itinerary to the user.

[0217] Specific behavior:

[0218] The server receives the conversational AI's response and sends it to the user's device.

[0219] Input and Output:

[0220] Input: Generated itinerary

[0221] Output: The itinerary sent to the user's device

[0222] Step 4: Providing content for virtual reality technology

[0223] 1. A user requests a virtual experience.

[0224] Specific behavior:

[0225] The user clicks the "Experience the Railway Museum virtually" button within the app.

[0226] Input and Output:

[0227] Input: Virtual experience request data

[0228] Output: Text data is sent to the terminal.

[0229] 2. The device sends the request data to the server.

[0230] Specific behavior:

[0231] The app converts the request content into JSON format and sends it to the server via an HTTP POST request.

[0232] Input and Output:

[0233] Input: Request data in JSON format (e.g., {"request": "Virtual experience at the Railway Museum"})

[0234] Output: The server receives the HTTP request.

[0235] 3. The server generates virtual reality technology content in real time based on the required data.

[0236] Specific behavior:

[0237] The server collects the relevant data and processes it to generate the virtual reality technology content.

[0238] Input and Output:

[0239] Input: Request data and related data

[0240] Output: Generated virtual reality technology content

[0241] 4. The server provides the generated virtual reality technology content to the user's terminal.

[0242] Specific behavior:

[0243] The server sends the generated content URL to the user's device, where the user experiences it using a dedicated app or VR headset.

[0244] Input and Output:

[0245] Input: Generated virtual reality technology content

[0246] Output: The content URL served to the user's device

[0247] (Application example 1)

[0248] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0249] Existing social media and information gathering methods lack the functionality to allow users to efficiently gather and share useful information related to niche hobbies. Furthermore, when physical or financial constraints make it difficult for users to visit the destination in person, their information gathering and experiences are limited. To solve these issues, efficient information organization, travel planning support, and the provision of virtual experiences using AR / VR technology are required.

[0250] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0251] In this invention, the server includes: means for receiving user posts from the device and storing them in a database; means for analyzing the data stored in the database using a generation AI and organizing and providing related information; means for the cloud server to periodically pass user data to the generation AI, which organizes and summarizes the related information and provides it to the user via a smartphone application; means for analyzing user inquiries using an interactive AI and proposing travel plans; means for dynamically generating and providing AR / VR content based on user requests; means for visualizing information provided to the device via a smartphone; means for sending input data to the cloud server and storing it in a database in text, image, or video format; and means for providing prompts to the generation AI based on user posts and requests to help organize information and generate travel plans. This allows users with niche hobbies to efficiently collect and share information and enjoy their hobbies through virtual experiences beyond physical and financial constraints.

[0252] A "terminal" is a device used to input user posts and send them to a server, such as a smartphone or tablet.

[0253] "User Posts" are data posted by users that includes information, experiences, opinions, images, and videos related to their specific hobbies or interests.

[0254] A "database" is a system that centrally stores and manages user posts and is operated by a server.

[0255] "Generative AI" is an artificial intelligence model used to analyze user posts and collect, organize, and summarize relevant information.

[0256] "Conversational AI" is an artificial intelligence model that analyzes user inquiries and generates optimal answers and suggestions.

[0257] A "travel plan" is a plan including destinations, transportation, accommodation, activities, etc., when a user travels.

[0258] "AR / VR content" refers to content that provides users with a virtual experience using augmented reality and virtual reality technologies.

[0259] A "cloud server" is a remote server used to store, process, and manage data over the Internet.

[0260] A "smartphone application" is software that runs primarily on smartphones and is used to collect user posts, organize information, provide travel plans, display AR / VR content, and more.

[0261] "Related information" is information extracted, organized, and summarized by the AI ​​based on information obtained from user posts, and includes content that may be useful to other users.

[0262] A "prompt sentence" is an input sentence that instructs the generation AI to perform a specific analysis or generate information.

[0263] The system for implementing this invention includes multiple means for efficiently analyzing data collected from users and organizing and suggesting information. As an application example, we will explain a specific system that allows users with niche hobbies to collect information related to their hobbies and interact with other users.

[0264] Hardware and software used

[0265] The hardware and software used to implement this invention are as follows:

[0266] Smartphone: A device used to input and visualize user posts and communicate with the cloud server.

[0267] Cloud server: A server for storing user posts and analyzing data using generation AI and conversational AI.

[0268] Database: Located on a cloud server, it centrally manages user-posted data.

[0269] Generative AI model: An artificial intelligence model (e.g., GPT-4) that analyzes user posts and organizes and summarizes relevant information.

[0270] Conversational AI model: An artificial intelligence model (e.g., Dialogflow) that analyzes user inquiries and generates and suggests optimal travel plans.

[0271] AR / VR Content Generation Tools: Tools for generating augmented reality and virtual reality content, such as Unity and Unreal Engine.

[0272] Program processing

[0273] Collecting user posts

[0274] Users use a smartphone application to input and post information about their hobbies. The smartphone sends this posted data to a cloud server and stores it in a database. The posted data can be in the form of text, images, videos, etc.

[0275] Information organization using generative AI

[0276] The cloud server periodically retrieves user posts from the database and passes them to the generation AI, which analyzes the data and organizes and summarizes the relevant information. The organized information is then made available to users in a form accessible through a smartphone application.

[0277] Travel planning support using conversational AI

[0278] When a user enters a travel planning question into a smartphone application, the smartphone sends the inquiry data to a cloud server, which uses conversational AI to generate an optimal travel plan based on the user's question and returns suggestions, including specific travel destinations, places to visit, and recommended activities.

[0279] Providing AR / VR content

[0280] When a user requests a virtual experience of a specific location or event, their smartphone sends the request to a cloud server. The cloud server generates AR / VR content in real time based on the necessary data and provides it to the user through a smartphone application. This allows users with physical or financial constraints to virtually experience activities that interest them.

[0281] Examples of concrete examples and prompts

[0282] For example, if a user interested in model trains posts that they want to go to a new model train exhibition, their smartphone will send this information to a server and store it in a database. Next, if another user asks, "What railway museum should I visit this weekend?", the conversational AI will suggest the optimal travel plan. Furthermore, users who cannot visit a railway museum can use AR / VR technology to virtually experience the inside of the museum.

[0283] Example prompt sentence:

[0284] "Please tell me the latest model train exhibition."

[0285] "Can you recommend a railway museum I can visit next weekend?"

[0286] "Show me a virtual tour of the model railroad exhibition."

[0287] In this way, the system utilizes generative and conversational AI, as well as AR / VR content generation tools, to enable users to gather information on their niche interests, plan trips, and provide virtual experiences.

[0288] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0289] Step 1:

[0290] Users use a smartphone application to input and post information about their hobbies. The input data includes text, images, videos, etc. The input data from the user is sent to the device.

[0291] Step 2:

[0292] The device sends user post data to the cloud server. The data arrives in the form of text, images, or videos and is stored in a database. The cloud server receives user posts and centralizes them in a database.

[0293] Step 3:

[0294] The cloud server periodically retrieves user posts from the database. This retrieved data is passed to the generation AI. The cloud server analyzes the contents of the database and passes the relevant data to the generation AI.

[0295] Step 4:

[0296] Generative AI analyzes user posts and organizes and summarizes relevant information. Input data is analyzed by generative AI, and the desired information is organized and output. Generative AI uses a model (e.g., GPT-4) to analyze text data and generate a summary of the information.

[0297] Step 5:

[0298] The cloud server retrieves the summarized data from the generated AI and provides it to the user via a smartphone application. The summarized data is output in a format that can be displayed on the user's smartphone. The cloud server then sends the organized and summarized information to the user.

[0299] Step 6:

[0300] A user inputs a question about their travel plan into a smartphone application. The user's question data is entered into the terminal. This input is sent to the conversational AI.

[0301] Step 7:

[0302] The device sends question data to the cloud server. The question data arrives at the cloud server and is received by the conversational AI. The conversational AI analyzes the question using a model (e.g., Dialogflow).

[0303] Step 8:

[0304] The conversational AI analyzes the user's inquiry and generates the optimal travel plan. The analyzed question data is processed by the conversational AI to generate the travel plan. The generated plan is sent to the cloud server.

[0305] Step 9:

[0306] The cloud server obtains the travel plan from the conversational AI and provides it to the user via a smartphone application. The generated travel plan is displayed on the smartphone in a format that is easy for the user to understand. The cloud server then sends the travel plan to the user's device.

[0307] Step 10:

[0308] A user requests a virtual experience of a specific location or event. The user's request data is entered into a smartphone application. This input is sent to a cloud server.

[0309] Step 11:

[0310] The cloud server receives the request and generates the virtual experience content using an AR / VR content generation tool (e.g., Unity, Unreal Engine). The request data reaches the cloud server, and the AR / VR content is generated based on the required data.

[0311] Step 12:

[0312] The cloud server provides the generated AR / VR content to the user via a smartphone application. The generated AR / VR content is displayed on the smartphone or head-mounted display. The cloud server provides the virtual experience to the user, allowing the user to view it.

[0313] This allows users to virtually experience activities that interest them, regardless of physical or financial constraints.

[0314] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0315] This invention is a system that allows users with particularly niche hobbies to collect information and interact with the community. This system combines the collection of user posts, information organization using generative AI, travel planning support using conversational AI, provision of AR / VR content, and an emotion engine that recognizes and responds to user emotions. The details of this system are described below.

[0316] Collecting user posts

[0317] Users post information about their hobbies and experiences through applications and websites. The device receives this posted data and sends it to a server. The server stores the received data in a database. This data can be in the form of text, images, videos, etc., and is centrally managed in the database.

[0318] Information organization using generative AI

[0319] The server periodically retrieves user posts from the database and passes them to the generation AI, which analyzes this data, organizes and summarizes the relevant information, allowing other users to efficiently obtain the information they need. The summarized information is then provided to users in an accessible format.

[0320] Travel planning support using conversational AI

[0321] When a user enters a travel planning question into the application, the device sends the query data to the server, which uses conversational AI to generate and return a travel itinerary that best suits the user's query, including specific travel destinations, places to visit, and recommended activities.

[0322] Providing AR / VR content

[0323] For users who find it difficult to travel or attend events, the server uses AR / VR technology to provide rich virtual experiences. When a user requests a virtual experience of a specific location or event, the device sends the request to the server. The server generates AR / VR content in real time based on the necessary data and provides the virtual experience. This allows users with financial or physical limitations to enjoy activities that interest them.

[0324] Response by Emotion Engine

[0325] While the user is using the system, the emotion engine analyzes the user's emotions. For example, it determines the user's emotional state from the user's text input, voice, and facial expressions. The server uses this emotion data to adjust the information provided by the generative AI and the travel plans proposed by the conversational AI. For example, if the user is feeling stressed, it can suggest travel destinations and activities aimed at relaxation.

[0326] Specific examples

[0327] For example, if a user interested in model trains posts with a positive emotion, "I want to go to a new model train exhibition," this information is stored in a database and shared with other users by the generative AI. Next, if another user uses the conversational AI to ask, "What train museums should I visit this weekend?", the conversational AI will suggest the best travel plan to answer that question. Similarly, if a user requests a virtual experience with an emotion expressing fatigue, the AI ​​can suggest relaxing AR / VR content.

[0328] This will enable users with particularly niche hobbies to enrich their hobby activities by collecting and sharing information, planning trips, and engaging in virtual experiences.The introduction of an emotion engine is expected to provide personalized experiences based on the user's emotional state, improving engagement across the community.

[0329] The processing flow will be explained below.

[0330] Collecting and organizing user posts

[0331] Collecting user posts

[0332] Step 1:

[0333] User: Enters submissions into application or website forms.

[0334] Step 2:

[0335] Terminal: Sends the entered post content to the server.

[0336] Step 3:

[0337] Server: Stores the received submission data in a database. The submission data includes text, images, videos, etc.

[0338] Information organization

[0339] Step 4:

[0340] Server: Periodically retrieves user posts from the database.

[0341] Step 5:

[0342] Server: Passes the acquired data to the generation AI, which analyzes and summarizes the information.

[0343] Step 6:

[0344] Server: Provides summarized information in a form that users can access, allowing other users to efficiently retrieve the information they need.

[0345] Travel planning support using conversational AI

[0346] Step 1:

[0347] User: Enters a travel planning question into a text box in the application.

[0348] Step 2:

[0349] Terminal: Sends the entered question to the server.

[0350] Step 3:

[0351] Server: Analyzes user questions using conversational AI and generates travel plans.

[0352] Step 4:

[0353] Server: Returns the generated itinerary to the user.

[0354] Step 5:

[0355] On the device: Show the user a suggested travel itinerary, including specific destinations, places to visit, and recommended activities.

[0356] Providing AR / VR content

[0357] Step 1:

[0358] User: Requests a virtual experience of a specific location or event through a form in the app.

[0359] Step 2:

[0360] Terminal: Sends the input request to the server.

[0361] Step 3:

[0362] Server: Retrieves relevant data from a database based on the requested location or event.

[0363] Step 4:

[0364] Server: Generates the required AR / VR content in real time.

[0365] Step 5:

[0366] Server: Sends the generated AR / VR content to the user.

[0367] Step 6:

[0368] Device: The received AR / VR content is displayed on the user's device. Users can enjoy the virtual experience using a dedicated device or app.

[0369] Response by Emotion Engine

[0370] Step 1:

[0371] User: Providing emotion-based text input, voice, facial expressions, etc. while using the system.

[0372] Step 2:

[0373] Device: Sends emotion-related data to the server.

[0374] Step 3:

[0375] Server: The emotion engine analyzes the received data and determines the user's emotional state.

[0376] Step 4:

[0377] Server: Based on the emotion data, adjust the information provided by the generative AI and the travel plan proposed by the conversational AI. For example, if the user expresses stress, suggest places and activities that will help them relax.

[0378] Step 5:

[0379] Server: Provide users with tailored information and plans. By providing information that is tailored to their emotions, users can have a more satisfying experience.

[0380] Specific examples

[0381] For example, consider the case of a user who wants to go to a model railroad exhibition.

[0382] Step 1:

[0383] User: Posts "I want to go to the new model train show." Shows positive sentiment.

[0384] Step 2:

[0385] Terminal: Sends the post to the server.

[0386] Step 3:

[0387] Server: Save the submitted data to a database.

[0388] Step 4:

[0389] Server: Periodically retrieves posted data and summarizes it using a generation AI.

[0390] Step 5:

[0391] Server: If the emotion engine confirms a user's positive emotions, it provides information that promotes a positive experience for other users.

[0392] Step 6:

[0393] User: "Tell me which train museum I should visit this weekend," asks the conversational AI, showing some tiredness.

[0394] Step 7:

[0395] Terminal: Sends the question to the server.

[0396] Step 8:

[0397] Server: Conversational AI generates optimal travel plans and suggests relaxing places that match the user's emotional state.

[0398] Step 9:

[0399] Server: Sends the proposal back to the user.

[0400] Step 10:

[0401] User: Requests AR / VR virtual experience with emotion indicating fatigue.

[0402] Step 11:

[0403] Terminal: Sends the request to the server.

[0404] Step 12:

[0405] Server: Generates and provides relaxing AR / VR content in real time.

[0406] Step 13:

[0407] Device: Displays the received AR / VR content on the user's device.

[0408] This will enable users with particularly niche hobbies to enrich their hobby activities by collecting and sharing information, planning trips, and engaging in virtual experiences.The introduction of an emotion engine is expected to provide personalized experiences based on the user's emotional state, improving engagement across the community.

[0409] Example 2

[0410] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0411] Until now, there have been no systems that can collect and share information, plan trips, provide virtual experiences, or respond to users' emotions for users with niche hobbies. As a result, users have been unable to efficiently obtain information that interests them, and have difficulty receiving appropriate activity suggestions and virtual experiences, resulting in a decline in user experience and community engagement.

[0412] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0413] In this invention, the server includes means for receiving user posts from the terminal and storing them in a database, means for analyzing the data stored in the database using a generation AI and organizing and providing related information, means for analyzing user inquiries using a conversational AI and proposing travel plans, means including an emotion engine for analyzing user emotions and adjusting the output of the generation AI and the conversational AI, and means for dynamically generating and providing AR / VR content based on user requests. This allows users to efficiently collect information and receive appropriate travel plans and virtual experiences, and further enables the provision of personalized experiences according to the user's emotions.

[0414] "Terminal" refers to a device used by a user, including smartphones and personal computers.

[0415] "User Posts" refers to hobby-related information and experiences posted by Users through the Application or Website.

[0416] "Database" refers to a data storage system for centrally storing and managing collected data.

[0417] "Generative AI" is a system that uses artificial intelligence to generate and analyze text, images, etc., and is used for the purpose of organizing and summarizing information.

[0418] "Conversational AI" is artificial intelligence that generates answers in natural language to user inquiries, and is used for question answering and information provision.

[0419] An "emotion engine" is a system that analyzes a user's text input, voice, facial expressions, etc. to determine their emotional state and adjusts the output of other functions based on the results.

[0420] "AR / VR content" refers to rich virtual experiences generated using augmented reality (AR) and virtual reality (VR) technologies.

[0421] The present invention is a system for users with particularly niche interests to collect information and interact with communities. This system combines collection of user posts, information organization using generative AI, travel planning support using conversational AI, provision of AR / VR content, and an emotion engine that recognizes and responds to user emotions. Specific embodiments of the present invention are described below.

[0422] Collecting user posts

[0423] A user accesses an application or website and posts information about their hobbies and experiences. The device receives this posted data and sends it to a server. The server stores the received data in a database. This data can be in the form of text, images, videos, etc., and is centrally managed in a database (e.g., MySQL).

[0424] Information organization using generative AI

[0425] The server periodically retrieves user posts from the database and passes them to the Generative AI, which (e.g., OpenAI GPT-4) analyzes the data and organizes and summarizes the relevant information. The summarized information is then made available by the server in a form that can be accessed by other users.

[0426] Travel planning support using conversational AI

[0427] When a user enters a travel plan question into the application, the device sends the question data to a server, which uses conversational AI (e.g., Google Dialogflow) to generate a travel plan that best suits the user's question and returns suggestions, including specific travel destinations, places to visit, and recommended activities.

[0428] Providing AR / VR content

[0429] For users who find it difficult to travel or attend events, the server uses AR / VR technology to provide rich virtual experiences. When a user requests a virtual experience of a specific location or event, the device sends the request to the server. The server generates AR / VR content in real time based on the necessary data and provides it to the user. This allows users with financial or physical constraints to enjoy activities that interest them. The specific software used includes Unity.

[0430] Response by Emotion Engine

[0431] While the user is using the system, the emotion engine analyzes the user's emotional state from text input, voice, facial expressions, etc. The server uses this emotional data to adjust the information provided by the generative AI and the travel plans suggested by the conversational AI. For example, if the user is feeling stressed, it can suggest travel destinations and activities aimed at relaxation. The specific software used includes Microsoft Azure Cognitive Services.

[0432] Specific examples

[0433] For example, if a user interested in model trains posts a positive message saying, "I want to go to a new model train exhibition," the device sends the message to the server, which stores it in a database. This information is then organized by the generative AI and shared with other users.

[0434] Next, another user can use the conversational AI to ask, "What railway museums should I visit this weekend?" and the conversational AI will suggest the best travel plan to answer that question.

[0435] In addition, for users who express fatigue and request a "relaxing virtual experience," the system can suggest relaxing AR / VR content.

[0436] In this way, the system of the present invention enables efficient information gathering, personalized travel planning, and virtual experiences, especially for users with niche hobbies, thereby increasing user engagement and further enriching their hobby activities.

[0437] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0438] Step 1:

[0439] Users access applications or websites and post information about their hobbies and experiences. The data entered can be in the form of text, images, videos, etc. The device receives this posted data and sends it to the server.

[0440] Specific behavior:

[0441] A user enters the text "I want to go to the new model train exhibition" into the app, attaches an image and posts it.

[0442] The terminal sends this as text data and image data to the server. The input is text and image data, and the output is data sent to the server.

[0443] Step 2:

[0444] The server stores the received submission data in a database. The data can be in various formats, such as text, images, and videos, and is managed centrally.

[0445] Specific behavior:

[0446] The server executes a SQL query to insert the received data (text and images) into the database. The input is the received data, and the output is saving it to the database.

[0447] Step 3:

[0448] The server periodically retrieves user posts from the database and passes them to the generation AI, which analyzes the data and organizes and summarizes the relevant information.

[0449] Specific behavior:

[0450] The server retrieves new post data saved from the previous day from the database at a fixed time every day.

[0451] The acquired data is input into the generative AI (OpenAI GPT-4) to generate summarized information. The input is data acquired from the database, and the output is summarized information.

[0452] Step 4:

[0453] The server stores the generated summary information back in a database, making it accessible to other users.

[0454] Specific behavior:

[0455] The server inserts the summary data output by the generation AI into the database.

[0456] This information is displayed on a website or application for users to view. The input is the output data of the generative AI, and the output is stored in a database and provided to the user.

[0457] Step 5:

[0458] The user inputs a travel planning question within the application, and the device sends the question data to the server.

[0459] Specific behavior:

[0460] A user types, "What train museum should I visit this weekend?"

[0461] The terminal sends this text data to the server. The input is the user's question data, and the output is the data sent to the server.

[0462] Step 6:

[0463] The server uses conversational AI to generate the optimal travel plan for the user's question. The conversational AI generates an answer to the question.

[0464] Specific behavior:

[0465] The server inputs question data into a conversational AI (Google Dialogflow) and generates a travel plan.

[0466] The conversational AI generates a list of optimal travel destinations and places to visit and returns it to the server. The input is the question data, and the output is the generated travel plan.

[0467] Step 7:

[0468] The server returns the generated itinerary to the terminal and displays it to the user.

[0469] Specific behavior:

[0470] The server sends the output of the conversational AI to the terminal.

[0471] The device displays the optimal travel plan to the user. The input is the output data of the conversational AI, and the output is the display to the user.

[0472] Step 8:

[0473] The user requests a virtual experience of a specific location or event, and the device sends this request to the server.

[0474] Specific behavior:

[0475] A user requests a virtual experience of a model railroad exhibition.

[0476] The terminal sends this request data to the server. The input is the user's request data, and the output is the data sent to the server.

[0477] Step 9:

[0478] The server uses AR / VR technology to generate a rich virtual experience and deliver it to the user's device. The software used includes Unity.

[0479] Specific behavior:

[0480] The server uses Unity to generate the virtual experience.

[0481] The generated AR / VR content is sent to the device so that the user can experience it. The input is the user's request data, and the output is the generated AR / VR content.

[0482] Step 10:

[0483] While the user is using the system, the emotion engine analyzes the user's emotions, and the server uses this emotional data to adjust the output of the generative and conversational AI.

[0484] Specific behavior:

[0485] When a user types a question within the app, the text, voice data, and facial expressions are input into an emotion analysis engine.

[0486] The server analyzes the emotion data using an emotion analysis engine (Microsoft Azure Cognitive Services) and feeds it back to the generative AI and conversational AI to provide information and adjust travel plans. The input is the user's emotion data, and the output is adjusted information.

[0487] (Application example 2)

[0488] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0489] Currently, there are limited platforms that allow users with diverse hobbies to efficiently gather information that matches their interests and interact with communities that share the same hobbies. Furthermore, users often have unsatisfactory experiences due to the effort required to gather information when planning a trip and the lack of personalized product purchasing experiences. Furthermore, users are not provided with information or experiences that are in line with their emotions, which leads to low user engagement.

[0490] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving user posts from the terminal and storing them in a database, means for analyzing the data stored in the database using a generation AI and organizing and providing related information, means for analyzing user inquiries using an interactive AI and proposing travel plans, means for dynamically generating and providing AR / VR content based on user requests, means for providing information on products and services related to the user posts in a virtual shopping mall, and an emotion engine for analyzing user emotions and providing personalized content and product suggestions based on the analysis. This not only enables users to efficiently collect information on niche hobbies and interact with communities, but also enables activities such as travel plans, virtual experiences, and product purchases to be personalized, enabling the provision of services that are tailored to their emotions.

[0491] A "terminal" is an electronic device through which a user inputs information and communicates with the system.

[0492] "User posts" are content such as text, images, and videos of information and experiences posted by users via their devices.

[0493] A "database" is a system that centrally stores and manages data such as received user posts.

[0494] "Generative AI" is an artificial intelligence technology that analyzes collected data and organizes and summarizes related information.

[0495] "Conversational AI" is an artificial intelligence technology that analyzes user inquiries and provides optimal answers and plans.

[0496] A "travel plan" is a plan that includes locations, activities, accommodations, etc., when a user travels.

[0497] "AR / VR content" means visual and experiential content created using augmented reality (AR) and virtual reality (VR) technologies.

[0498] A "virtual shopping mall" is an e-commerce platform where users can explore and purchase goods and services in a virtual environment.

[0499] The "emotion engine" is a function that analyzes the user's emotional state and provides personalized services and content based on the results.

[0500] A "system" is a set of technical components in which multiple means work together to achieve a specific purpose.

[0501] The system for realizing this invention operates by combining multiple means. The system is mainly composed of terminals, servers, analysis of user posts, and the introduction of new technologies.

[0502] First, users use their devices to post information about their hobbies and experiences. This information is received in the form of text, images, and videos and is stored in a database via a server. The database serves to organize and centrally manage the information.

[0503] The server then periodically retrieves user posts from the database and analyzes them using a generative AI, which uses AI technologies such as TensorFlow.js to organize and summarize the collected data into relevant information, allowing other users to efficiently access the information they need.

[0504] Furthermore, when users ask questions about their travel plans, the server uses conversational AI to suggest optimal travel plans, such as using Dialogflow to generate answers to the user's questions and provide them to the user. This plan includes travel destinations, places to visit, and recommended activities.

[0505] AR / VR content can also be provided through the device. When a user requests a virtual experience of a specific location or event, the server generates the AR / VR content in real time using Unity or WebXR and provides it to the user. This allows users with financial or physical limitations to virtually experience activities that interest them.

[0506] Finally, an emotion engine is used to analyze the user's emotional state, for example, by determining emotions from the user's text input, voice, or facial expressions, and then adjust the information provided by the generative AI and the travel itinerary suggested by the conversational AI. The emotion engine then provides personalized product suggestions and content to the user.

[0507] Specific examples

[0508] For example, if a user interested in model trains posts with positive emotions, "I want to go to a new model train exhibition," this information is stored in a database and shared with other users by the generative AI. Next, if another user uses the conversational AI to ask, "What railway museum should I visit this weekend?", the conversational AI will suggest the best travel plan. Additionally, if a user feels tired, the AI ​​can suggest relaxing AR / VR content.

[0509] Prompt Sentence Examples

[0510] "Can you give me some information about the latest model railroad exhibition?"

[0511] "Show me your new model train station."

[0512] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0513] Step 1:

[0514] Users post information about their hobbies and experiences through their devices. These posts are entered in the form of text, images, videos, etc., and are sent from the device to the server. The server receives this data and stores it in a database. During this process, the server properly organizes the data format and metadata to centrally manage the posted data.

[0515] Step 2:

[0516] The server periodically retrieves user posts from the database. It then uses a generative AI to analyze the data. This analysis is used to organize and summarize relevant information. The generative AI uses TensorFlow.js to perform semantic analysis of text data and image recognition to effectively organize information. For example, it extracts keywords from posted text and uses them to summarize related information.

[0517] Step 3:

[0518] The server receives questions from users. For example, if a user types "Please tell me which railway museum I should visit this weekend" through a terminal, this question is sent to the server. The server analyzes this inquiry using a conversational AI (e.g., Dialogflow). Based on the results of the analysis, it generates an optimal travel plan and proposes it to the user. In this case, the server obtains answer data for the question from the generation AI and provides a personalized answer.

[0519] Step 4:

[0520] When a user requests a virtual experience of a specific location or event, the device sends this request to the server. The server uses Unity or WebXR to dynamically generate AR / VR content based on the request. The generated content is then delivered to the user in real time. For example, if a user requests, "Show me the new model train exhibition," the server generates a virtual exhibition and delivers it to the user.

[0521] Step 5:

[0522] The server analyzes the user's emotional state. It collects data such as text input, voice, and facial expressions while the user is using the device and analyzes it using an emotion engine. Based on this analysis, it personalizes the information and content provided to the user. For example, if the user is feeling stressed, the emotion engine will suggest content and products suitable for relaxation.

[0523] As described above, each step works in conjunction to create a system that provides users with an efficient and personalized experience.

[0524] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0525] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0526] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0527] [Second embodiment]

[0528] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0529] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0530] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0531] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0532] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0533] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0534] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0535] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0536] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0537] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0538] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0539] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0540] This invention is a system for users with particularly niche interests to collect information and interact with the community. This system has four main functions: collecting user posts, organizing information using generative AI, supporting travel planning using conversational AI, and providing AR / VR content. The details of this system are explained below.

[0541] Collecting user posts

[0542] Users post information about their hobbies and experiences through applications and websites. The device receives this posted data and sends it to a server. The server stores the received data in a database. This data can be in the form of text, images, videos, etc., and is centrally managed in the database.

[0543] Information organization using generative AI

[0544] The server periodically retrieves user posts from the database and passes them to the generation AI, which analyzes this data, organizes and summarizes the relevant information, allowing other users to efficiently obtain the information they need. The summarized information is then provided to users in an accessible format.

[0545] Travel planning support using conversational AI

[0546] When a user enters a travel planning question into the application, the device sends the query data to the server, which uses conversational AI to generate and return a travel itinerary that best suits the user's query, including specific travel destinations, places to visit, and recommended activities.

[0547] Providing AR / VR content

[0548] For users who find it difficult to travel or attend events, the server uses AR / VR technology to provide rich virtual experiences. When a user requests a virtual experience of a specific location or event, the device sends the request to the server. The server generates AR / VR content in real time based on the necessary data and provides the virtual experience. This allows users with financial or physical limitations to enjoy activities that interest them.

[0549] Specific examples

[0550] For example, if a user interested in model trains posts, "I want to go to a new model train exhibition," this information is stored in a database. Then, if another user uses the conversational AI to ask, "What train museum should I visit this weekend?", the conversational AI will suggest the best travel plan to answer that question. Furthermore, users who cannot physically visit museums can use AR / VR technology to virtually experience the inside of the museum.

[0551] This invention enables users with particularly niche hobbies to enrich their hobby activities by collecting and sharing information, planning trips, and engaging in virtual experiences, which is expected to promote interaction between users and improve engagement across the community.

[0552] The processing flow will be explained below.

[0553] Collecting and organizing user posts

[0554] Collecting user posts

[0555] Step 1:

[0556] User: Enters submissions into application or website forms.

[0557] Step 2:

[0558] Terminal: Sends the entered post content to the server.

[0559] Step 3:

[0560] Server: Stores the received submission data in a database.

[0561] Information organization

[0562] Step 4:

[0563] Server: Periodically retrieves user posts from the database.

[0564] Step 5:

[0565] Server: Passes the acquired data to the generation AI, which analyzes and summarizes the information.

[0566] Step 6:

[0567] Server: Provides summarized information in a form that users can access.

[0568] Travel planning support using conversational AI

[0569] Step 1:

[0570] User: Enters a travel planning question into a text box in the application.

[0571] Step 2:

[0572] Terminal: Sends the entered question to the server.

[0573] Step 3:

[0574] Server: Analyzes user questions using conversational AI and generates travel plans.

[0575] Step 4:

[0576] Server: Returns the generated itinerary to the user.

[0577] Step 5:

[0578] Terminal: Shows suggested travel itineraries to the user.

[0579] Providing AR / VR content

[0580] Step 1:

[0581] User: Requests a virtual experience of a specific location or event through a form in the app.

[0582] Step 2:

[0583] Terminal: Sends the input request to the server.

[0584] Step 3:

[0585] Server: Retrieves relevant data from a database based on the requested location or event.

[0586] Step 4:

[0587] Server: Generates the required AR / VR content in real time.

[0588] Step 5:

[0589] Server: Sends the generated AR / VR content to the user.

[0590] Step 6:

[0591] Device: Displays the received AR / VR content on the user's device.

[0592] Specific examples

[0593] Step 1:

[0594] User: Posts information about a model railroad show.

[0595] Step 2:

[0596] Terminal: Sends the post to the server.

[0597] Step 3:

[0598] Server: Stores the received posts in a database.

[0599] Step 4:

[0600] Server: Periodically retrieves posted data and summarizes it using a generation AI.

[0601] Step 5:

[0602] User: Finds museums to visit next weekend and asks the conversational AI a question.

[0603] Step 6:

[0604] Terminal: Sends the question to the server.

[0605] Step 7:

[0606] Server: Conversational AI generates optimal travel plans and suggests them to users.

[0607] Step 8:

[0608] User: If you can't make it to the museum, request a virtual experience.

[0609] Step 9:

[0610] Terminal: Sends the request to the server.

[0611] Step 10:

[0612] Server: Generates AR / VR content and provides it to users.

[0613] Step 11:

[0614] Device: Displays the received AR / VR content on the user's device.

[0615] This allows users to efficiently gather relevant information and enjoy travel and virtual experiences through a dedicated system, particularly for those with physical limitations, allowing them to enrich their hobbies.

[0616] Example 1

[0617] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0618] Conventional information gathering and community systems make it difficult for users with niche interests to efficiently collect and share information in an organized manner. They also lack the means for users to receive interactive assistance when planning trips, or for users with physical limitations to enjoy rich virtual experiences. As a result, it is difficult to provide comprehensive services to users with specific interests.

[0619] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0620] In this invention, the server includes means for receiving user posts from terminals and storing them in a database, means for analyzing the data stored in the database using a natural language processing model and organizing and providing related information, means for analyzing user inquiries using interactive artificial intelligence and proposing travel plans, means for dynamically generating and providing virtual reality technology content based on user requests, and means for retrieving user posts from the database and summarizing them using a natural language processing model. This allows users to efficiently collect information, receive travel plan suggestions, and enjoy rich experiences through virtual reality technology.

[0621] A "terminal" is a device that allows a user to input or receive information, and is often a smartphone, tablet, or computer.

[0622] "User Posts" refers to hobby-related information and experiences that users share through applications and websites, often in the form of text, images, videos, etc.

[0623] "Database" refers to a system for efficiently managing, storing, and retrieving data in a structured format.

[0624] A "natural language processing model" refers to an artificial intelligence technology that analyzes natural language data entered by a user and understands, generates, and summarizes human language.

[0625] "Conversational artificial intelligence" refers to a conversational artificial intelligence system that can provide natural responses to user questions and commands.

[0626] "Virtual reality technology" refers to technology that allows users to visually and aurally experience places and experiences that do not actually exist in the physical world.

[0627] "Means for organizing and providing information" refers to a method for using generative AI to analyze information stored in a database and provide related information to users in an easy-to-understand format.

[0628] "Means for proposing travel plans" refers to a method for using conversational AI to generate and propose optimal travel plans in response to user inquiries.

[0629] "Dynamic generation and serving means" refers to a method for creating and serving content in real time based on user requests.

[0630] "Means for summarizing" refers to a method for concisely summarizing multiple user-posted information using a natural language processing model.

[0631] This invention is a system for users with particularly niche interests to collect information and interact with the community. The system has four main functions: collecting user posts, organizing information using natural language processing models, assisting with travel planning using conversational artificial intelligence, and providing content using virtual reality technology.

[0632] Collecting user posts

[0633] Users post information about their hobbies and experiences through applications and websites. This data can be in the form of text, images, videos, etc.

[0634] The device receives the user's posted data and sends it to the server. For example, if a user posts "I want to go to the new model train exhibition," the device sends this to the server.

[0635] The server stores the received data in a database, which is a means of efficiently managing and centralizing this data.

[0636] Information organization using natural language processing models

[0637] The server periodically retrieves user posts from the database and passes them to a natural language processing model, such as OpenAI's GPT-3.

[0638] Natural language processing models analyze user posts and organize and summarize relevant information. The generated summaries are a means for other users to efficiently access information. For example, they can concisely summarize reviews of a model train exhibition posted by multiple users.

[0639] The server stores the summarized information in a database and makes it accessible to users.

[0640] Travel planning support using conversational AI

[0641] A user enters a travel planning question into an application, for example, "What train museums should I visit this weekend?"

[0642] The terminal transmits this inquiry data to the server.

[0643] The server uses conversational artificial intelligence (e.g., Dialogflow) to generate an optimal travel plan based on the user's questions, including destinations, places to visit, and recommended activities.

[0644] The server returns the generated itinerary to the user, who can view it within the app.

[0645] Virtual reality technology content provision

[0646] A user requests a virtual experience. For example, a request to "virtually experience the Railway Museum" is made within the app.

[0647] The terminal transmits this request data to the server.

[0648] The server collects relevant data and generates content in real time using virtual reality technology, which transcends physical limitations to provide users with a rich experience.

[0649] The server sends the generated content to the user's device, and the user experiences it using a dedicated app or VR headset.

[0650] Examples and prompts

[0651] For example, if a user who is interested in model trains posts that they want to go to a new model train exhibition, this information is stored in the database. Then, if another user uses the conversational AI to ask, "What train museum should I visit this weekend?", the conversational AI will suggest the best travel plan to answer that question. Additionally, users who cannot physically visit museums can use virtual reality technology to virtually experience the inside of the museum.

[0652] An example of a prompt to input to a generative AI model is as follows:

[0653] "I'm interested in model trains. Which railway museum should I visit this weekend? Also, what should I look out for when I visit?"

[0654] This invention is expected to enable users with particularly niche interests to efficiently gather information, receive travel plan suggestions, and enjoy rich virtual experiences, thereby increasing engagement across the community.

[0655] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0656] Processing Steps

[0657] Step 1: Collect user posts

[0658] 1. Users post information and experiences about their hobbies through the application or website.

[0659] Specific behavior:

[0660] A user posts, "I want to go to a model train exhibition."

[0661] Input and Output:

[0662] Input: Text data entered by the user (e.g., "I want to go to a model train exhibition")

[0663] Output: Text data is sent to the terminal.

[0664] 2. The terminal receives the posted data from the user and sends it to the server.

[0665] Specific behavior:

[0666] The app converts user input into JSON format and sends it to the server via an HTTP POST request.

[0667] Input and Output:

[0668] Input: JSON format text data (e.g., {"post": "I want to go to a model train exhibition"})

[0669] Output: The server receives the HTTP request.

[0670] 3. The server stores the received data in a database.

[0671] Specific behavior:

[0672] The server parses the data from the received POST request and stores it as a new entry in the database.

[0673] Input and Output:

[0674] Input: HTTP request content data

[0675] Output: A new entry is created in the database.

[0676] Step 2: Organizing information using natural language processing models

[0677] 1. The server periodically retrieves user posts from the database.

[0678] Specific behavior:

[0679] A scheduler on the server runs every day at 2 AM and retrieves all recently added user posts.

[0680] Input and Output:

[0681] Input: User posted data from database

[0682] Output: A list of retrieved user post data

[0683] 2. The server passes the acquired data to the natural language processing model.

[0684] Specific behavior:

[0685] The server formats user posts into API requests and sends them to a natural language processing model.

[0686] Input and Output:

[0687] Input: User post data in API request format

[0688] Output: Data sent to the natural language processing model

[0689] 3. A natural language processing model analyzes the data and generates summary information.

[0690] Specific behavior:

[0691] A natural language processing model analyzes multiple posts and summarizes relevant information.

[0692] Input and Output:

[0693] Input: User Post Data

[0694] Output: Summary information (e.g., "Summary of reviews of recent model railroad shows")

[0695] 4. The server stores the generated summary information in a database and provides it in a user-accessible format.

[0696] Specific behavior:

[0697] The server stores the summary information in a database and displays it in specific sections of the website or app.

[0698] Input and Output:

[0699] Input: Summary information

[0700] Output: Summary information stored in a database, summary information displayed on a website or app

[0701] Step 3: Travel planning assistance with conversational AI

[0702] 1. The user enters a travel planning question into the application.

[0703] Specific behavior:

[0704] A user types, "What train museum should I visit this weekend?"

[0705] Input and Output:

[0706] Input: User question text

[0707] Output: Text data is sent to the terminal.

[0708] 2. The device sends the query data to the server.

[0709] Specific behavior:

[0710] The app converts the question into JSON format and sends it to the server via an HTTP POST request.

[0711] Input and Output:

[0712] Input: JSON-formatted question data (e.g., {"query": "What railway museum should I visit this weekend?"})

[0713] Output: The server receives the HTTP request.

[0714] 3. The server uses conversational artificial intelligence to generate the best travel plan for the query.

[0715] Specific behavior:

[0716] The server analyzes the question data and sends it to a conversational artificial intelligence, which then generates travel destination and activity suggestions.

[0717] Input and Output:

[0718] Input: User question data

[0719] Output: Generated itinerary

[0720] 4. The server returns the generated itinerary to the user.

[0721] Specific behavior:

[0722] The server receives the conversational AI's response and sends it to the user's device.

[0723] Input and Output:

[0724] Input: Generated itinerary

[0725] Output: The itinerary sent to the user's device

[0726] Step 4: Providing content for virtual reality technology

[0727] 1. A user requests a virtual experience.

[0728] Specific behavior:

[0729] The user clicks the "Experience the Railway Museum virtually" button within the app.

[0730] Input and Output:

[0731] Input: Virtual experience request data

[0732] Output: Text data is sent to the terminal.

[0733] 2. The device sends the request data to the server.

[0734] Specific behavior:

[0735] The app converts the request content into JSON format and sends it to the server via an HTTP POST request.

[0736] Input and Output:

[0737] Input: Request data in JSON format (e.g., {"request": "Virtual experience at the Railway Museum"})

[0738] Output: The server receives the HTTP request.

[0739] 3. The server generates virtual reality technology content in real time based on the required data.

[0740] Specific behavior:

[0741] The server collects the relevant data and processes it to generate the virtual reality technology content.

[0742] Input and Output:

[0743] Input: Request data and related data

[0744] Output: Generated virtual reality technology content

[0745] 4. The server provides the generated virtual reality technology content to the user's terminal.

[0746] Specific behavior:

[0747] The server sends the generated content URL to the user's device, where the user experiences it using a dedicated app or VR headset.

[0748] Input and Output:

[0749] Input: Generated virtual reality technology content

[0750] Output: The content URL served to the user's device

[0751] (Application example 1)

[0752] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0753] Existing social media and information gathering methods lack the functionality to allow users to efficiently gather and share useful information related to niche hobbies. Furthermore, when physical or financial constraints make it difficult for users to visit the destination in person, their information gathering and experiences are limited. To solve these issues, efficient information organization, travel planning support, and the provision of virtual experiences using AR / VR technology are required.

[0754] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0755] In this invention, the server includes: means for receiving user posts from the device and storing them in a database; means for analyzing the data stored in the database using a generation AI and organizing and providing related information; means for the cloud server to periodically pass user data to the generation AI, which organizes and summarizes the related information and provides it to the user via a smartphone application; means for analyzing user inquiries using an interactive AI and proposing travel plans; means for dynamically generating and providing AR / VR content based on user requests; means for visualizing information provided to the device via a smartphone; means for sending input data to the cloud server and storing it in a database in text, image, or video format; and means for providing prompts to the generation AI based on user posts and requests to help organize information and generate travel plans. This allows users with niche hobbies to efficiently collect and share information and enjoy their hobbies through virtual experiences beyond physical and financial constraints.

[0756] A "terminal" is a device used to input user posts and send them to a server, such as a smartphone or tablet.

[0757] "User Posts" are data posted by users that includes information, experiences, opinions, images, and videos related to their specific hobbies or interests.

[0758] A "database" is a system that centrally stores and manages user posts and is operated by a server.

[0759] "Generative AI" is an artificial intelligence model used to analyze user posts and collect, organize, and summarize relevant information.

[0760] "Conversational AI" is an artificial intelligence model that analyzes user inquiries and generates optimal answers and suggestions.

[0761] A "travel plan" is a plan including destinations, transportation, accommodation, activities, etc., when a user travels.

[0762] "AR / VR content" refers to content that provides users with a virtual experience using augmented reality and virtual reality technologies.

[0763] A "cloud server" is a remote server used to store, process, and manage data over the Internet.

[0764] A "smartphone application" is software that runs primarily on smartphones and is used to collect user posts, organize information, provide travel plans, display AR / VR content, and more.

[0765] "Related information" is information extracted, organized, and summarized by the AI ​​based on information obtained from user posts, and includes content that may be useful to other users.

[0766] A "prompt sentence" is an input sentence that instructs the generation AI to perform a specific analysis or generate information.

[0767] The system for implementing this invention includes multiple means for efficiently analyzing data collected from users and organizing and suggesting information. As an application example, we will explain a specific system that allows users with niche hobbies to collect information related to their hobbies and interact with other users.

[0768] Hardware and software used

[0769] The hardware and software used to implement this invention are as follows:

[0770] Smartphone: A device used to input and visualize user posts and communicate with the cloud server.

[0771] Cloud server: A server for storing user posts and analyzing data using generation AI and conversational AI.

[0772] Database: Located on a cloud server, it centrally manages user-posted data.

[0773] Generative AI model: An artificial intelligence model (e.g., GPT-4) that analyzes user posts and organizes and summarizes relevant information.

[0774] Conversational AI model: An artificial intelligence model (e.g., Dialogflow) that analyzes user inquiries and generates and suggests optimal travel plans.

[0775] AR / VR Content Generation Tools: Tools for generating augmented reality and virtual reality content, such as Unity and Unreal Engine.

[0776] Program processing

[0777] Collecting user posts

[0778] Users use a smartphone application to input and post information about their hobbies. The smartphone sends this posted data to a cloud server and stores it in a database. The posted data can be in the form of text, images, videos, etc.

[0779] Information organization using generative AI

[0780] The cloud server periodically retrieves user posts from the database and passes them to the generation AI, which analyzes the data and organizes and summarizes the relevant information. The organized information is then made available to users in a form accessible through a smartphone application.

[0781] Travel planning support using conversational AI

[0782] When a user enters a travel planning question into a smartphone application, the smartphone sends the inquiry data to a cloud server, which uses conversational AI to generate an optimal travel plan based on the user's question and returns suggestions, including specific travel destinations, places to visit, and recommended activities.

[0783] Providing AR / VR content

[0784] When a user requests a virtual experience of a specific location or event, their smartphone sends the request to a cloud server. The cloud server generates AR / VR content in real time based on the necessary data and provides it to the user through a smartphone application. This allows users with physical or financial constraints to virtually experience activities that interest them.

[0785] Examples of concrete examples and prompts

[0786] For example, if a user interested in model trains posts that they want to go to a new model train exhibition, their smartphone will send this information to a server and store it in a database. Next, if another user asks, "What railway museum should I visit this weekend?", the conversational AI will suggest the optimal travel plan. Furthermore, users who cannot visit a railway museum can use AR / VR technology to virtually experience the inside of the museum.

[0787] Example prompt sentence:

[0788] "Please tell me the latest model train exhibition."

[0789] "Can you recommend a railway museum I can visit next weekend?"

[0790] "Show me a virtual tour of the model railroad exhibition."

[0791] In this way, the system utilizes generative and conversational AI, as well as AR / VR content generation tools, to enable users to gather information on their niche interests, plan trips, and provide virtual experiences.

[0792] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0793] Step 1:

[0794] Users use a smartphone application to input and post information about their hobbies. The input data includes text, images, videos, etc. The input data from the user is sent to the device.

[0795] Step 2:

[0796] The device sends user post data to the cloud server. The data arrives in the form of text, images, or videos and is stored in a database. The cloud server receives user posts and centralizes them in a database.

[0797] Step 3:

[0798] The cloud server periodically retrieves user posts from the database. This retrieved data is passed to the generation AI. The cloud server analyzes the contents of the database and passes the relevant data to the generation AI.

[0799] Step 4:

[0800] Generative AI analyzes user posts and organizes and summarizes relevant information. Input data is analyzed by generative AI, and the desired information is organized and output. Generative AI uses a model (e.g., GPT-4) to analyze text data and generate a summary of the information.

[0801] Step 5:

[0802] The cloud server retrieves the summarized data from the generated AI and provides it to the user via a smartphone application. The summarized data is output in a format that can be displayed on the user's smartphone. The cloud server then sends the organized and summarized information to the user.

[0803] Step 6:

[0804] A user inputs a question about their travel plan into a smartphone application. The user's question data is entered into the terminal. This input is sent to the conversational AI.

[0805] Step 7:

[0806] The device sends question data to the cloud server. The question data arrives at the cloud server and is received by the conversational AI. The conversational AI analyzes the question using a model (e.g., Dialogflow).

[0807] Step 8:

[0808] The conversational AI analyzes the user's inquiry and generates the optimal travel plan. The analyzed question data is processed by the conversational AI to generate the travel plan. The generated plan is sent to the cloud server.

[0809] Step 9:

[0810] The cloud server obtains the travel plan from the conversational AI and provides it to the user via a smartphone application. The generated travel plan is displayed on the smartphone in a format that is easy for the user to understand. The cloud server then sends the travel plan to the user's device.

[0811] Step 10:

[0812] A user requests a virtual experience of a specific location or event. The user's request data is entered into a smartphone application. This input is sent to a cloud server.

[0813] Step 11:

[0814] The cloud server receives the request and generates the virtual experience content using an AR / VR content generation tool (e.g., Unity, Unreal Engine). The request data reaches the cloud server, and the AR / VR content is generated based on the required data.

[0815] Step 12:

[0816] The cloud server provides the generated AR / VR content to the user via a smartphone application. The generated AR / VR content is displayed on the smartphone or head-mounted display. The cloud server provides the virtual experience to the user, allowing the user to view it.

[0817] This allows users to virtually experience activities that interest them, regardless of physical or financial constraints.

[0818] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0819] This invention is a system that allows users with particularly niche hobbies to collect information and interact with the community. This system combines the collection of user posts, information organization using generative AI, travel planning support using conversational AI, provision of AR / VR content, and an emotion engine that recognizes and responds to user emotions. The details of this system are described below.

[0820] Collecting user posts

[0821] Users post information about their hobbies and experiences through applications and websites. The device receives this posted data and sends it to a server. The server stores the received data in a database. This data can be in the form of text, images, videos, etc., and is centrally managed in the database.

[0822] Information organization using generative AI

[0823] The server periodically retrieves user posts from the database and passes them to the generation AI, which analyzes this data, organizes and summarizes the relevant information, allowing other users to efficiently obtain the information they need. The summarized information is then provided to users in an accessible format.

[0824] Travel planning support using conversational AI

[0825] When a user enters a travel planning question into the application, the device sends the query data to the server, which uses conversational AI to generate and return a travel itinerary that best suits the user's query, including specific travel destinations, places to visit, and recommended activities.

[0826] Providing AR / VR content

[0827] For users who find it difficult to travel or attend events, the server uses AR / VR technology to provide rich virtual experiences. When a user requests a virtual experience of a specific location or event, the device sends the request to the server. The server generates AR / VR content in real time based on the necessary data and provides the virtual experience. This allows users with financial or physical limitations to enjoy activities that interest them.

[0828] Response by Emotion Engine

[0829] While the user is using the system, the emotion engine analyzes the user's emotions. For example, it determines the user's emotional state from the user's text input, voice, and facial expressions. The server uses this emotion data to adjust the information provided by the generative AI and the travel plans proposed by the conversational AI. For example, if the user is feeling stressed, it can suggest travel destinations and activities aimed at relaxation.

[0830] Specific examples

[0831] For example, if a user interested in model trains posts with a positive emotion, "I want to go to a new model train exhibition," this information is stored in a database and shared with other users by the generative AI. Next, if another user uses the conversational AI to ask, "What train museums should I visit this weekend?", the conversational AI will suggest the best travel plan to answer that question. Similarly, if a user requests a virtual experience with an emotion expressing fatigue, the AI ​​can suggest relaxing AR / VR content.

[0832] This will enable users with particularly niche hobbies to enrich their hobby activities by collecting and sharing information, planning trips, and engaging in virtual experiences.The introduction of an emotion engine is expected to provide personalized experiences based on the user's emotional state, improving engagement across the community.

[0833] The processing flow will be explained below.

[0834] Collecting and organizing user posts

[0835] Collecting user posts

[0836] Step 1:

[0837] User: Enters submissions into application or website forms.

[0838] Step 2:

[0839] Terminal: Sends the entered post content to the server.

[0840] Step 3:

[0841] Server: Stores the received submission data in a database. The submission data includes text, images, videos, etc.

[0842] Information organization

[0843] Step 4:

[0844] Server: Periodically retrieves user posts from the database.

[0845] Step 5:

[0846] Server: Passes the acquired data to the generation AI, which analyzes and summarizes the information.

[0847] Step 6:

[0848] Server: Provides summarized information in a form that users can access, allowing other users to efficiently retrieve the information they need.

[0849] Travel planning support using conversational AI

[0850] Step 1:

[0851] User: Enters a travel planning question into a text box in the application.

[0852] Step 2:

[0853] Terminal: Sends the entered question to the server.

[0854] Step 3:

[0855] Server: Analyzes user questions using conversational AI and generates travel plans.

[0856] Step 4:

[0857] Server: Returns the generated itinerary to the user.

[0858] Step 5:

[0859] On the device: Show the user a suggested travel itinerary, including specific destinations, places to visit, and recommended activities.

[0860] Providing AR / VR content

[0861] Step 1:

[0862] User: Requests a virtual experience of a specific location or event through a form in the app.

[0863] Step 2:

[0864] Terminal: Sends the input request to the server.

[0865] Step 3:

[0866] Server: Retrieves relevant data from a database based on the requested location or event.

[0867] Step 4:

[0868] Server: Generates the required AR / VR content in real time.

[0869] Step 5:

[0870] Server: Sends the generated AR / VR content to the user.

[0871] Step 6:

[0872] Device: The received AR / VR content is displayed on the user's device. Users can enjoy the virtual experience using a dedicated device or app.

[0873] Response by Emotion Engine

[0874] Step 1:

[0875] User: Providing emotion-based text input, voice, facial expressions, etc. while using the system.

[0876] Step 2:

[0877] Device: Sends emotion-related data to the server.

[0878] Step 3:

[0879] Server: The emotion engine analyzes the received data and determines the user's emotional state.

[0880] Step 4:

[0881] Server: Based on the emotion data, adjust the information provided by the generative AI and the travel plan proposed by the conversational AI. For example, if the user expresses stress, suggest places and activities that will help them relax.

[0882] Step 5:

[0883] Server: Provide users with tailored information and plans. By providing information that is tailored to their emotions, users can have a more satisfying experience.

[0884] Specific examples

[0885] For example, consider the case of a user who wants to go to a model railroad exhibition.

[0886] Step 1:

[0887] User: Posts "I want to go to the new model train show." Shows positive sentiment.

[0888] Step 2:

[0889] Terminal: Sends the post to the server.

[0890] Step 3:

[0891] Server: Save the submitted data to a database.

[0892] Step 4:

[0893] Server: Periodically retrieves posted data and summarizes it using a generation AI.

[0894] Step 5:

[0895] Server: If the emotion engine confirms a user's positive emotions, it provides information that promotes a positive experience for other users.

[0896] Step 6:

[0897] User: "Tell me which train museum I should visit this weekend," asks the conversational AI, showing some tiredness.

[0898] Step 7:

[0899] Terminal: Sends the question to the server.

[0900] Step 8:

[0901] Server: Conversational AI generates optimal travel plans and suggests relaxing places that match the user's emotional state.

[0902] Step 9:

[0903] Server: Sends the proposal back to the user.

[0904] Step 10:

[0905] User: Requests AR / VR virtual experience with emotion indicating fatigue.

[0906] Step 11:

[0907] Terminal: Sends the request to the server.

[0908] Step 12:

[0909] Server: Generates and provides relaxing AR / VR content in real time.

[0910] Step 13:

[0911] Device: Displays the received AR / VR content on the user's device.

[0912] This will enable users with particularly niche hobbies to enrich their hobby activities by collecting and sharing information, planning trips, and engaging in virtual experiences.The introduction of an emotion engine is expected to provide personalized experiences based on the user's emotional state, improving engagement across the community.

[0913] Example 2

[0914] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0915] Until now, there have been no systems that can collect and share information, plan trips, provide virtual experiences, or respond to users' emotions for users with niche hobbies. As a result, users have been unable to efficiently obtain information that interests them, and have difficulty receiving appropriate activity suggestions and virtual experiences, resulting in a decline in user experience and community engagement.

[0916] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0917] In this invention, the server includes means for receiving user posts from the terminal and storing them in a database, means for analyzing the data stored in the database using a generation AI and organizing and providing related information, means for analyzing user inquiries using a conversational AI and proposing travel plans, means including an emotion engine for analyzing user emotions and adjusting the output of the generation AI and the conversational AI, and means for dynamically generating and providing AR / VR content based on user requests. This allows users to efficiently collect information and receive appropriate travel plans and virtual experiences, and further enables the provision of personalized experiences according to the user's emotions.

[0918] "Terminal" refers to a device used by a user, including smartphones and personal computers.

[0919] "User Posts" refers to hobby-related information and experiences posted by Users through the Application or Website.

[0920] "Database" refers to a data storage system for centrally storing and managing collected data.

[0921] "Generative AI" is a system that uses artificial intelligence to generate and analyze text, images, etc., and is used for the purpose of organizing and summarizing information.

[0922] "Conversational AI" is artificial intelligence that generates answers in natural language to user inquiries, and is used for question answering and information provision.

[0923] An "emotion engine" is a system that analyzes a user's text input, voice, facial expressions, etc. to determine their emotional state and adjusts the output of other functions based on the results.

[0924] "AR / VR content" refers to rich virtual experiences generated using augmented reality (AR) and virtual reality (VR) technologies.

[0925] The present invention is a system for users with particularly niche interests to collect information and interact with communities. This system combines collection of user posts, information organization using generative AI, travel planning support using conversational AI, provision of AR / VR content, and an emotion engine that recognizes and responds to user emotions. Specific embodiments of the present invention are described below.

[0926] Collecting user posts

[0927] A user accesses an application or website and posts information about their hobbies and experiences. The device receives this posted data and sends it to a server. The server stores the received data in a database. This data can be in the form of text, images, videos, etc., and is centrally managed in a database (e.g., MySQL).

[0928] Information organization using generative AI

[0929] The server periodically retrieves user posts from the database and passes them to the Generative AI, which (e.g., OpenAI GPT-4) analyzes the data and organizes and summarizes the relevant information. The summarized information is then made available by the server in a form that can be accessed by other users.

[0930] Travel planning support using conversational AI

[0931] When a user enters a travel plan question into the application, the device sends the question data to a server, which uses conversational AI (e.g., Google Dialogflow) to generate a travel plan that best suits the user's question and returns suggestions, including specific travel destinations, places to visit, and recommended activities.

[0932] Providing AR / VR content

[0933] For users who find it difficult to travel or attend events, the server uses AR / VR technology to provide rich virtual experiences. When a user requests a virtual experience of a specific location or event, the device sends the request to the server. The server generates AR / VR content in real time based on the necessary data and provides it to the user. This allows users with financial or physical constraints to enjoy activities that interest them. The specific software used includes Unity.

[0934] Response by Emotion Engine

[0935] While the user is using the system, the emotion engine analyzes the user's emotional state from text input, voice, facial expressions, etc. The server uses this emotional data to adjust the information provided by the generative AI and the travel plans suggested by the conversational AI. For example, if the user is feeling stressed, it can suggest travel destinations and activities aimed at relaxation. The specific software used includes Microsoft Azure Cognitive Services.

[0936] Specific examples

[0937] For example, if a user interested in model trains posts a positive message saying, "I want to go to a new model train exhibition," the device sends the message to the server, which stores it in a database. This information is then organized by the generative AI and shared with other users.

[0938] Next, another user can use the conversational AI to ask, "What railway museums should I visit this weekend?" and the conversational AI will suggest the best travel plan to answer that question.

[0939] In addition, for users who express fatigue and request a "relaxing virtual experience," the system can suggest relaxing AR / VR content.

[0940] In this way, the system of the present invention enables efficient information gathering, personalized travel planning, and virtual experiences, especially for users with niche hobbies, thereby increasing user engagement and further enriching their hobby activities.

[0941] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0942] Step 1:

[0943] Users access applications or websites and post information about their hobbies and experiences. The data entered can be in the form of text, images, videos, etc. The device receives this posted data and sends it to the server.

[0944] Specific behavior:

[0945] A user enters the text "I want to go to the new model train exhibition" into the app, attaches an image and posts it.

[0946] The terminal sends this as text data and image data to the server. The input is text and image data, and the output is data sent to the server.

[0947] Step 2:

[0948] The server stores the received submission data in a database. The data can be in various formats, such as text, images, and videos, and is managed centrally.

[0949] Specific behavior:

[0950] The server executes a SQL query to insert the received data (text and images) into the database. The input is the received data, and the output is saving it to the database.

[0951] Step 3:

[0952] The server periodically retrieves user posts from the database and passes them to the generation AI, which analyzes the data and organizes and summarizes the relevant information.

[0953] Specific behavior:

[0954] The server retrieves new post data saved from the previous day from the database at a fixed time every day.

[0955] The acquired data is input into the generative AI (OpenAI GPT-4) to generate summarized information. The input is data acquired from the database, and the output is summarized information.

[0956] Step 4:

[0957] The server stores the generated summary information back in a database, making it accessible to other users.

[0958] Specific behavior:

[0959] The server inserts the summary data output by the generation AI into the database.

[0960] This information is displayed on a website or application for users to view. The input is the output data of the generative AI, and the output is stored in a database and provided to the user.

[0961] Step 5:

[0962] The user inputs a travel planning question within the application, and the device sends the question data to the server.

[0963] Specific behavior:

[0964] A user types, "What train museum should I visit this weekend?"

[0965] The terminal sends this text data to the server. The input is the user's question data, and the output is the data sent to the server.

[0966] Step 6:

[0967] The server uses conversational AI to generate the optimal travel plan for the user's question. The conversational AI generates an answer to the question.

[0968] Specific behavior:

[0969] The server inputs question data into a conversational AI (Google Dialogflow) and generates a travel plan.

[0970] The conversational AI generates a list of optimal travel destinations and places to visit and returns it to the server. The input is the question data, and the output is the generated travel plan.

[0971] Step 7:

[0972] The server returns the generated itinerary to the terminal and displays it to the user.

[0973] Specific behavior:

[0974] The server sends the output of the conversational AI to the terminal.

[0975] The device displays the optimal travel plan to the user. The input is the output data of the conversational AI, and the output is the display to the user.

[0976] Step 8:

[0977] The user requests a virtual experience of a specific location or event, and the device sends this request to the server.

[0978] Specific behavior:

[0979] A user requests a virtual experience of a model railroad exhibition.

[0980] The terminal sends this request data to the server. The input is the user's request data, and the output is the data sent to the server.

[0981] Step 9:

[0982] The server uses AR / VR technology to generate a rich virtual experience and deliver it to the user's device. The software used includes Unity.

[0983] Specific behavior:

[0984] The server uses Unity to generate the virtual experience.

[0985] The generated AR / VR content is sent to the device so that the user can experience it. The input is the user's request data, and the output is the generated AR / VR content.

[0986] Step 10:

[0987] While the user is using the system, the emotion engine analyzes the user's emotions, and the server uses this emotional data to adjust the output of the generative and conversational AI.

[0988] Specific behavior:

[0989] When a user types a question within the app, the text, voice data, and facial expressions are input into an emotion analysis engine.

[0990] The server analyzes the emotion data using an emotion analysis engine (Microsoft Azure Cognitive Services) and feeds it back to the generative AI and conversational AI to provide information and adjust travel plans. The input is the user's emotion data, and the output is adjusted information.

[0991] (Application example 2)

[0992] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0993] Currently, there are limited platforms that allow users with diverse hobbies to efficiently gather information that matches their interests and interact with communities that share the same hobbies. Furthermore, users often have unsatisfactory experiences due to the effort required to gather information when planning a trip and the lack of personalized product purchasing experiences. Furthermore, users are not provided with information or experiences that are in line with their emotions, which leads to low user engagement.

[0994] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving user posts from the terminal and storing them in a database, means for analyzing the data stored in the database using a generation AI and organizing and providing related information, means for analyzing user inquiries using an interactive AI and proposing travel plans, means for dynamically generating and providing AR / VR content based on user requests, means for providing information on products and services related to the user posts in a virtual shopping mall, and an emotion engine for analyzing user emotions and providing personalized content and product suggestions based on the analysis. This not only enables users to efficiently collect information on niche hobbies and interact with communities, but also enables activities such as travel plans, virtual experiences, and product purchases to be personalized, enabling the provision of services that are tailored to their emotions.

[0995] A "terminal" is an electronic device through which a user inputs information and communicates with the system.

[0996] "User posts" are content such as text, images, and videos of information and experiences posted by users via their devices.

[0997] A "database" is a system that centrally stores and manages data such as received user posts.

[0998] "Generative AI" is an artificial intelligence technology that analyzes collected data and organizes and summarizes related information.

[0999] "Conversational AI" is an artificial intelligence technology that analyzes user inquiries and provides optimal answers and plans.

[1000] A "travel plan" is a plan that includes locations, activities, accommodations, etc., when a user travels.

[1001] "AR / VR content" means visual and experiential content created using augmented reality (AR) and virtual reality (VR) technologies.

[1002] A "virtual shopping mall" is an e-commerce platform where users can explore and purchase goods and services in a virtual environment.

[1003] The "emotion engine" is a function that analyzes the user's emotional state and provides personalized services and content based on the results.

[1004] A "system" is a set of technical components in which multiple means work together to achieve a specific purpose.

[1005] The system for realizing this invention operates by combining multiple means. The system is mainly composed of terminals, servers, analysis of user posts, and the introduction of new technologies.

[1006] First, users use their devices to post information about their hobbies and experiences. This information is received in the form of text, images, and videos and is stored in a database via a server. The database serves to organize and centrally manage the information.

[1007] The server then periodically retrieves user posts from the database and analyzes them using a generative AI, which uses AI technologies such as TensorFlow.js to organize and summarize the collected data into relevant information, allowing other users to efficiently access the information they need.

[1008] Furthermore, when users ask questions about their travel plans, the server uses conversational AI to suggest optimal travel plans, such as using Dialogflow to generate answers to the user's questions and provide them to the user. This plan includes travel destinations, places to visit, and recommended activities.

[1009] AR / VR content can also be provided through the device. When a user requests a virtual experience of a specific location or event, the server generates the AR / VR content in real time using Unity or WebXR and provides it to the user. This allows users with financial or physical limitations to virtually experience activities that interest them.

[1010] Finally, an emotion engine is used to analyze the user's emotional state, for example, by determining emotions from the user's text input, voice, or facial expressions, and then adjust the information provided by the generative AI and the travel itinerary suggested by the conversational AI. The emotion engine then provides personalized product suggestions and content to the user.

[1011] Specific examples

[1012] For example, if a user interested in model trains posts with positive emotions, "I want to go to a new model train exhibition," this information is stored in a database and shared with other users by the generative AI. Next, if another user uses the conversational AI to ask, "What railway museum should I visit this weekend?", the conversational AI will suggest the best travel plan. Additionally, if a user feels tired, the AI ​​can suggest relaxing AR / VR content.

[1013] Prompt Sentence Examples

[1014] "Can you give me some information about the latest model railroad exhibition?"

[1015] "Show me your new model train station."

[1016] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1017] Step 1:

[1018] Users post information about their hobbies and experiences through their devices. These posts are entered in the form of text, images, videos, etc., and are sent from the device to the server. The server receives this data and stores it in a database. During this process, the server properly organizes the data format and metadata to centrally manage the posted data.

[1019] Step 2:

[1020] The server periodically retrieves user posts from the database. It then uses a generative AI to analyze the data. This analysis is used to organize and summarize relevant information. The generative AI uses TensorFlow.js to perform semantic analysis of text data and image recognition to effectively organize information. For example, it extracts keywords from posted text and uses them to summarize related information.

[1021] Step 3:

[1022] The server receives questions from users. For example, if a user types "Please tell me which railway museum I should visit this weekend" through a terminal, this question is sent to the server. The server analyzes this inquiry using a conversational AI (e.g., Dialogflow). Based on the results of the analysis, it generates an optimal travel plan and proposes it to the user. In this case, the server obtains answer data for the question from the generation AI and provides a personalized answer.

[1023] Step 4:

[1024] When a user requests a virtual experience of a specific location or event, the device sends this request to the server. The server uses Unity or WebXR to dynamically generate AR / VR content based on the request. The generated content is then delivered to the user in real time. For example, if a user requests, "Show me the new model train exhibition," the server generates a virtual exhibition and delivers it to the user.

[1025] Step 5:

[1026] The server analyzes the user's emotional state. It collects data such as text input, voice, and facial expressions while the user is using the device and analyzes it using an emotion engine. Based on this analysis, it personalizes the information and content provided to the user. For example, if the user is feeling stressed, the emotion engine will suggest content and products suitable for relaxation.

[1027] As described above, each step works in conjunction to create a system that provides users with an efficient and personalized experience.

[1028] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1029] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1030] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1031] [Third embodiment]

[1032] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1033] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1034] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1035] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1036] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1037] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1038] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1039] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1040] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1041] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1042] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1043] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1044] This invention is a system for users with particularly niche interests to collect information and interact with the community. This system has four main functions: collecting user posts, organizing information using generative AI, supporting travel planning using conversational AI, and providing AR / VR content. The details of this system are explained below.

[1045] Collecting user posts

[1046] Users post information about their hobbies and experiences through applications and websites. The device receives this posted data and sends it to a server. The server stores the received data in a database. This data can be in the form of text, images, videos, etc., and is centrally managed in the database.

[1047] Information organization using generative AI

[1048] The server periodically retrieves user posts from the database and passes them to the generation AI, which analyzes this data, organizes and summarizes the relevant information, allowing other users to efficiently obtain the information they need. The summarized information is then provided to users in an accessible format.

[1049] Travel planning support using conversational AI

[1050] When a user enters a travel planning question into the application, the device sends the query data to the server, which uses conversational AI to generate and return a travel itinerary that best suits the user's query, including specific travel destinations, places to visit, and recommended activities.

[1051] Providing AR / VR content

[1052] For users who find it difficult to travel or attend events, the server uses AR / VR technology to provide rich virtual experiences. When a user requests a virtual experience of a specific location or event, the device sends the request to the server. The server generates AR / VR content in real time based on the necessary data and provides the virtual experience. This allows users with financial or physical limitations to enjoy activities that interest them.

[1053] Specific examples

[1054] For example, if a user interested in model trains posts, "I want to go to a new model train exhibition," this information is stored in a database. Then, if another user uses the conversational AI to ask, "What train museum should I visit this weekend?", the conversational AI will suggest the best travel plan to answer that question. Furthermore, users who cannot physically visit museums can use AR / VR technology to virtually experience the inside of the museum.

[1055] This invention enables users with particularly niche hobbies to enrich their hobby activities by collecting and sharing information, planning trips, and engaging in virtual experiences, which is expected to promote interaction between users and improve engagement across the community.

[1056] The processing flow will be explained below.

[1057] Collecting and organizing user posts

[1058] Collecting user posts

[1059] Step 1:

[1060] User: Enters submissions into application or website forms.

[1061] Step 2:

[1062] Terminal: Sends the entered post content to the server.

[1063] Step 3:

[1064] Server: Stores the received submission data in a database.

[1065] Information organization

[1066] Step 4:

[1067] Server: Periodically retrieves user posts from the database.

[1068] Step 5:

[1069] Server: Passes the acquired data to the generation AI, which analyzes and summarizes the information.

[1070] Step 6:

[1071] Server: Provides summarized information in a form that users can access.

[1072] Travel planning support using conversational AI

[1073] Step 1:

[1074] User: Enters a travel planning question into a text box in the application.

[1075] Step 2:

[1076] Terminal: Sends the entered question to the server.

[1077] Step 3:

[1078] Server: Analyzes user questions using conversational AI and generates travel plans.

[1079] Step 4:

[1080] Server: Returns the generated itinerary to the user.

[1081] Step 5:

[1082] Terminal: Shows suggested travel itineraries to the user.

[1083] Providing AR / VR content

[1084] Step 1:

[1085] User: Requests a virtual experience of a specific location or event through a form in the app.

[1086] Step 2:

[1087] Terminal: Sends the input request to the server.

[1088] Step 3:

[1089] Server: Retrieves relevant data from a database based on the requested location or event.

[1090] Step 4:

[1091] Server: Generates the required AR / VR content in real time.

[1092] Step 5:

[1093] Server: Sends the generated AR / VR content to the user.

[1094] Step 6:

[1095] Device: Displays the received AR / VR content on the user's device.

[1096] Specific examples

[1097] Step 1:

[1098] User: Posts information about a model railroad show.

[1099] Step 2:

[1100] Terminal: Sends the post to the server.

[1101] Step 3:

[1102] Server: Stores the received posts in a database.

[1103] Step 4:

[1104] Server: Periodically retrieves posted data and summarizes it using a generation AI.

[1105] Step 5:

[1106] User: Finds museums to visit next weekend and asks the conversational AI a question.

[1107] Step 6:

[1108] Terminal: Sends the question to the server.

[1109] Step 7:

[1110] Server: Conversational AI generates optimal travel plans and suggests them to users.

[1111] Step 8:

[1112] User: If you can't make it to the museum, request a virtual experience.

[1113] Step 9:

[1114] Terminal: Sends the request to the server.

[1115] Step 10:

[1116] Server: Generates AR / VR content and provides it to users.

[1117] Step 11:

[1118] Device: Displays the received AR / VR content on the user's device.

[1119] This allows users to efficiently gather relevant information and enjoy travel and virtual experiences through a dedicated system, particularly for those with physical limitations, allowing them to enrich their hobbies.

[1120] Example 1

[1121] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1122] Conventional information gathering and community systems make it difficult for users with niche interests to efficiently collect and share information in an organized manner. They also lack the means for users to receive interactive assistance when planning trips, or for users with physical limitations to enjoy rich virtual experiences. As a result, it is difficult to provide comprehensive services to users with specific interests.

[1123] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1124] In this invention, the server includes means for receiving user posts from terminals and storing them in a database, means for analyzing the data stored in the database using a natural language processing model and organizing and providing related information, means for analyzing user inquiries using interactive artificial intelligence and proposing travel plans, means for dynamically generating and providing virtual reality technology content based on user requests, and means for retrieving user posts from the database and summarizing them using a natural language processing model. This allows users to efficiently collect information, receive travel plan suggestions, and enjoy rich experiences through virtual reality technology.

[1125] A "terminal" is a device that allows a user to input or receive information, and is often a smartphone, tablet, or computer.

[1126] "User Posts" refers to hobby-related information and experiences that users share through applications and websites, often in the form of text, images, videos, etc.

[1127] "Database" refers to a system for efficiently managing, storing, and retrieving data in a structured format.

[1128] A "natural language processing model" refers to an artificial intelligence technology that analyzes natural language data entered by a user and understands, generates, and summarizes human language.

[1129] "Conversational artificial intelligence" refers to a conversational artificial intelligence system that can provide natural responses to user questions and commands.

[1130] "Virtual reality technology" refers to technology that allows users to visually and aurally experience places and experiences that do not actually exist in the physical world.

[1131] "Means for organizing and providing information" refers to a method for using generative AI to analyze information stored in a database and provide related information to users in an easy-to-understand format.

[1132] "Means for proposing travel plans" refers to a method for using conversational AI to generate and propose optimal travel plans in response to user inquiries.

[1133] "Dynamic generation and serving means" refers to a method for creating and serving content in real time based on user requests.

[1134] "Means for summarizing" refers to a method for concisely summarizing multiple user-posted information using a natural language processing model.

[1135] This invention is a system for users with particularly niche interests to collect information and interact with the community. The system has four main functions: collecting user posts, organizing information using natural language processing models, assisting with travel planning using conversational artificial intelligence, and providing content using virtual reality technology.

[1136] Collecting user posts

[1137] Users post information about their hobbies and experiences through applications and websites. This data can be in the form of text, images, videos, etc.

[1138] The device receives the user's posted data and sends it to the server. For example, if a user posts "I want to go to the new model train exhibition," the device sends this to the server.

[1139] The server stores the received data in a database, which is a means of efficiently managing and centralizing this data.

[1140] Information organization using natural language processing models

[1141] The server periodically retrieves user posts from the database and passes them to a natural language processing model, such as OpenAI's GPT-3.

[1142] Natural language processing models analyze user posts and organize and summarize relevant information. The generated summaries are a means for other users to efficiently access information. For example, they can concisely summarize reviews of a model train exhibition posted by multiple users.

[1143] The server stores the summarized information in a database and makes it accessible to users.

[1144] Travel planning support using conversational AI

[1145] A user enters a travel planning question into an application, for example, "What train museums should I visit this weekend?"

[1146] The terminal transmits this inquiry data to the server.

[1147] The server uses conversational artificial intelligence (e.g., Dialogflow) to generate an optimal travel plan based on the user's questions, including destinations, places to visit, and recommended activities.

[1148] The server returns the generated itinerary to the user, who can view it within the app.

[1149] Virtual reality technology content provision

[1150] A user requests a virtual experience. For example, a request to "virtually experience the Railway Museum" is made within the app.

[1151] The terminal transmits this request data to the server.

[1152] The server collects relevant data and generates content in real time using virtual reality technology, which transcends physical limitations to provide users with a rich experience.

[1153] The server sends the generated content to the user's device, and the user experiences it using a dedicated app or VR headset.

[1154] Examples and prompts

[1155] For example, if a user who is interested in model trains posts that they want to go to a new model train exhibition, this information is stored in the database. Then, if another user uses the conversational AI to ask, "What train museum should I visit this weekend?", the conversational AI will suggest the best travel plan to answer that question. Additionally, users who cannot physically visit museums can use virtual reality technology to virtually experience the inside of the museum.

[1156] An example of a prompt to input to a generative AI model is as follows:

[1157] "I'm interested in model trains. Which railway museum should I visit this weekend? Also, what should I look out for when I visit?"

[1158] This invention is expected to enable users with particularly niche interests to efficiently gather information, receive travel plan suggestions, and enjoy rich virtual experiences, thereby increasing engagement across the community.

[1159] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1160] Processing Steps

[1161] Step 1: Collect user posts

[1162] 1. Users post information and experiences about their hobbies through the application or website.

[1163] Specific behavior:

[1164] A user posts, "I want to go to a model train exhibition."

[1165] Input and Output:

[1166] Input: Text data entered by the user (e.g., "I want to go to a model train exhibition")

[1167] Output: Text data is sent to the terminal.

[1168] 2. The terminal receives the posted data from the user and sends it to the server.

[1169] Specific behavior:

[1170] The app converts user input into JSON format and sends it to the server via an HTTP POST request.

[1171] Input and Output:

[1172] Input: JSON format text data (e.g., {"post": "I want to go to a model train exhibition"})

[1173] Output: The server receives the HTTP request.

[1174] 3. The server stores the received data in a database.

[1175] Specific behavior:

[1176] The server parses the data from the received POST request and stores it as a new entry in the database.

[1177] Input and Output:

[1178] Input: HTTP request content data

[1179] Output: A new entry is created in the database.

[1180] Step 2: Organizing information using natural language processing models

[1181] 1. The server periodically retrieves user posts from the database.

[1182] Specific behavior:

[1183] A scheduler on the server runs every day at 2 AM and retrieves all recently added user posts.

[1184] Input and Output:

[1185] Input: User posted data from database

[1186] Output: A list of retrieved user post data

[1187] 2. The server passes the acquired data to the natural language processing model.

[1188] Specific behavior:

[1189] The server formats user posts into API requests and sends them to a natural language processing model.

[1190] Input and Output:

[1191] Input: User post data in API request format

[1192] Output: Data sent to the natural language processing model

[1193] 3. A natural language processing model analyzes the data and generates summary information.

[1194] Specific behavior:

[1195] A natural language processing model analyzes multiple posts and summarizes relevant information.

[1196] Input and Output:

[1197] Input: User Post Data

[1198] Output: Summary information (e.g., "Summary of reviews of recent model railroad shows")

[1199] 4. The server stores the generated summary information in a database and provides it in a user-accessible format.

[1200] Specific behavior:

[1201] The server stores the summary information in a database and displays it in specific sections of the website or app.

[1202] Input and Output:

[1203] Input: Summary information

[1204] Output: Summary information stored in a database, summary information displayed on a website or app

[1205] Step 3: Travel planning assistance with conversational AI

[1206] 1. The user enters a travel planning question into the application.

[1207] Specific behavior:

[1208] A user types, "What train museum should I visit this weekend?"

[1209] Input and Output:

[1210] Input: User question text

[1211] Output: Text data is sent to the terminal.

[1212] 2. The device sends the query data to the server.

[1213] Specific behavior:

[1214] The app converts the question into JSON format and sends it to the server via an HTTP POST request.

[1215] Input and Output:

[1216] Input: JSON-formatted question data (e.g., {"query": "What railway museum should I visit this weekend?"})

[1217] Output: The server receives the HTTP request.

[1218] 3. The server uses conversational artificial intelligence to generate the best travel plan for the query.

[1219] Specific behavior:

[1220] The server analyzes the question data and sends it to a conversational artificial intelligence, which then generates travel destination and activity suggestions.

[1221] Input and Output:

[1222] Input: User question data

[1223] Output: Generated itinerary

[1224] 4. The server returns the generated itinerary to the user.

[1225] Specific behavior:

[1226] The server receives the conversational AI's response and sends it to the user's device.

[1227] Input and Output:

[1228] Input: Generated itinerary

[1229] Output: The itinerary sent to the user's device

[1230] Step 4: Providing content for virtual reality technology

[1231] 1. A user requests a virtual experience.

[1232] Specific behavior:

[1233] The user clicks the "Experience the Railway Museum virtually" button within the app.

[1234] Input and Output:

[1235] Input: Virtual experience request data

[1236] Output: Text data is sent to the terminal.

[1237] 2. The device sends the request data to the server.

[1238] Specific behavior:

[1239] The app converts the request content into JSON format and sends it to the server via an HTTP POST request.

[1240] Input and Output:

[1241] Input: Request data in JSON format (e.g., {"request": "Virtual experience at the Railway Museum"})

[1242] Output: The server receives the HTTP request.

[1243] 3. The server generates virtual reality technology content in real time based on the required data.

[1244] Specific behavior:

[1245] The server collects the relevant data and processes it to generate the virtual reality technology content.

[1246] Input and Output:

[1247] Input: Request data and related data

[1248] Output: Generated virtual reality technology content

[1249] 4. The server provides the generated virtual reality technology content to the user's terminal.

[1250] Specific behavior:

[1251] The server sends the generated content URL to the user's device, where the user experiences it using a dedicated app or VR headset.

[1252] Input and Output:

[1253] Input: Generated virtual reality technology content

[1254] Output: The content URL served to the user's device

[1255] (Application example 1)

[1256] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1257] Existing social media and information gathering methods lack the functionality to allow users to efficiently gather and share useful information related to niche hobbies. Furthermore, when physical or financial constraints make it difficult for users to visit the destination in person, their information gathering and experiences are limited. To solve these issues, efficient information organization, travel planning support, and the provision of virtual experiences using AR / VR technology are required.

[1258] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1259] In this invention, the server includes: means for receiving user posts from the device and storing them in a database; means for analyzing the data stored in the database using a generation AI and organizing and providing related information; means for the cloud server to periodically pass user data to the generation AI, which organizes and summarizes the related information and provides it to the user via a smartphone application; means for analyzing user inquiries using an interactive AI and proposing travel plans; means for dynamically generating and providing AR / VR content based on user requests; means for visualizing information provided to the device via a smartphone; means for sending input data to the cloud server and storing it in a database in text, image, or video format; and means for providing prompts to the generation AI based on user posts and requests to help organize information and generate travel plans. This allows users with niche hobbies to efficiently collect and share information and enjoy their hobbies through virtual experiences beyond physical and financial constraints.

[1260] A "terminal" is a device used to input user posts and send them to a server, such as a smartphone or tablet.

[1261] "User Posts" are data posted by users that includes information, experiences, opinions, images, and videos related to their specific hobbies or interests.

[1262] A "database" is a system that centrally stores and manages user posts and is operated by a server.

[1263] "Generative AI" is an artificial intelligence model used to analyze user posts and collect, organize, and summarize relevant information.

[1264] "Conversational AI" is an artificial intelligence model that analyzes user inquiries and generates optimal answers and suggestions.

[1265] A "travel plan" is a plan including destinations, transportation, accommodation, activities, etc., when a user travels.

[1266] "AR / VR content" refers to content that provides users with a virtual experience using augmented reality and virtual reality technologies.

[1267] A "cloud server" is a remote server used to store, process, and manage data over the Internet.

[1268] A "smartphone application" is software that runs primarily on smartphones and is used to collect user posts, organize information, provide travel plans, display AR / VR content, and more.

[1269] "Related information" is information extracted, organized, and summarized by the AI ​​based on information obtained from user posts, and includes content that may be useful to other users.

[1270] A "prompt sentence" is an input sentence that instructs the generation AI to perform a specific analysis or generate information.

[1271] The system for implementing this invention includes multiple means for efficiently analyzing data collected from users and organizing and suggesting information. As an application example, we will explain a specific system that allows users with niche hobbies to collect information related to their hobbies and interact with other users.

[1272] Hardware and software used

[1273] The hardware and software used to implement this invention are as follows:

[1274] Smartphone: A device used to input and visualize user posts and communicate with the cloud server.

[1275] Cloud server: A server for storing user posts and analyzing data using generation AI and conversational AI.

[1276] Database: Located on a cloud server, it centrally manages user-posted data.

[1277] Generative AI model: An artificial intelligence model (e.g., GPT-4) that analyzes user posts and organizes and summarizes relevant information.

[1278] Conversational AI model: An artificial intelligence model (e.g., Dialogflow) that analyzes user inquiries and generates and suggests optimal travel plans.

[1279] AR / VR Content Generation Tools: Tools for generating augmented reality and virtual reality content, such as Unity and Unreal Engine.

[1280] Program processing

[1281] Collecting user posts

[1282] Users use a smartphone application to input and post information about their hobbies. The smartphone sends this posted data to a cloud server and stores it in a database. The posted data can be in the form of text, images, videos, etc.

[1283] Information organization using generative AI

[1284] The cloud server periodically retrieves user posts from the database and passes them to the generation AI, which analyzes the data and organizes and summarizes the relevant information. The organized information is then made available to users in a form accessible through a smartphone application.

[1285] Travel planning support using conversational AI

[1286] When a user enters a travel planning question into a smartphone application, the smartphone sends the inquiry data to a cloud server, which uses conversational AI to generate an optimal travel plan based on the user's question and returns suggestions, including specific travel destinations, places to visit, and recommended activities.

[1287] Providing AR / VR content

[1288] When a user requests a virtual experience of a specific location or event, their smartphone sends the request to a cloud server. The cloud server generates AR / VR content in real time based on the necessary data and provides it to the user through a smartphone application. This allows users with physical or financial constraints to virtually experience activities that interest them.

[1289] Examples of concrete examples and prompts

[1290] For example, if a user interested in model trains posts that they want to go to a new model train exhibition, their smartphone will send this information to a server and store it in a database. Next, if another user asks, "What railway museum should I visit this weekend?", the conversational AI will suggest the optimal travel plan. Furthermore, users who cannot visit a railway museum can use AR / VR technology to virtually experience the inside of the museum.

[1291] Example prompt sentence:

[1292] "Please tell me the latest model train exhibition."

[1293] "Can you recommend a railway museum I can visit next weekend?"

[1294] "Show me a virtual tour of the model railroad exhibition."

[1295] In this way, the system utilizes generative and conversational AI, as well as AR / VR content generation tools, to enable users to gather information on their niche interests, plan trips, and provide virtual experiences.

[1296] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1297] Step 1:

[1298] Users use a smartphone application to input and post information about their hobbies. The input data includes text, images, videos, etc. The input data from the user is sent to the device.

[1299] Step 2:

[1300] The device sends user post data to the cloud server. The data arrives in the form of text, images, or videos and is stored in a database. The cloud server receives user posts and centralizes them in a database.

[1301] Step 3:

[1302] The cloud server periodically retrieves user posts from the database. This retrieved data is passed to the generation AI. The cloud server analyzes the contents of the database and passes the relevant data to the generation AI.

[1303] Step 4:

[1304] Generative AI analyzes user posts and organizes and summarizes relevant information. Input data is analyzed by generative AI, and the desired information is organized and output. Generative AI uses a model (e.g., GPT-4) to analyze text data and generate a summary of the information.

[1305] Step 5:

[1306] The cloud server retrieves the summarized data from the generated AI and provides it to the user via a smartphone application. The summarized data is output in a format that can be displayed on the user's smartphone. The cloud server then sends the organized and summarized information to the user.

[1307] Step 6:

[1308] A user inputs a question about their travel plan into a smartphone application. The user's question data is entered into the terminal. This input is sent to the conversational AI.

[1309] Step 7:

[1310] The device sends question data to the cloud server. The question data arrives at the cloud server and is received by the conversational AI. The conversational AI analyzes the question using a model (e.g., Dialogflow).

[1311] Step 8:

[1312] The conversational AI analyzes the user's inquiry and generates the optimal travel plan. The analyzed question data is processed by the conversational AI to generate the travel plan. The generated plan is sent to the cloud server.

[1313] Step 9:

[1314] The cloud server obtains the travel plan from the conversational AI and provides it to the user via a smartphone application. The generated travel plan is displayed on the smartphone in a format that is easy for the user to understand. The cloud server then sends the travel plan to the user's device.

[1315] Step 10:

[1316] A user requests a virtual experience of a specific location or event. The user's request data is entered into a smartphone application. This input is sent to a cloud server.

[1317] Step 11:

[1318] The cloud server receives the request and generates the virtual experience content using an AR / VR content generation tool (e.g., Unity, Unreal Engine). The request data reaches the cloud server, and the AR / VR content is generated based on the required data.

[1319] Step 12:

[1320] The cloud server provides the generated AR / VR content to the user via a smartphone application. The generated AR / VR content is displayed on the smartphone or head-mounted display. The cloud server provides the virtual experience to the user, allowing the user to view it.

[1321] This allows users to virtually experience activities that interest them, regardless of physical or financial constraints.

[1322] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1323] This invention is a system that allows users with particularly niche hobbies to collect information and interact with the community. This system combines the collection of user posts, information organization using generative AI, travel planning support using conversational AI, provision of AR / VR content, and an emotion engine that recognizes and responds to user emotions. The details of this system are described below.

[1324] Collecting user posts

[1325] Users post information about their hobbies and experiences through applications and websites. The device receives this posted data and sends it to a server. The server stores the received data in a database. This data can be in the form of text, images, videos, etc., and is centrally managed in the database.

[1326] Information organization using generative AI

[1327] The server periodically retrieves user posts from the database and passes them to the generation AI, which analyzes this data, organizes and summarizes the relevant information, allowing other users to efficiently obtain the information they need. The summarized information is then provided to users in an accessible format.

[1328] Travel planning support using conversational AI

[1329] When a user enters a travel planning question into the application, the device sends the query data to the server, which uses conversational AI to generate and return a travel itinerary that best suits the user's query, including specific travel destinations, places to visit, and recommended activities.

[1330] Providing AR / VR content

[1331] For users who find it difficult to travel or attend events, the server uses AR / VR technology to provide rich virtual experiences. When a user requests a virtual experience of a specific location or event, the device sends the request to the server. The server generates AR / VR content in real time based on the necessary data and provides the virtual experience. This allows users with financial or physical limitations to enjoy activities that interest them.

[1332] Response by Emotion Engine

[1333] While the user is using the system, the emotion engine analyzes the user's emotions. For example, it determines the user's emotional state from the user's text input, voice, and facial expressions. The server uses this emotion data to adjust the information provided by the generative AI and the travel plans proposed by the conversational AI. For example, if the user is feeling stressed, it can suggest travel destinations and activities aimed at relaxation.

[1334] Specific examples

[1335] For example, if a user interested in model trains posts with a positive emotion, "I want to go to a new model train exhibition," this information is stored in a database and shared with other users by the generative AI. Next, if another user uses the conversational AI to ask, "What train museums should I visit this weekend?", the conversational AI will suggest the best travel plan to answer that question. Similarly, if a user requests a virtual experience with an emotion expressing fatigue, the AI ​​can suggest relaxing AR / VR content.

[1336] This will enable users with particularly niche hobbies to enrich their hobby activities by collecting and sharing information, planning trips, and engaging in virtual experiences.The introduction of an emotion engine is expected to provide personalized experiences based on the user's emotional state, improving engagement across the community.

[1337] The processing flow will be explained below.

[1338] Collecting and organizing user posts

[1339] Collecting user posts

[1340] Step 1:

[1341] User: Enters submissions into application or website forms.

[1342] Step 2:

[1343] Terminal: Sends the entered post content to the server.

[1344] Step 3:

[1345] Server: Stores the received submission data in a database. The submission data includes text, images, videos, etc.

[1346] Information organization

[1347] Step 4:

[1348] Server: Periodically retrieves user posts from the database.

[1349] Step 5:

[1350] Server: Passes the acquired data to the generation AI, which analyzes and summarizes the information.

[1351] Step 6:

[1352] Server: Provides summarized information in a form that users can access, allowing other users to efficiently retrieve the information they need.

[1353] Travel planning support using conversational AI

[1354] Step 1:

[1355] User: Enters a travel planning question into a text box in the application.

[1356] Step 2:

[1357] Terminal: Sends the entered question to the server.

[1358] Step 3:

[1359] Server: Analyzes user questions using conversational AI and generates travel plans.

[1360] Step 4:

[1361] Server: Returns the generated itinerary to the user.

[1362] Step 5:

[1363] On the device: Show the user a suggested travel itinerary, including specific destinations, places to visit, and recommended activities.

[1364] Providing AR / VR content

[1365] Step 1:

[1366] User: Requests a virtual experience of a specific location or event through a form in the app.

[1367] Step 2:

[1368] Terminal: Sends the input request to the server.

[1369] Step 3:

[1370] Server: Retrieves relevant data from a database based on the requested location or event.

[1371] Step 4:

[1372] Server: Generates the required AR / VR content in real time.

[1373] Step 5:

[1374] Server: Sends the generated AR / VR content to the user.

[1375] Step 6:

[1376] Device: The received AR / VR content is displayed on the user's device. Users can enjoy the virtual experience using a dedicated device or app.

[1377] Response by Emotion Engine

[1378] Step 1:

[1379] User: Providing emotion-based text input, voice, facial expressions, etc. while using the system.

[1380] Step 2:

[1381] Device: Sends emotion-related data to the server.

[1382] Step 3:

[1383] Server: The emotion engine analyzes the received data and determines the user's emotional state.

[1384] Step 4:

[1385] Server: Based on the emotion data, adjust the information provided by the generative AI and the travel plan proposed by the conversational AI. For example, if the user expresses stress, suggest places and activities that will help them relax.

[1386] Step 5:

[1387] Server: Provide users with tailored information and plans. By providing information that is tailored to their emotions, users can have a more satisfying experience.

[1388] Specific examples

[1389] For example, consider the case of a user who wants to go to a model railroad exhibition.

[1390] Step 1:

[1391] User: Posts "I want to go to the new model train show." Shows positive sentiment.

[1392] Step 2:

[1393] Terminal: Sends the post to the server.

[1394] Step 3:

[1395] Server: Save the submitted data to a database.

[1396] Step 4:

[1397] Server: Periodically retrieves posted data and summarizes it using a generation AI.

[1398] Step 5:

[1399] Server: If the emotion engine confirms a user's positive emotions, it provides information that promotes a positive experience for other users.

[1400] Step 6:

[1401] User: "Tell me which train museum I should visit this weekend," asks the conversational AI, showing some tiredness.

[1402] Step 7:

[1403] Terminal: Sends the question to the server.

[1404] Step 8:

[1405] Server: Conversational AI generates optimal travel plans and suggests relaxing places that match the user's emotional state.

[1406] Step 9:

[1407] Server: Sends the proposal back to the user.

[1408] Step 10:

[1409] User: Requests AR / VR virtual experience with emotion indicating fatigue.

[1410] Step 11:

[1411] Terminal: Sends the request to the server.

[1412] Step 12:

[1413] Server: Generates and provides relaxing AR / VR content in real time.

[1414] Step 13:

[1415] Device: Displays the received AR / VR content on the user's device.

[1416] This will enable users with particularly niche hobbies to enrich their hobby activities by collecting and sharing information, planning trips, and engaging in virtual experiences.The introduction of an emotion engine is expected to provide personalized experiences based on the user's emotional state, improving engagement across the community.

[1417] Example 2

[1418] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1419] Until now, there have been no systems that can collect and share information, plan trips, provide virtual experiences, or respond to users' emotions for users with niche hobbies. As a result, users have been unable to efficiently obtain information that interests them, and have difficulty receiving appropriate activity suggestions and virtual experiences, resulting in a decline in user experience and community engagement.

[1420] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1421] In this invention, the server includes means for receiving user posts from the terminal and storing them in a database, means for analyzing the data stored in the database using a generation AI and organizing and providing related information, means for analyzing user inquiries using a conversational AI and proposing travel plans, means including an emotion engine for analyzing user emotions and adjusting the output of the generation AI and the conversational AI, and means for dynamically generating and providing AR / VR content based on user requests. This allows users to efficiently collect information and receive appropriate travel plans and virtual experiences, and further enables the provision of personalized experiences according to the user's emotions.

[1422] "Terminal" refers to a device used by a user, including smartphones and personal computers.

[1423] "User Posts" refers to hobby-related information and experiences posted by Users through the Application or Website.

[1424] "Database" refers to a data storage system for centrally storing and managing collected data.

[1425] "Generative AI" is a system that uses artificial intelligence to generate and analyze text, images, etc., and is used for the purpose of organizing and summarizing information.

[1426] "Conversational AI" is artificial intelligence that generates answers in natural language to user inquiries, and is used for question answering and information provision.

[1427] An "emotion engine" is a system that analyzes a user's text input, voice, facial expressions, etc. to determine their emotional state and adjusts the output of other functions based on the results.

[1428] "AR / VR content" refers to rich virtual experiences generated using augmented reality (AR) and virtual reality (VR) technologies.

[1429] The present invention is a system for users with particularly niche interests to collect information and interact with communities. This system combines collection of user posts, information organization using generative AI, travel planning support using conversational AI, provision of AR / VR content, and an emotion engine that recognizes and responds to user emotions. Specific embodiments of the present invention are described below.

[1430] Collecting user posts

[1431] A user accesses an application or website and posts information about their hobbies and experiences. The device receives this posted data and sends it to a server. The server stores the received data in a database. This data can be in the form of text, images, videos, etc., and is centrally managed in a database (e.g., MySQL).

[1432] Information organization using generative AI

[1433] The server periodically retrieves user posts from the database and passes them to the Generative AI, which (e.g., OpenAI GPT-4) analyzes the data and organizes and summarizes the relevant information. The summarized information is then made available by the server in a form that can be accessed by other users.

[1434] Travel planning support using conversational AI

[1435] When a user enters a travel plan question into the application, the device sends the question data to a server, which uses conversational AI (e.g., Google Dialogflow) to generate a travel plan that best suits the user's question and returns suggestions, including specific travel destinations, places to visit, and recommended activities.

[1436] Providing AR / VR content

[1437] For users who find it difficult to travel or attend events, the server uses AR / VR technology to provide rich virtual experiences. When a user requests a virtual experience of a specific location or event, the device sends the request to the server. The server generates AR / VR content in real time based on the necessary data and provides it to the user. This allows users with financial or physical constraints to enjoy activities that interest them. The specific software used includes Unity.

[1438] Response by Emotion Engine

[1439] While the user is using the system, the emotion engine analyzes the user's emotional state from text input, voice, facial expressions, etc. The server uses this emotional data to adjust the information provided by the generative AI and the travel plans suggested by the conversational AI. For example, if the user is feeling stressed, it can suggest travel destinations and activities aimed at relaxation. The specific software used includes Microsoft Azure Cognitive Services.

[1440] Specific examples

[1441] For example, if a user interested in model trains posts a positive message saying, "I want to go to a new model train exhibition," the device sends the message to the server, which stores it in a database. This information is then organized by the generative AI and shared with other users.

[1442] Next, another user can use the conversational AI to ask, "What railway museums should I visit this weekend?" and the conversational AI will suggest the best travel plan to answer that question.

[1443] In addition, for users who express fatigue and request a "relaxing virtual experience," the system can suggest relaxing AR / VR content.

[1444] In this way, the system of the present invention enables efficient information gathering, personalized travel planning, and virtual experiences, especially for users with niche hobbies, thereby increasing user engagement and further enriching their hobby activities.

[1445] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1446] Step 1:

[1447] Users access applications or websites and post information about their hobbies and experiences. The data entered can be in the form of text, images, videos, etc. The device receives this posted data and sends it to the server.

[1448] Specific behavior:

[1449] A user enters the text "I want to go to the new model train exhibition" into the app, attaches an image and posts it.

[1450] The terminal sends this as text data and image data to the server. The input is text and image data, and the output is data sent to the server.

[1451] Step 2:

[1452] The server stores the received submission data in a database. The data can be in various formats, such as text, images, and videos, and is managed centrally.

[1453] Specific behavior:

[1454] The server executes a SQL query to insert the received data (text and images) into the database. The input is the received data, and the output is saving it to the database.

[1455] Step 3:

[1456] The server periodically retrieves user posts from the database and passes them to the generation AI, which analyzes the data and organizes and summarizes the relevant information.

[1457] Specific behavior:

[1458] The server retrieves new post data saved from the previous day from the database at a fixed time every day.

[1459] The acquired data is input into the generative AI (OpenAI GPT-4) to generate summarized information. The input is data acquired from the database, and the output is summarized information.

[1460] Step 4:

[1461] The server stores the generated summary information back in a database, making it accessible to other users.

[1462] Specific behavior:

[1463] The server inserts the summary data output by the generation AI into the database.

[1464] This information is displayed on a website or application for users to view. The input is the output data of the generative AI, and the output is stored in a database and provided to the user.

[1465] Step 5:

[1466] The user inputs a travel planning question within the application, and the device sends the question data to the server.

[1467] Specific behavior:

[1468] A user types, "What train museum should I visit this weekend?"

[1469] The terminal sends this text data to the server. The input is the user's question data, and the output is the data sent to the server.

[1470] Step 6:

[1471] The server uses conversational AI to generate the optimal travel plan for the user's question. The conversational AI generates an answer to the question.

[1472] Specific behavior:

[1473] The server inputs question data into a conversational AI (Google Dialogflow) and generates a travel plan.

[1474] The conversational AI generates a list of optimal travel destinations and places to visit and returns it to the server. The input is the question data, and the output is the generated travel plan.

[1475] Step 7:

[1476] The server returns the generated itinerary to the terminal and displays it to the user.

[1477] Specific behavior:

[1478] The server sends the output of the conversational AI to the terminal.

[1479] The device displays the optimal travel plan to the user. The input is the output data of the conversational AI, and the output is the display to the user.

[1480] Step 8:

[1481] The user requests a virtual experience of a specific location or event, and the device sends this request to the server.

[1482] Specific behavior:

[1483] A user requests a virtual experience of a model railroad exhibition.

[1484] The terminal sends this request data to the server. The input is the user's request data, and the output is the data sent to the server.

[1485] Step 9:

[1486] The server uses AR / VR technology to generate a rich virtual experience and deliver it to the user's device. The software used includes Unity.

[1487] Specific behavior:

[1488] The server uses Unity to generate the virtual experience.

[1489] The generated AR / VR content is sent to the device so that the user can experience it. The input is the user's request data, and the output is the generated AR / VR content.

[1490] Step 10:

[1491] While the user is using the system, the emotion engine analyzes the user's emotions, and the server uses this emotional data to adjust the output of the generative and conversational AI.

[1492] Specific behavior:

[1493] When a user types a question within the app, the text, voice data, and facial expressions are input into an emotion analysis engine.

[1494] The server analyzes the emotion data using an emotion analysis engine (Microsoft Azure Cognitive Services) and feeds it back to the generative AI and conversational AI to provide information and adjust travel plans. The input is the user's emotion data, and the output is adjusted information.

[1495] (Application example 2)

[1496] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1497] Currently, there are limited platforms that allow users with diverse hobbies to efficiently gather information that matches their interests and interact with communities that share the same hobbies. Furthermore, users often have unsatisfactory experiences due to the effort required to gather information when planning a trip and the lack of personalized product purchasing experiences. Furthermore, users are not provided with information or experiences that are in line with their emotions, which leads to low user engagement.

[1498] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving user posts from the terminal and storing them in a database, means for analyzing the data stored in the database using a generation AI and organizing and providing related information, means for analyzing user inquiries using an interactive AI and proposing travel plans, means for dynamically generating and providing AR / VR content based on user requests, means for providing information on products and services related to the user posts in a virtual shopping mall, and an emotion engine for analyzing user emotions and providing personalized content and product suggestions based on the analysis. This not only enables users to efficiently collect information on niche hobbies and interact with communities, but also enables activities such as travel plans, virtual experiences, and product purchases to be personalized, enabling the provision of services that are tailored to their emotions.

[1499] A "terminal" is an electronic device through which a user inputs information and communicates with the system.

[1500] "User posts" are content such as text, images, and videos of information and experiences posted by users via their devices.

[1501] A "database" is a system that centrally stores and manages data such as received user posts.

[1502] "Generative AI" is an artificial intelligence technology that analyzes collected data and organizes and summarizes related information.

[1503] "Conversational AI" is an artificial intelligence technology that analyzes user inquiries and provides optimal answers and plans.

[1504] A "travel plan" is a plan that includes locations, activities, accommodations, etc., when a user travels.

[1505] "AR / VR content" means visual and experiential content created using augmented reality (AR) and virtual reality (VR) technologies.

[1506] A "virtual shopping mall" is an e-commerce platform where users can explore and purchase goods and services in a virtual environment.

[1507] The "emotion engine" is a function that analyzes the user's emotional state and provides personalized services and content based on the results.

[1508] A "system" is a set of technical components in which multiple means work together to achieve a specific purpose.

[1509] The system for realizing this invention operates by combining multiple means. The system is mainly composed of terminals, servers, analysis of user posts, and the introduction of new technologies.

[1510] First, users use their devices to post information about their hobbies and experiences. This information is received in the form of text, images, and videos and is stored in a database via a server. The database serves to organize and centrally manage the information.

[1511] The server then periodically retrieves user posts from the database and analyzes them using a generative AI, which uses AI technologies such as TensorFlow.js to organize and summarize the collected data into relevant information, allowing other users to efficiently access the information they need.

[1512] Furthermore, when users ask questions about their travel plans, the server uses conversational AI to suggest optimal travel plans, such as using Dialogflow to generate answers to the user's questions and provide them to the user. This plan includes travel destinations, places to visit, and recommended activities.

[1513] AR / VR content can also be provided through the device. When a user requests a virtual experience of a specific location or event, the server generates the AR / VR content in real time using Unity or WebXR and provides it to the user. This allows users with financial or physical limitations to virtually experience activities that interest them.

[1514] Finally, an emotion engine is used to analyze the user's emotional state, for example, by determining emotions from the user's text input, voice, or facial expressions, and then adjust the information provided by the generative AI and the travel itinerary suggested by the conversational AI. The emotion engine then provides personalized product suggestions and content to the user.

[1515] Specific examples

[1516] For example, if a user interested in model trains posts with positive emotions, "I want to go to a new model train exhibition," this information is stored in a database and shared with other users by the generative AI. Next, if another user uses the conversational AI to ask, "What railway museum should I visit this weekend?", the conversational AI will suggest the best travel plan. Additionally, if a user feels tired, the AI ​​can suggest relaxing AR / VR content.

[1517] Prompt Sentence Examples

[1518] "Can you give me some information about the latest model railroad exhibition?"

[1519] "Show me your new model train station."

[1520] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1521] Step 1:

[1522] Users post information about their hobbies and experiences through their devices. These posts are entered in the form of text, images, videos, etc., and are sent from the device to the server. The server receives this data and stores it in a database. During this process, the server properly organizes the data format and metadata to centrally manage the posted data.

[1523] Step 2:

[1524] The server periodically retrieves user posts from the database. It then uses a generative AI to analyze the data. This analysis is used to organize and summarize relevant information. The generative AI uses TensorFlow.js to perform semantic analysis of text data and image recognition to effectively organize information. For example, it extracts keywords from posted text and uses them to summarize related information.

[1525] Step 3:

[1526] The server receives questions from users. For example, if a user types "Please tell me which railway museum I should visit this weekend" through a terminal, this question is sent to the server. The server analyzes this inquiry using a conversational AI (e.g., Dialogflow). Based on the results of the analysis, it generates an optimal travel plan and proposes it to the user. In this case, the server obtains answer data for the question from the generation AI and provides a personalized answer.

[1527] Step 4:

[1528] When a user requests a virtual experience of a specific location or event, the device sends this request to the server. The server uses Unity or WebXR to dynamically generate AR / VR content based on the request. The generated content is then delivered to the user in real time. For example, if a user requests, "Show me the new model train exhibition," the server generates a virtual exhibition and delivers it to the user.

[1529] Step 5:

[1530] The server analyzes the user's emotional state. It collects data such as text input, voice, and facial expressions while the user is using the device and analyzes it using an emotion engine. Based on this analysis, it personalizes the information and content provided to the user. For example, if the user is feeling stressed, the emotion engine will suggest content and products suitable for relaxation.

[1531] As described above, each step works in conjunction to create a system that provides users with an efficient and personalized experience.

[1532] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1533] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1534] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1535] [Fourth embodiment]

[1536] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1537] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1538] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1539] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1540] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1541] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1542] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1543] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1544] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1545] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1546] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1547] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1548] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1549] This invention is a system for users with particularly niche interests to collect information and interact with the community. This system has four main functions: collecting user posts, organizing information using generative AI, supporting travel planning using conversational AI, and providing AR / VR content. The details of this system are explained below.

[1550] Collecting user posts

[1551] Users post information about their hobbies and experiences through applications and websites. The device receives this posted data and sends it to a server. The server stores the received data in a database. This data can be in the form of text, images, videos, etc., and is centrally managed in the database.

[1552] Information organization using generative AI

[1553] The server periodically retrieves user posts from the database and passes them to the generation AI, which analyzes this data, organizes and summarizes the relevant information, allowing other users to efficiently obtain the information they need. The summarized information is then provided to users in an accessible format.

[1554] Travel planning support using conversational AI

[1555] When a user enters a travel planning question into the application, the device sends the query data to the server, which uses conversational AI to generate and return a travel itinerary that best suits the user's query, including specific travel destinations, places to visit, and recommended activities.

[1556] Providing AR / VR content

[1557] For users who find it difficult to travel or attend events, the server uses AR / VR technology to provide rich virtual experiences. When a user requests a virtual experience of a specific location or event, the device sends the request to the server. The server generates AR / VR content in real time based on the necessary data and provides the virtual experience. This allows users with financial or physical limitations to enjoy activities that interest them.

[1558] Specific examples

[1559] For example, if a user interested in model trains posts, "I want to go to a new model train exhibition," this information is stored in a database. Then, if another user uses the conversational AI to ask, "What train museum should I visit this weekend?", the conversational AI will suggest the best travel plan to answer that question. Furthermore, users who cannot physically visit museums can use AR / VR technology to virtually experience the inside of the museum.

[1560] This invention enables users with particularly niche hobbies to enrich their hobby activities by collecting and sharing information, planning trips, and engaging in virtual experiences, which is expected to promote interaction between users and improve engagement across the community.

[1561] The processing flow will be explained below.

[1562] Collecting and organizing user posts

[1563] Collecting user posts

[1564] Step 1:

[1565] User: Enters submissions into application or website forms.

[1566] Step 2:

[1567] Terminal: Sends the entered post content to the server.

[1568] Step 3:

[1569] Server: Stores the received submission data in a database.

[1570] Information organization

[1571] Step 4:

[1572] Server: Periodically retrieves user posts from the database.

[1573] Step 5:

[1574] Server: Passes the acquired data to the generation AI, which analyzes and summarizes the information.

[1575] Step 6:

[1576] Server: Provides summarized information in a form that users can access.

[1577] Travel planning support using conversational AI

[1578] Step 1:

[1579] User: Enters a travel planning question into a text box in the application.

[1580] Step 2:

[1581] Terminal: Sends the entered question to the server.

[1582] Step 3:

[1583] Server: Analyzes user questions using conversational AI and generates travel plans.

[1584] Step 4:

[1585] Server: Returns the generated itinerary to the user.

[1586] Step 5:

[1587] Terminal: Shows suggested travel itineraries to the user.

[1588] Providing AR / VR content

[1589] Step 1:

[1590] User: Requests a virtual experience of a specific location or event through a form in the app.

[1591] Step 2:

[1592] Terminal: Sends the input request to the server.

[1593] Step 3:

[1594] Server: Retrieves relevant data from a database based on the requested location or event.

[1595] Step 4:

[1596] Server: Generates the required AR / VR content in real time.

[1597] Step 5:

[1598] Server: Sends the generated AR / VR content to the user.

[1599] Step 6:

[1600] Device: Displays the received AR / VR content on the user's device.

[1601] Specific examples

[1602] Step 1:

[1603] User: Posts information about a model railroad show.

[1604] Step 2:

[1605] Terminal: Sends the post to the server.

[1606] Step 3:

[1607] Server: Stores the received posts in a database.

[1608] Step 4:

[1609] Server: Periodically retrieves posted data and summarizes it using a generation AI.

[1610] Step 5:

[1611] User: Finds museums to visit next weekend and asks the conversational AI a question.

[1612] Step 6:

[1613] Terminal: Sends the question to the server.

[1614] Step 7:

[1615] Server: Conversational AI generates optimal travel plans and suggests them to users.

[1616] Step 8:

[1617] User: If you can't make it to the museum, request a virtual experience.

[1618] Step 9:

[1619] Terminal: Sends the request to the server.

[1620] Step 10:

[1621] Server: Generates AR / VR content and provides it to users.

[1622] Step 11:

[1623] Device: Displays the received AR / VR content on the user's device.

[1624] This allows users to efficiently gather relevant information and enjoy travel and virtual experiences through a dedicated system, particularly for those with physical limitations, allowing them to enrich their hobbies.

[1625] Example 1

[1626] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1627] Conventional information gathering and community systems make it difficult for users with niche interests to efficiently collect and share information in an organized manner. They also lack the means for users to receive interactive assistance when planning trips, or for users with physical limitations to enjoy rich virtual experiences. As a result, it is difficult to provide comprehensive services to users with specific interests.

[1628] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1629] In this invention, the server includes means for receiving user posts from terminals and storing them in a database, means for analyzing the data stored in the database using a natural language processing model and organizing and providing related information, means for analyzing user inquiries using interactive artificial intelligence and proposing travel plans, means for dynamically generating and providing virtual reality technology content based on user requests, and means for retrieving user posts from the database and summarizing them using a natural language processing model. This allows users to efficiently collect information, receive travel plan suggestions, and enjoy rich experiences through virtual reality technology.

[1630] A "terminal" is a device that allows a user to input or receive information, and is often a smartphone, tablet, or computer.

[1631] "User Posts" refers to hobby-related information and experiences that users share through applications and websites, often in the form of text, images, videos, etc.

[1632] "Database" refers to a system for efficiently managing, storing, and retrieving data in a structured format.

[1633] A "natural language processing model" refers to an artificial intelligence technology that analyzes natural language data entered by a user and understands, generates, and summarizes human language.

[1634] "Conversational artificial intelligence" refers to a conversational artificial intelligence system that can provide natural responses to user questions and commands.

[1635] "Virtual reality technology" refers to technology that allows users to visually and aurally experience places and experiences that do not actually exist in the physical world.

[1636] "Means for organizing and providing information" refers to a method for using generative AI to analyze information stored in a database and provide related information to users in an easy-to-understand format.

[1637] "Means for proposing travel plans" refers to a method for using conversational AI to generate and propose optimal travel plans in response to user inquiries.

[1638] "Dynamic generation and serving means" refers to a method for creating and serving content in real time based on user requests.

[1639] "Means for summarizing" refers to a method for concisely summarizing multiple user-posted information using a natural language processing model.

[1640] This invention is a system for users with particularly niche interests to collect information and interact with the community. The system has four main functions: collecting user posts, organizing information using natural language processing models, assisting with travel planning using conversational artificial intelligence, and providing content using virtual reality technology.

[1641] Collecting user posts

[1642] Users post information about their hobbies and experiences through applications and websites. This data can be in the form of text, images, videos, etc.

[1643] The device receives the user's posted data and sends it to the server. For example, if a user posts "I want to go to the new model train exhibition," the device sends this to the server.

[1644] The server stores the received data in a database, which is a means of efficiently managing and centralizing this data.

[1645] Information organization using natural language processing models

[1646] The server periodically retrieves user posts from the database and passes them to a natural language processing model, such as OpenAI's GPT-3.

[1647] Natural language processing models analyze user posts and organize and summarize relevant information. The generated summaries are a means for other users to efficiently access information. For example, they can concisely summarize reviews of a model train exhibition posted by multiple users.

[1648] The server stores the summarized information in a database and makes it accessible to users.

[1649] Travel planning support using conversational AI

[1650] A user enters a travel planning question into an application, for example, "What train museums should I visit this weekend?"

[1651] The terminal transmits this inquiry data to the server.

[1652] The server uses conversational artificial intelligence (e.g., Dialogflow) to generate an optimal travel plan based on the user's questions, including destinations, places to visit, and recommended activities.

[1653] The server returns the generated itinerary to the user, who can view it within the app.

[1654] Virtual reality technology content provision

[1655] A user requests a virtual experience. For example, a request to "virtually experience the Railway Museum" is made within the app.

[1656] The terminal transmits this request data to the server.

[1657] The server collects relevant data and generates content in real time using virtual reality technology, which transcends physical limitations to provide users with a rich experience.

[1658] The server sends the generated content to the user's device, and the user experiences it using a dedicated app or VR headset.

[1659] Examples and prompts

[1660] For example, if a user who is interested in model trains posts that they want to go to a new model train exhibition, this information is stored in the database. Then, if another user uses the conversational AI to ask, "What train museum should I visit this weekend?", the conversational AI will suggest the best travel plan to answer that question. Additionally, users who cannot physically visit museums can use virtual reality technology to virtually experience the inside of the museum.

[1661] An example of a prompt to input to a generative AI model is as follows:

[1662] "I'm interested in model trains. Which railway museum should I visit this weekend? Also, what should I look out for when I visit?"

[1663] This invention is expected to enable users with particularly niche interests to efficiently gather information, receive travel plan suggestions, and enjoy rich virtual experiences, thereby increasing engagement across the community.

[1664] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1665] Processing Steps

[1666] Step 1: Collect user posts

[1667] 1. Users post information and experiences about their hobbies through the application or website.

[1668] Specific behavior:

[1669] A user posts, "I want to go to a model train exhibition."

[1670] Input and Output:

[1671] Input: Text data entered by the user (e.g., "I want to go to a model train exhibition")

[1672] Output: Text data is sent to the terminal.

[1673] 2. The terminal receives the posted data from the user and sends it to the server.

[1674] Specific behavior:

[1675] The app converts user input into JSON format and sends it to the server via an HTTP POST request.

[1676] Input and Output:

[1677] Input: JSON format text data (e.g., {"post": "I want to go to a model train exhibition"})

[1678] Output: The server receives the HTTP request.

[1679] 3. The server stores the received data in a database.

[1680] Specific behavior:

[1681] The server parses the data from the received POST request and stores it as a new entry in the database.

[1682] Input and Output:

[1683] Input: HTTP request content data

[1684] Output: A new entry is created in the database.

[1685] Step 2: Organizing information using natural language processing models

[1686] 1. The server periodically retrieves user posts from the database.

[1687] Specific behavior:

[1688] A scheduler on the server runs every day at 2 AM and retrieves all recently added user posts.

[1689] Input and Output:

[1690] Input: User posted data from database

[1691] Output: A list of retrieved user post data

[1692] 2. The server passes the acquired data to the natural language processing model.

[1693] Specific behavior:

[1694] The server formats user posts into API requests and sends them to a natural language processing model.

[1695] Input and Output:

[1696] Input: User post data in API request format

[1697] Output: Data sent to the natural language processing model

[1698] 3. A natural language processing model analyzes the data and generates summary information.

[1699] Specific behavior:

[1700] A natural language processing model analyzes multiple posts and summarizes relevant information.

[1701] Input and Output:

[1702] Input: User Post Data

[1703] Output: Summary information (e.g., "Summary of reviews of recent model railroad shows")

[1704] 4. The server stores the generated summary information in a database and provides it in a user-accessible format.

[1705] Specific behavior:

[1706] The server stores the summary information in a database and displays it in specific sections of the website or app.

[1707] Input and Output:

[1708] Input: Summary information

[1709] Output: Summary information stored in a database, summary information displayed on a website or app

[1710] Step 3: Travel planning assistance with conversational AI

[1711] 1. The user enters a travel planning question into the application.

[1712] Specific behavior:

[1713] A user types, "What train museum should I visit this weekend?"

[1714] Input and Output:

[1715] Input: User question text

[1716] Output: Text data is sent to the terminal.

[1717] 2. The device sends the query data to the server.

[1718] Specific behavior:

[1719] The app converts the question into JSON format and sends it to the server via an HTTP POST request.

[1720] Input and Output:

[1721] Input: JSON-formatted question data (e.g., {"query": "What railway museum should I visit this weekend?"})

[1722] Output: The server receives the HTTP request.

[1723] 3. The server uses conversational artificial intelligence to generate the best travel plan for the query.

[1724] Specific behavior:

[1725] The server analyzes the question data and sends it to a conversational artificial intelligence, which then generates travel destination and activity suggestions.

[1726] Input and Output:

[1727] Input: User question data

[1728] Output: Generated itinerary

[1729] 4. The server returns the generated itinerary to the user.

[1730] Specific behavior:

[1731] The server receives the conversational AI's response and sends it to the user's device.

[1732] Input and Output:

[1733] Input: Generated itinerary

[1734] Output: The itinerary sent to the user's device

[1735] Step 4: Providing content for virtual reality technology

[1736] 1. A user requests a virtual experience.

[1737] Specific behavior:

[1738] The user clicks the "Experience the Railway Museum virtually" button within the app.

[1739] Input and Output:

[1740] Input: Virtual experience request data

[1741] Output: Text data is sent to the terminal.

[1742] 2. The device sends the request data to the server.

[1743] Specific behavior:

[1744] The app converts the request content into JSON format and sends it to the server via an HTTP POST request.

[1745] Input and Output:

[1746] Input: Request data in JSON format (e.g., {"request": "Virtual experience at the Railway Museum"})

[1747] Output: The server receives the HTTP request.

[1748] 3. The server generates virtual reality technology content in real time based on the required data.

[1749] Specific behavior:

[1750] The server collects the relevant data and processes it to generate the virtual reality technology content.

[1751] Input and Output:

[1752] Input: Request data and related data

[1753] Output: Generated virtual reality technology content

[1754] 4. The server provides the generated virtual reality technology content to the user's terminal.

[1755] Specific behavior:

[1756] The server sends the generated content URL to the user's device, where the user experiences it using a dedicated app or VR headset.

[1757] Input and Output:

[1758] Input: Generated virtual reality technology content

[1759] Output: The content URL served to the user's device

[1760] (Application example 1)

[1761] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1762] Existing social media and information gathering methods lack the functionality to allow users to efficiently gather and share useful information related to niche hobbies. Furthermore, when physical or financial constraints make it difficult for users to visit the destination in person, their information gathering and experiences are limited. To solve these issues, efficient information organization, travel planning support, and the provision of virtual experiences using AR / VR technology are required.

[1763] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1764] In this invention, the server includes: means for receiving user posts from the device and storing them in a database; means for analyzing the data stored in the database using a generation AI and organizing and providing related information; means for the cloud server to periodically pass user data to the generation AI, which organizes and summarizes the related information and provides it to the user via a smartphone application; means for analyzing user inquiries using an interactive AI and proposing travel plans; means for dynamically generating and providing AR / VR content based on user requests; means for visualizing information provided to the device via a smartphone; means for sending input data to the cloud server and storing it in a database in text, image, or video format; and means for providing prompts to the generation AI based on user posts and requests to help organize information and generate travel plans. This allows users with niche hobbies to efficiently collect and share information and enjoy their hobbies through virtual experiences beyond physical and financial constraints.

[1765] A "terminal" is a device used to input user posts and send them to a server, such as a smartphone or tablet.

[1766] "User Posts" are data posted by users that includes information, experiences, opinions, images, and videos related to their specific hobbies or interests.

[1767] A "database" is a system that centrally stores and manages user posts and is operated by a server.

[1768] "Generative AI" is an artificial intelligence model used to analyze user posts and collect, organize, and summarize relevant information.

[1769] "Conversational AI" is an artificial intelligence model that analyzes user inquiries and generates optimal answers and suggestions.

[1770] A "travel plan" is a plan including destinations, transportation, accommodation, activities, etc., when a user travels.

[1771] "AR / VR content" refers to content that provides users with a virtual experience using augmented reality and virtual reality technologies.

[1772] A "cloud server" is a remote server used to store, process, and manage data over the Internet.

[1773] A "smartphone application" is software that runs primarily on smartphones and is used to collect user posts, organize information, provide travel plans, display AR / VR content, and more.

[1774] "Related information" is information extracted, organized, and summarized by the AI ​​based on information obtained from user posts, and includes content that may be useful to other users.

[1775] A "prompt sentence" is an input sentence that instructs the generation AI to perform a specific analysis or generate information.

[1776] The system for implementing this invention includes multiple means for efficiently analyzing data collected from users and organizing and suggesting information. As an application example, we will explain a specific system that allows users with niche hobbies to collect information related to their hobbies and interact with other users.

[1777] Hardware and software used

[1778] The hardware and software used to implement this invention are as follows:

[1779] Smartphone: A device used to input and visualize user posts and communicate with the cloud server.

[1780] Cloud server: A server for storing user posts and analyzing data using generation AI and conversational AI.

[1781] Database: Located on a cloud server, it centrally manages user-posted data.

[1782] Generative AI model: An artificial intelligence model (e.g., GPT-4) that analyzes user posts and organizes and summarizes relevant information.

[1783] Conversational AI model: An artificial intelligence model (e.g., Dialogflow) that analyzes user inquiries and generates and suggests optimal travel plans.

[1784] AR / VR Content Generation Tools: Tools for generating augmented reality and virtual reality content, such as Unity and Unreal Engine.

[1785] Program processing

[1786] Collecting user posts

[1787] Users use a smartphone application to input and post information about their hobbies. The smartphone sends this posted data to a cloud server and stores it in a database. The posted data can be in the form of text, images, videos, etc.

[1788] Information organization using generative AI

[1789] The cloud server periodically retrieves user posts from the database and passes them to the generation AI, which analyzes the data and organizes and summarizes the relevant information. The organized information is then made available to users in a form accessible through a smartphone application.

[1790] Travel planning support using conversational AI

[1791] When a user enters a travel planning question into a smartphone application, the smartphone sends the inquiry data to a cloud server, which uses conversational AI to generate an optimal travel plan based on the user's question and returns suggestions, including specific travel destinations, places to visit, and recommended activities.

[1792] Providing AR / VR content

[1793] When a user requests a virtual experience of a specific location or event, their smartphone sends the request to a cloud server. The cloud server generates AR / VR content in real time based on the necessary data and provides it to the user through a smartphone application. This allows users with physical or financial constraints to virtually experience activities that interest them.

[1794] Examples of concrete examples and prompts

[1795] For example, if a user interested in model trains posts that they want to go to a new model train exhibition, their smartphone will send this information to a server and store it in a database. Next, if another user asks, "What railway museum should I visit this weekend?", the conversational AI will suggest the optimal travel plan. Furthermore, users who cannot visit a railway museum can use AR / VR technology to virtually experience the inside of the museum.

[1796] Example prompt sentence:

[1797] "Please tell me the latest model train exhibition."

[1798] "Can you recommend a railway museum I can visit next weekend?"

[1799] "Show me a virtual tour of the model railroad exhibition."

[1800] In this way, the system utilizes generative and conversational AI, as well as AR / VR content generation tools, to enable users to gather information on their niche interests, plan trips, and provide virtual experiences.

[1801] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1802] Step 1:

[1803] Users use a smartphone application to input and post information about their hobbies. The input data includes text, images, videos, etc. The input data from the user is sent to the device.

[1804] Step 2:

[1805] The device sends user post data to the cloud server. The data arrives in the form of text, images, or videos and is stored in a database. The cloud server receives user posts and centralizes them in a database.

[1806] Step 3:

[1807] The cloud server periodically retrieves user posts from the database. This retrieved data is passed to the generation AI. The cloud server analyzes the contents of the database and passes the relevant data to the generation AI.

[1808] Step 4:

[1809] Generative AI analyzes user posts and organizes and summarizes relevant information. Input data is analyzed by generative AI, and the desired information is organized and output. Generative AI uses a model (e.g., GPT-4) to analyze text data and generate a summary of the information.

[1810] Step 5:

[1811] The cloud server retrieves the summarized data from the generated AI and provides it to the user via a smartphone application. The summarized data is output in a format that can be displayed on the user's smartphone. The cloud server then sends the organized and summarized information to the user.

[1812] Step 6:

[1813] A user inputs a question about their travel plan into a smartphone application. The user's question data is entered into the terminal. This input is sent to the conversational AI.

[1814] Step 7:

[1815] The device sends question data to the cloud server. The question data arrives at the cloud server and is received by the conversational AI. The conversational AI analyzes the question using a model (e.g., Dialogflow).

[1816] Step 8:

[1817] The conversational AI analyzes the user's inquiry and generates the optimal travel plan. The analyzed question data is processed by the conversational AI to generate the travel plan. The generated plan is sent to the cloud server.

[1818] Step 9:

[1819] The cloud server obtains the travel plan from the conversational AI and provides it to the user via a smartphone application. The generated travel plan is displayed on the smartphone in a format that is easy for the user to understand. The cloud server then sends the travel plan to the user's device.

[1820] Step 10:

[1821] A user requests a virtual experience of a specific location or event. The user's request data is entered into a smartphone application. This input is sent to a cloud server.

[1822] Step 11:

[1823] The cloud server receives the request and generates the virtual experience content using an AR / VR content generation tool (e.g., Unity, Unreal Engine). The request data reaches the cloud server, and the AR / VR content is generated based on the required data.

[1824] Step 12:

[1825] The cloud server provides the generated AR / VR content to the user via a smartphone application. The generated AR / VR content is displayed on the smartphone or head-mounted display. The cloud server provides the virtual experience to the user, allowing the user to view it.

[1826] This allows users to virtually experience activities that interest them, regardless of physical or financial constraints.

[1827] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1828] This invention is a system that allows users with particularly niche hobbies to collect information and interact with the community. This system combines the collection of user posts, information organization using generative AI, travel planning support using conversational AI, provision of AR / VR content, and an emotion engine that recognizes and responds to user emotions. The details of this system are described below.

[1829] Collecting user posts

[1830] Users post information about their hobbies and experiences through applications and websites. The device receives this posted data and sends it to a server. The server stores the received data in a database. This data can be in the form of text, images, videos, etc., and is centrally managed in the database.

[1831] Information organization using generative AI

[1832] The server periodically retrieves user posts from the database and passes them to the generation AI, which analyzes this data, organizes and summarizes the relevant information, allowing other users to efficiently obtain the information they need. The summarized information is then provided to users in an accessible format.

[1833] Travel planning support using conversational AI

[1834] When a user enters a travel planning question into the application, the device sends the query data to the server, which uses conversational AI to generate and return a travel itinerary that best suits the user's query, including specific travel destinations, places to visit, and recommended activities.

[1835] Providing AR / VR content

[1836] For users who find it difficult to travel or attend events, the server uses AR / VR technology to provide rich virtual experiences. When a user requests a virtual experience of a specific location or event, the device sends the request to the server. The server generates AR / VR content in real time based on the necessary data and provides the virtual experience. This allows users with financial or physical limitations to enjoy activities that interest them.

[1837] Response by Emotion Engine

[1838] While the user is using the system, the emotion engine analyzes the user's emotions. For example, it determines the user's emotional state from the user's text input, voice, and facial expressions. The server uses this emotion data to adjust the information provided by the generative AI and the travel plans proposed by the conversational AI. For example, if the user is feeling stressed, it can suggest travel destinations and activities aimed at relaxation.

[1839] Specific examples

[1840] For example, if a user interested in model trains posts with a positive emotion, "I want to go to a new model train exhibition," this information is stored in a database and shared with other users by the generative AI. Next, if another user uses the conversational AI to ask, "What train museums should I visit this weekend?", the conversational AI will suggest the best travel plan to answer that question. Similarly, if a user requests a virtual experience with an emotion expressing fatigue, the AI ​​can suggest relaxing AR / VR content.

[1841] This will enable users with particularly niche hobbies to enrich their hobby activities by collecting and sharing information, planning trips, and engaging in virtual experiences.The introduction of an emotion engine is expected to provide personalized experiences based on the user's emotional state, improving engagement across the community.

[1842] The processing flow will be explained below.

[1843] Collecting and organizing user posts

[1844] Collecting user posts

[1845] Step 1:

[1846] User: Enters submissions into application or website forms.

[1847] Step 2:

[1848] Terminal: Sends the entered post content to the server.

[1849] Step 3:

[1850] Server: Stores the received submission data in a database. The submission data includes text, images, videos, etc.

[1851] Information organization

[1852] Step 4:

[1853] Server: Periodically retrieves user posts from the database.

[1854] Step 5:

[1855] Server: Passes the acquired data to the generation AI, which analyzes and summarizes the information.

[1856] Step 6:

[1857] Server: Provides summarized information in a form that users can access, allowing other users to efficiently retrieve the information they need.

[1858] Travel planning support using conversational AI

[1859] Step 1:

[1860] User: Enters a travel planning question into a text box in the application.

[1861] Step 2:

[1862] Terminal: Sends the entered question to the server.

[1863] Step 3:

[1864] Server: Analyzes user questions using conversational AI and generates travel plans.

[1865] Step 4:

[1866] Server: Returns the generated itinerary to the user.

[1867] Step 5:

[1868] On the device: Show the user a suggested travel itinerary, including specific destinations, places to visit, and recommended activities.

[1869] Providing AR / VR content

[1870] Step 1:

[1871] User: Requests a virtual experience of a specific location or event through a form in the app.

[1872] Step 2:

[1873] Terminal: Sends the input request to the server.

[1874] Step 3:

[1875] Server: Retrieves relevant data from a database based on the requested location or event.

[1876] Step 4:

[1877] Server: Generates the required AR / VR content in real time.

[1878] Step 5:

[1879] Server: Sends the generated AR / VR content to the user.

[1880] Step 6:

[1881] Device: The received AR / VR content is displayed on the user's device. Users can enjoy the virtual experience using a dedicated device or app.

[1882] Response by Emotion Engine

[1883] Step 1:

[1884] User: Providing emotion-based text input, voice, facial expressions, etc. while using the system.

[1885] Step 2:

[1886] Device: Sends emotion-related data to the server.

[1887] Step 3:

[1888] Server: The emotion engine analyzes the received data and determines the user's emotional state.

[1889] Step 4:

[1890] Server: Based on the emotion data, adjust the information provided by the generative AI and the travel plan proposed by the conversational AI. For example, if the user expresses stress, suggest places and activities that will help them relax.

[1891] Step 5:

[1892] Server: Provide users with tailored information and plans. By providing information that is tailored to their emotions, users can have a more satisfying experience.

[1893] Specific examples

[1894] For example, consider the case of a user who wants to go to a model railroad exhibition.

[1895] Step 1:

[1896] User: Posts "I want to go to the new model train show." Shows positive sentiment.

[1897] Step 2:

[1898] Terminal: Sends the post to the server.

[1899] Step 3:

[1900] Server: Save the submitted data to a database.

[1901] Step 4:

[1902] Server: Periodically retrieves posted data and summarizes it using a generation AI.

[1903] Step 5:

[1904] Server: If the emotion engine confirms a user's positive emotions, it provides information that promotes a positive experience for other users.

[1905] Step 6:

[1906] User: "Tell me which train museum I should visit this weekend," asks the conversational AI, showing some tiredness.

[1907] Step 7:

[1908] Terminal: Sends the question to the server.

[1909] Step 8:

[1910] Server: Conversational AI generates optimal travel plans and suggests relaxing places that match the user's emotional state.

[1911] Step 9:

[1912] Server: Sends the proposal back to the user.

[1913] Step 10:

[1914] User: Requests AR / VR virtual experience with emotion indicating fatigue.

[1915] Step 11:

[1916] Terminal: Sends the request to the server.

[1917] Step 12:

[1918] Server: Generates and provides relaxing AR / VR content in real time.

[1919] Step 13:

[1920] Device: Displays the received AR / VR content on the user's device.

[1921] This will enable users with particularly niche hobbies to enrich their hobby activities by collecting and sharing information, planning trips, and engaging in virtual experiences.The introduction of an emotion engine is expected to provide personalized experiences based on the user's emotional state, improving engagement across the community.

[1922] Example 2

[1923] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1924] Until now, there have been no systems that can collect and share information, plan trips, provide virtual experiences, or respond to users' emotions for users with niche hobbies. As a result, users have been unable to efficiently obtain information that interests them, and have difficulty receiving appropriate activity suggestions and virtual experiences, resulting in a decline in user experience and community engagement.

[1925] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1926] In this invention, the server includes means for receiving user posts from the terminal and storing them in a database, means for analyzing the data stored in the database using a generation AI and organizing and providing related information, means for analyzing user inquiries using a conversational AI and proposing travel plans, means including an emotion engine for analyzing user emotions and adjusting the output of the generation AI and the conversational AI, and means for dynamically generating and providing AR / VR content based on user requests. This allows users to efficiently collect information and receive appropriate travel plans and virtual experiences, and further enables the provision of personalized experiences according to the user's emotions.

[1927] "Terminal" refers to a device used by a user, including smartphones and personal computers.

[1928] "User Posts" refers to hobby-related information and experiences posted by Users through the Application or Website.

[1929] "Database" refers to a data storage system for centrally storing and managing collected data.

[1930] "Generative AI" is a system that uses artificial intelligence to generate and analyze text, images, etc., and is used for the purpose of organizing and summarizing information.

[1931] "Conversational AI" is artificial intelligence that generates answers in natural language to user inquiries, and is used for question answering and information provision.

[1932] An "emotion engine" is a system that analyzes a user's text input, voice, facial expressions, etc. to determine their emotional state and adjusts the output of other functions based on the results.

[1933] "AR / VR content" refers to rich virtual experiences generated using augmented reality (AR) and virtual reality (VR) technologies.

[1934] The present invention is a system for users with particularly niche interests to collect information and interact with communities. This system combines collection of user posts, information organization using generative AI, travel planning support using conversational AI, provision of AR / VR content, and an emotion engine that recognizes and responds to user emotions. Specific embodiments of the present invention are described below.

[1935] Collecting user posts

[1936] A user accesses an application or website and posts information about their hobbies and experiences. The device receives this posted data and sends it to a server. The server stores the received data in a database. This data can be in the form of text, images, videos, etc., and is centrally managed in a database (e.g., MySQL).

[1937] Information organization using generative AI

[1938] The server periodically retrieves user posts from the database and passes them to the Generative AI, which (e.g., OpenAI GPT-4) analyzes the data and organizes and summarizes the relevant information. The summarized information is then made available by the server in a form that can be accessed by other users.

[1939] Travel planning support using conversational AI

[1940] When a user enters a travel plan question into the application, the device sends the question data to a server, which uses conversational AI (e.g., Google Dialogflow) to generate a travel plan that best suits the user's question and returns suggestions, including specific travel destinations, places to visit, and recommended activities.

[1941] Providing AR / VR content

[1942] For users who find it difficult to travel or attend events, the server uses AR / VR technology to provide rich virtual experiences. When a user requests a virtual experience of a specific location or event, the device sends the request to the server. The server generates AR / VR content in real time based on the necessary data and provides it to the user. This allows users with financial or physical constraints to enjoy activities that interest them. The specific software used includes Unity.

[1943] Response by Emotion Engine

[1944] While the user is using the system, the emotion engine analyzes the user's emotional state from text input, voice, facial expressions, etc. The server uses this emotional data to adjust the information provided by the generative AI and the travel plans suggested by the conversational AI. For example, if the user is feeling stressed, it can suggest travel destinations and activities aimed at relaxation. The specific software used includes Microsoft Azure Cognitive Services.

[1945] Specific examples

[1946] For example, if a user interested in model trains posts a positive message saying, "I want to go to a new model train exhibition," the device sends the message to the server, which stores it in a database. This information is then organized by the generative AI and shared with other users.

[1947] Next, another user can use the conversational AI to ask, "What railway museums should I visit this weekend?" and the conversational AI will suggest the best travel plan to answer that question.

[1948] In addition, for users who express fatigue and request a "relaxing virtual experience," the system can suggest relaxing AR / VR content.

[1949] In this way, the system of the present invention enables efficient information gathering, personalized travel planning, and virtual experiences, especially for users with niche hobbies, thereby increasing user engagement and further enriching their hobby activities.

[1950] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1951] Step 1:

[1952] Users access applications or websites and post information about their hobbies and experiences. The data entered can be in the form of text, images, videos, etc. The device receives this posted data and sends it to the server.

[1953] Specific behavior:

[1954] A user enters the text "I want to go to the new model train exhibition" into the app, attaches an image and posts it.

[1955] The terminal sends this as text data and image data to the server. The input is text and image data, and the output is data sent to the server.

[1956] Step 2:

[1957] The server stores the received submission data in a database. The data can be in various formats, such as text, images, and videos, and is managed centrally.

[1958] Specific behavior:

[1959] The server executes a SQL query to insert the received data (text and images) into the database. The input is the received data, and the output is saving it to the database.

[1960] Step 3:

[1961] The server periodically retrieves user posts from the database and passes them to the generation AI, which analyzes the data and organizes and summarizes the relevant information.

[1962] Specific behavior:

[1963] The server retrieves new post data saved from the previous day from the database at a fixed time every day.

[1964] The acquired data is input into the generative AI (OpenAI GPT-4) to generate summarized information. The input is data acquired from the database, and the output is summarized information.

[1965] Step 4:

[1966] The server stores the generated summary information back in a database, making it accessible to other users.

[1967] Specific behavior:

[1968] The server inserts the summary data output by the generation AI into the database.

[1969] This information is displayed on a website or application for users to view. The input is the output data of the generative AI, and the output is stored in a database and provided to the user.

[1970] Step 5:

[1971] The user inputs a travel planning question within the application, and the device sends the question data to the server.

[1972] Specific behavior:

[1973] A user types, "What train museum should I visit this weekend?"

[1974] The terminal sends this text data to the server. The input is the user's question data, and the output is the data sent to the server.

[1975] Step 6:

[1976] The server uses conversational AI to generate the optimal travel plan for the user's question. The conversational AI generates an answer to the question.

[1977] Specific behavior:

[1978] The server inputs question data into a conversational AI (Google Dialogflow) and generates a travel plan.

[1979] The conversational AI generates a list of optimal travel destinations and places to visit and returns it to the server. The input is the question data, and the output is the generated travel plan.

[1980] Step 7:

[1981] The server returns the generated itinerary to the terminal and displays it to the user.

[1982] Specific behavior:

[1983] The server sends the output of the conversational AI to the terminal.

[1984] The device displays the optimal travel plan to the user. The input is the output data of the conversational AI, and the output is the display to the user.

[1985] Step 8:

[1986] The user requests a virtual experience of a specific location or event, and the device sends this request to the server.

[1987] Specific behavior:

[1988] A user requests a virtual experience of a model railroad exhibition.

[1989] The terminal sends this request data to the server. The input is the user's request data, and the output is the data sent to the server.

[1990] Step 9:

[1991] The server uses AR / VR technology to generate a rich virtual experience and deliver it to the user's device. The software used includes Unity.

[1992] Specific behavior:

[1993] The server uses Unity to generate the virtual experience.

[1994] The generated AR / VR content is sent to the device so that the user can experience it. The input is the user's request data, and the output is the generated AR / VR content.

[1995] Step 10:

[1996] While the user is using the system, the emotion engine analyzes the user's emotions, and the server uses this emotional data to adjust the output of the generative and conversational AI.

[1997] Specific behavior:

[1998] When a user types a question within the app, the text, voice data, and facial expressions are input into an emotion analysis engine.

[1999] The server analyzes the emotion data using an emotion analysis engine (Microsoft Azure Cognitive Services) and feeds it back to the generative AI and conversational AI to provide information and adjust travel plans. The input is the user's emotion data, and the output is adjusted information.

[2000] (Application example 2)

[2001] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2002] Currently, there are limited platforms that allow users with diverse hobbies to efficiently gather information that matches their interests and interact with communities that share the same hobbies. Furthermore, users often have unsatisfactory experiences due to the effort required to gather information when planning a trip and the lack of personalized product purchasing experiences. Furthermore, users are not provided with information or experiences that are in line with their emotions, which leads to low user engagement.

[2003] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving user posts from the terminal and storing them in a database, means for analyzing the data stored in the database using a generation AI and organizing and providing related information, means for analyzing user inquiries using an interactive AI and proposing travel plans, means for dynamically generating and providing AR / VR content based on user requests, means for providing information on products and services related to the user posts in a virtual shopping mall, and an emotion engine for analyzing user emotions and providing personalized content and product suggestions based on the analysis. This not only enables users to efficiently collect information on niche hobbies and interact with communities, but also enables activities such as travel plans, virtual experiences, and product purchases to be personalized, enabling the provision of services that are tailored to their emotions.

[2004] A "terminal" is an electronic device through which a user inputs information and communicates with the system.

[2005] "User posts" are content such as text, images, and videos of information and experiences posted by users via their devices.

[2006] A "database" is a system that centrally stores and manages data such as received user posts.

[2007] "Generative AI" is an artificial intelligence technology that analyzes collected data and organizes and summarizes related information.

[2008] "Conversational AI" is an artificial intelligence technology that analyzes user inquiries and provides optimal answers and plans.

[2009] A "travel plan" is a plan that includes locations, activities, accommodations, etc., when a user travels.

[2010] "AR / VR content" means visual and experiential content created using augmented reality (AR) and virtual reality (VR) technologies.

[2011] A "virtual shopping mall" is an e-commerce platform where users can explore and purchase goods and services in a virtual environment.

[2012] The "emotion engine" is a function that analyzes the user's emotional state and provides personalized services and content based on the results.

[2013] A "system" is a set of technical components in which multiple means work together to achieve a specific purpose.

[2014] The system for realizing this invention operates by combining multiple means. The system is mainly composed of terminals, servers, analysis of user posts, and the introduction of new technologies.

[2015] First, users use their devices to post information about their hobbies and experiences. This information is received in the form of text, images, and videos and is stored in a database via a server. The database serves to organize and centrally manage the information.

[2016] The server then periodically retrieves user posts from the database and analyzes them using a generative AI, which uses AI technologies such as TensorFlow.js to organize and summarize the collected data into relevant information, allowing other users to efficiently access the information they need.

[2017] Furthermore, when users ask questions about their travel plans, the server uses conversational AI to suggest optimal travel plans, such as using Dialogflow to generate answers to the user's questions and provide them to the user. This plan includes travel destinations, places to visit, and recommended activities.

[2018] AR / VR content can also be provided through the device. When a user requests a virtual experience of a specific location or event, the server generates the AR / VR content in real time using Unity or WebXR and provides it to the user. This allows users with financial or physical limitations to virtually experience activities that interest them.

[2019] Finally, an emotion engine is used to analyze the user's emotional state, for example, by determining emotions from the user's text input, voice, or facial expressions, and then adjust the information provided by the generative AI and the travel itinerary suggested by the conversational AI. The emotion engine then provides personalized product suggestions and content to the user.

[2020] Specific examples

[2021] For example, if a user interested in model trains posts with positive emotions, "I want to go to a new model train exhibition," this information is stored in a database and shared with other users by the generative AI. Next, if another user uses the conversational AI to ask, "What railway museum should I visit this weekend?", the conversational AI will suggest the best travel plan. Additionally, if a user feels tired, the AI ​​can suggest relaxing AR / VR content.

[2022] Prompt Sentence Examples

[2023] "Can you give me some information about the latest model railroad exhibition?"

[2024] "Show me your new model train station."

[2025] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2026] Step 1:

[2027] Users post information about their hobbies and experiences through their devices. These posts are entered in the form of text, images, videos, etc., and are sent from the device to the server. The server receives this data and stores it in a database. During this process, the server properly organizes the data format and metadata to centrally manage the posted data.

[2028] Step 2:

[2029] The server periodically retrieves user posts from the database. It then uses a generative AI to analyze the data. This analysis is used to organize and summarize relevant information. The generative AI uses TensorFlow.js to perform semantic analysis of text data and image recognition to effectively organize information. For example, it extracts keywords from posted text and uses them to summarize related information.

[2030] Step 3:

[2031] The server receives questions from users. For example, if a user types "Please tell me which railway museum I should visit this weekend" through a terminal, this question is sent to the server. The server analyzes this inquiry using a conversational AI (e.g., Dialogflow). Based on the results of the analysis, it generates an optimal travel plan and proposes it to the user. In this case, the server obtains answer data for the question from the generation AI and provides a personalized answer.

[2032] Step 4:

[2033] When a user requests a virtual experience of a specific location or event, the device sends this request to the server. The server uses Unity or WebXR to dynamically generate AR / VR content based on the request. The generated content is then delivered to the user in real time. For example, if a user requests, "Show me the new model train exhibition," the server generates a virtual exhibition and delivers it to the user.

[2034] Step 5:

[2035] The server analyzes the user's emotional state. It collects data such as text input, voice, and facial expressions while the user is using the device and analyzes it using an emotion engine. Based on this analysis, it personalizes the information and content provided to the user. For example, if the user is feeling stressed, the emotion engine will suggest content and products suitable for relaxation.

[2036] As described above, each step works in conjunction to create a system that provides users with an efficient and personalized experience.

[2037] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2038] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2039] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2040] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2041] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2042] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2043] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2044] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2045] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2046] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2047] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2048] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2049] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2050] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2051] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[2052] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[2053] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[2054] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[2055] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[2056] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[2057] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[2058] The following is further disclosed regarding the above embodiment.

[2059] (Claim 1)

[2060] A means for receiving user posts from the terminal and storing them in a database;

[2061] A means for analyzing the data stored in the database using AI generation and organizing and providing related information;

[2062] A means to analyze user inquiries using conversational AI and suggest travel plans,

[2063] A means for dynamically generating and serving AR / VR content based on user requests;

[2064] A system including:

[2065] (Claim 2)

[2066] 10. The system of claim 1, further comprising means for retrieving user-posted data from a database and summarizing the information using generative AI.

[2067] (Claim 3)

[2068] 10. The system of claim 1, further comprising means for receiving a travel plan inquiry from a user and generating and suggesting an optimal travel plan using conversational AI.

[2069] "Example 1"

[2070] (Claim 1)

[2071] A means for receiving user posts from the terminal and storing them in a database;

[2072] A means for analyzing the data stored in the database using a natural language processing model and organizing and providing related information;

[2073] A means of analyzing user inquiries using interactive artificial intelligence and proposing travel plans;

[2074] means for dynamically generating and providing virtual reality technology content based on user requests;

[2075] A means of retrieving user posts from a database and summarizing them using a natural language processing model;

[2076] A system including:

[2077] (Claim 2)

[2078] 10. The system of claim 1, further comprising means for summarizing information using a natural language processing model.

[2079] (Claim 3)

[2080] 10. The system of claim 1, further comprising means for receiving a travel planning inquiry from a user and generating and suggesting an optimal travel plan using interactive artificial intelligence.

[2081] "Application Example 1"

[2082] (Claim 1)

[2083] A means for receiving user posts from the terminal and storing them in a database;

[2084] A means for analyzing the data stored in the database using AI generation and organizing and providing related information;

[2085] A means to analyze user inquiries using conversational AI and suggest travel plans,

[2086] A means for dynamically generating and serving AR / VR content based on user requests;

[2087] A means for visualizing the information provided to the terminal via a smartphone, and a means for sending the input data to a cloud server and storing it in a database in text, image, or video format;

[2088] The cloud server periodically passes user data to the generation AI, organizes and summarizes relevant information, and provides it to the user via a smartphone application.

[2089] A means to provide prompts to the AI ​​based on user posts and requests to help organize information and generate travel plans.

[2090] A system including:

[2091] (Claim 2)

[2092] 10. The system of claim 1, further comprising means for retrieving user-posted data from a database and summarizing the information using generative AI.

[2093] (Claim 3)

[2094] 10. The system of claim 1, further comprising: means for receiving a travel plan inquiry from a user and generating and suggesting an optimal travel plan using conversational AI; and means for dynamically generating and providing AR / VR content.

[2095] "Example 2: Combining Emotion Engines"

[2096] (Claim 1)

[2097] A means for receiving user posts from the terminal and storing them in a database;

[2098] A means for analyzing the data stored in the database using AI generation and organizing and providing related information;

[2099] A means to analyze user inquiries using conversational AI and suggest travel plans,

[2100] a means for analyzing user emotions and adjusting the output of the generative AI and the conversational AI, the means including an emotion engine;

[2101] A means for dynamically generating and serving AR / VR content based on user requests;

[2102] A system including:

[2103] (Claim 2)

[2104] 10. The system of claim 1, further comprising means for retrieving user-posted data from a database and summarizing the information using generative AI.

[2105] (Claim 3)

[2106] 10. The system of claim 1, further comprising means for receiving a travel plan inquiry from a user and generating and suggesting an optimal travel plan using conversational AI.

[2107] "Application example 2 when combining emotion engines"

[2108] (Claim 1)

[2109] A means for receiving user posts from the terminal and storing them in a database;

[2110] A means for analyzing the data stored in the database using AI generation and organizing and providing related information;

[2111] A means to analyze user inquiries using conversational AI and suggest travel plans,

[2112] A means for dynamically generating and serving AR / VR content based on user requests;

[2113] A means for providing information on products and services related to user posts in a virtual shopping mall;

[2114] a means including an emotion engine for analyzing user emotions and providing personalized content and product suggestions based thereon;

[2115] A system including:

[2116] (Claim 2)

[2117] 10. The system of claim 1, further comprising means for retrieving user-posted data from a database and summarizing the information using generative AI.

[2118] (Claim 3)

[2119] 10. The system of claim 1, further comprising means for receiving a travel plan inquiry from a user and generating and suggesting an optimal travel plan using conversational AI. [Explanation of symbols]

[2120] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for receiving user posts from the terminal and storing them in a database; A means of analyzing the data stored in the above database using generation AI and organizing and providing related information; A means to analyze user inquiries using conversational AI and suggest travel plans, A means for dynamically generating and serving AR / VR content based on user requests; A system including:

2. The system of claim 1 further comprising means for retrieving user-posted data from a database and summarizing the information using a generative AI.

3. The system of claim 1 , further comprising means for receiving a travel plan inquiry from a user and generating and suggesting an optimal travel plan using conversational AI.

Images