system

Abstract

We provide the system. [Solution] An information processing means that analyzes behavioral data to generate personalized visit plans based on the user's interests, A display means that provides visual information to users using virtual reality technology, A data processing method that analyzes and presents congestion status and waiting times in real time, A data correction method that collects feedback and uses it to improve the system, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, and includes steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In existing entertainment facilities, it is difficult to provide a personalized visit plan corresponding to the interests and behaviors of users, and often has to rely on general guidance information. For this reason, users cannot obtain an efficient and highly satisfactory experience, and there is a problem that the utilization value of the facility is not fully exerted.

Means for Solving the Problems

[0005] This invention provides a personalized experience by incorporating information processing means that analyze user behavior data and generate individualized visit plans based on user interests. Furthermore, it improves the immediacy and intuitiveness of information for users by providing visual information using virtual reality technology. In addition, by incorporating data processing means that analyze and present congestion levels and waiting times in real time, it enables efficient movement within the facility and improves user satisfaction. Moreover, by collecting user feedback and utilizing it to improve the system, the accuracy of suggestions can be continuously enhanced.

[0006] "Behavioral data" refers to information collected based on records of users' movements, choices, and time allocation within the facility.

[0007] A "personalized visit plan" is a plan that suggests the optimal route within the facility and the attractions based on the user's interests and behavioral patterns.

[0008] "Information processing means" refers to technical processes or devices for analyzing data and generating results according to a specific purpose.

[0009] "Virtual reality technology" is a technology that uses computers to provide a visual experience similar to reality, enabling users to experience visual and interactive information.

[0010] "Visual information" refers to information that users can receive through their eyes, and is provided in formats that include text, images, and videos.

[0011] "Data processing means" refers to a method or apparatus for processing large amounts of data quickly and efficiently to obtain necessary analysis results or information.

[0012] "Feedback" refers to the opinions and evaluations that users provide regarding a system or experience, and this feedback is used to improve the system. [Brief explanation of the drawing]

[0013] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2, which combines an emotion engine. [Modes for carrying out the invention]

[0014] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0015] First, the terms used in the following description will be explained.

[0016] In the following embodiments, a numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0017] In the following embodiments, a numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0018] In the following embodiments, a numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (eefore.g., hard disks), or magnetic tapes, etc.

[0019] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0020] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0021] [First Embodiment]

[0022] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0023] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0024] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0025] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0026] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0027] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0028] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0029] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0030] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0031] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0032] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0033] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0034] This invention implements a system that provides personalized experiences to users when they visit entertainment facilities. This system is mainly composed of a server, a terminal, and a user, and its embodiments are shown below.

[0035] server:

[0036] The server functions as a central information processing unit, analyzing behavioral data collected from users to generate personalized visit plans. This process takes into account the user's past behavioral history and interests to create the optimal attraction route. The server also manages real-time data on crowd levels and wait times, transmitting this information to the terminals.

[0037] Terminal:

[0038] The terminal functions as a smartphone or AR device, presenting users with visit plans and local information provided by a server. As users move around the facility, the terminal uses virtual reality technology to provide visual information about attractions and animals. For example, when a user sees an attraction through the terminal's camera, it displays a description of that attraction and the current wait time.

[0039] User:

[0040] Users can smoothly enjoy the facility by reviewing the visit plan suggested by the system based on their interests. By operating a terminal, users can receive real-time information and efficiently tour the facility. Furthermore, users can provide feedback after their visit, and the server uses this information to improve the system, thereby increasing the accuracy of suggestions for future visits.

[0041] Specific example:

[0042] For example, if a user prefers adventure-type attractions in an amusement park, the server will take this interest into account and suggest a schedule that includes a roller coaster first, followed by a visit to a haunted house. The device will display the wait time for the roller coaster and suggest other options if the wait is long. Furthermore, when approaching a haunted house via the device's camera, it will provide visual information about its background story and safety guidelines.

[0043] In summary, the present invention personalizes the user's experience when visiting entertainment facilities, enabling a more fulfilling visit.

[0044] The following describes the processing flow.

[0045] Step 1:

[0046] Users launch the application, enter their personal information and interests, and register.

[0047] Step 2:

[0048] The server receives user registration information, stores it in a database, and generates an interest-based profile.

[0049] Step 3:

[0050] The user selects the facility they plan to visit from the application and enters the date.

[0051] Step 4:

[0052] The server collects attraction information, operating hours, and crowd prediction data for the selected facility, and combines this with the user's profile to generate an optimal visit plan.

[0053] Step 5:

[0054] The terminal displays the visit plan received from the server on its screen, visually presenting it to the user.

[0055] Step 6:

[0056] The user reviews the proposed visit plan and makes adjustments as needed.

[0057] Step 7:

[0058] On the day of the visit, the device uses GPS functionality to obtain the user's location information and provides guidance to attractions based on their current location.

[0059] Step 8:

[0060] The server collects real-time wait time data for each attraction and sends it to the terminal.

[0061] Step 9:

[0062] The device uses augmented reality (AR) technology to display visual information about attractions around the user via its camera, showing crowd levels and background information.

[0063] Step 10:

[0064] Users can efficiently navigate attractions and facilities through their devices, enjoying the experience.

[0065] Step 11:

[0066] After visiting, users enter feedback into the application and send their evaluation of the experience to the server.

[0067] Step 12:

[0068] The server analyzes user feedback and adjusts the system to improve the accuracy of future suggestions.

[0069] (Example 1)

[0070] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0071] Traditionally, when visiting entertainment facilities, it was difficult for visitors to plan their visits while fully considering their interests and the expected crowd levels, resulting in an inefficient enjoyment of the facilities. Furthermore, there were insufficient methods for effectively utilizing feedback to improve the visit experience. As a result, it was difficult to provide highly satisfying experiences tailored to individual interests.

[0072] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0073] In this invention, the server includes a storage means for accumulating user behavior history and generating a user profile, an information processing means for generating personalized visit plans using a generated AI model, and a display means for providing visual presentations using virtual reality technology. This makes it possible to provide personalized visit plans based on the user's interests and real-time congestion data.

[0074] "User activity history" refers to information about the user's past activities and visits to attractions within the facility.

[0075] A "user profile" is a collection of attribute information that reflects a user's interests and preferences, and is used to provide personalized services.

[0076] "Memory devices" refer to devices or methods for storing data long-term or short-term, and are responsible for storing user profiles and behavioral history.

[0077] A "generative AI model" is an artificial intelligence platform that analyzes user data and behavioral history to make predictions and suggestions based on specific objectives.

[0078] "Information processing means" refers to technical means for collecting, analyzing, and processing data to generate useful information.

[0079] "Virtual reality technology" is a technology that visually presents a virtual environment generated by a computer.

[0080] "Visual presentation" refers to a method of providing information to users visually, and specifically includes displays using screens or projectors.

[0081] "Real-time data" refers to data that is updated and provided in real time, such as information on congestion levels and waiting times.

[0082] "Data analysis" refers to statistical or computational methods used to extract useful information from collected data.

[0083] "Feedback" refers to the opinions and evaluations that users provide regarding a service or experience, and this information is used to improve the system.

[0084] This invention aims to realize a system that personalizes the user's visit experience in entertainment facilities, providing an efficient and highly satisfying experience. Specific embodiments are described below.

[0085] server:

[0086] The server utilizes a database management system and AI technology to memorize user behavior history and generate profiles. Specifically, it stores user behavior data in databases using MySQL® or PostgreSQL. The generated AI model is built using artificial intelligence libraries such as TENSORFLOW® and PyTorch, which generates personalized visit plans. The server also has the function of collecting and analyzing real-time data on facility congestion and waiting times. This is achieved by real-time data streaming using Apache® Kafka.

[0087] Terminal:

[0088] The device functions as a smartphone or AR device, presenting users with plans and local information. Users can operate applications on the device to check visit plans and real-time information. This application is developed using Java® or Kotlin for Android® devices and Swift for iOS devices. The device uses AR technology to visually present information about each attraction. This utilizes ARCore on Android and ARKit on iOS.

[0089] User:

[0090] Users can efficiently enjoy the facilities by following the visit plan provided by the system. They can use their devices to check detailed information and crowd conditions for attractions, allowing them to make informed decisions. Furthermore, it is expected that users will provide feedback via their devices after their visit, improving the quality of future visit plans.

[0091] Specific example:

[0092] For example, if a user prefers adventure attractions, the server can generate a plan based on this preference, visiting a roller coaster first, followed by a haunted house. The terminal can display the roller coaster's wait time and crowd status in real time and offer alternative suggestions as needed. As the user approaches the haunted house, relevant background stories and safety guidelines are visually provided through the terminal. This plan generation can use a prompt such as, "Suggest a visit plan for a trip to the amusement park. The user likes adventure and wants to avoid crowded places."

[0093] This system allows users to have a highly satisfying browsing experience based on personalized information.

[0094] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0095] Step 1:

[0096] The server collects user activity history data from the database. The input for this process is past facility visit records based on the user ID, and the output is a dataset of activity history. The server uses database queries to extract information about attractions the user has visited in the past and how frequently they were visited.

[0097] Step 2:

[0098] The server generates and stores user profiles using behavioral history. This profile includes information about the user's interests and tendencies. The input is behavioral history data obtained from step 1, and the output is the user's interest profile. The server applies a data analysis algorithm to update this profile to the latest state.

[0099] Step 3:

[0100] The server generates personalized visit plans using a generative AI model. Inputs are user profiles and real-time facility data (e.g., congestion information, wait times, etc.), and output is a personalized visit plan. The generative AI model uses machine learning techniques to perform calculations to provide the optimal attraction order that reflects the user's interests.

[0101] Step 4:

[0102] The server collects real-time data from sensors installed at various points within the facility and transmits it to the terminal. The input for this process is real-time data from IoT sensors, and the output is analyzed congestion status and waiting time information. The server uses stream processing to analyze this data in a timely manner and transfer it to the user's terminal.

[0103] Step 5:

[0104] The terminal displays visit plans and real-time information sent from the server to the user. Inputs are visit plan data and congestion information received from the server, while output is visual information presented to the user. The terminal uses a user interface to display attraction wait times and tourist information.

[0105] Step 6:

[0106] Users provide feedback on their experience at the facility via a terminal after their visit. The input consists of the user's subjective experience, while the output is feedback data sent to the server. Users register their ratings and opinions using the feedback form on the terminal.

[0107] Step 7:

[0108] The server analyzes feedback collected from users and uses it to improve the next visit plan. The input to this process is feedback data, and the output is updated generative AI model parameters. The server performs data analysis and retrains the AI ​​model to improve the accuracy of the suggestions.

[0109] (Application Example 1)

[0110] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0111] Traditional tourist information systems required users to research a large amount of information in advance and select destinations that matched their interests. This presented a problem in that selecting appropriate tourist spots and creating efficient travel plans was difficult, especially in unfamiliar areas. Furthermore, congestion and unexpected waiting times on the day of visit often prevented visitors from sticking to their plans. Improving these conditions and enhancing the tourist experience is crucial.

[0112] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0113] In this invention, the server includes means for analyzing behavioral history and preference data to generate personalized visit routes based on the user's interests, means for providing visual information to the user using augmented reality technology, and means for analyzing congestion and waiting times in real time and providing that information. As a result, users can efficiently visit tourist destinations based on personalized visit plans, and flexible plan changes can be made in response to real-time conditions.

[0114] "Activity history" refers to a record of various actions and movements that a user has taken up to that point.

[0115] "Preference data" refers to data that indicates a user's interests and preferences, and is information collected based on past choices and hobbies.

[0116] A "personalized itinerary" is an optimal route for sightseeing or visiting that is specially created taking into account the user's specific interests and behavioral history.

[0117] Augmented reality technology is a technology that overlays digital information onto the real world, thereby improving the user's visual experience.

[0118] A "means for analyzing congestion and waiting times in real time" refers to a system that enables the immediate analysis of the current level of congestion and waiting times at a location and provides this information to users.

[0119] A "server" is a central information processing device that processes and stores data via a network and provides analysis results to users.

[0120] A "terminal device" is a device that a user can carry and use, and whose role is to receive information from a server and present it to the user.

[0121] "Data improvement methods" refer to data processing techniques that collect feedback from users and contribute to improving system performance and the accuracy of suggestions.

[0122] "Route generation means for dynamically adjusting visit routes between facilities in the tourism sector" refers to a technology that generates and modifies the optimal visit route in real time according to the user's situation and the facility's situation.

[0123] This invention provides a system that personalizes the tourism experience based on the user's preferences and behavioral history. Specifically, the server collects and analyzes past behavioral history and interest data provided by the user. The software used in this process includes Apache Hadoop for managing and analyzing big data, and Scikit-learn for analyzing data using machine learning algorithms. This generates a travel route tailored to the user's specific preferences.

[0124] Simultaneously, the server collects real-time data on congestion levels and waiting times at tourist destinations, and immediately displays this information to users. This allows users to check whether their pre-planned schedule is appropriate for the current situation at the destination.

[0125] The terminal plays a role in providing users with visual information using augmented reality technology. Specifically, the terminal combines the user's location information with information provided by the server to display detailed information and visual guides about facilities that interest the user on the screen. This process utilizes AR devices (for example, Microsoft® HoloLens®) to leverage augmented reality technology. This makes it possible to directly overlay information about visited locations and attractions onto the user's field of view.

[0126] Furthermore, users are required to provide feedback after their visit. This feedback is sent to the server and used by data refinement tools to improve the experience on subsequent visits. Through this iterative process, the AI ​​model continuously learns, improving the accuracy of suggestions and providing a more personalized user experience.

[0127] For example, if a user is interested in history, the server will take that interest into account and suggest routes to historically significant museums and landmarks. Based on this, the terminal can display real-time congestion information and inform tourists of the best time to visit.

[0128] An example of a prompt for a generative AI model might be a command such as, "Analyze the user's history and suggest the optimal visiting route for them using an AR device." This prompt allows the AI ​​to provide personalized sightseeing plans in real time.

[0129] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0130] Step 1:

[0131] The server receives user behavior history and preference data. Input data includes past visits, activities, and categories of interest. Based on this, Apache Hadoop is used for big data management and analysis to generate user preference profiles. The output is an individualized preference profile.

[0132] Step 2:

[0133] The server uses the generated preference profile to create the optimal travel route. Scikit-learn is used to determine the priority of potential destinations. The input is the preference profile and available tourist destination information. The output is a prioritized list of destinations and the travel route.

[0134] Step 3:

[0135] The server collects real-time congestion and wait time data for tourist destinations. This uses real-time data from an external API. The input is the data from the API response, which is analyzed to obtain congestion and wait time information. The output is the latest congestion and wait time information.

[0136] Step 4:

[0137] The terminal receives visit route and congestion information provided by the server and presents it to the user. This process uses AR technology to visually display the information. The terminal's input is data from the server, and its output is the presentation of visual information to the user. Specifically, the terminal activates an AR device (e.g., Microsoft HoloLens) and virtually overlays tourist information onto it.

[0138] Step 5:

[0139] Users follow a suggested itinerary displayed on their device while sightseeing. During the tour, they can directly input their satisfaction level at each tourist spot. This enables real-time feedback. Input consists of user ratings and feedback, while output is their satisfaction data.

[0140] Step 6:

[0141] The server collects feedback from users and uses the data to improve the accuracy of future suggestions. Based on the collected data, the generating AI model is fed into a feedback loop to improve the accuracy of the suggestions. The input is user feedback data, and the output is an updated database and an improved AI model.

[0142] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0143] This invention combines an emotion engine with a system that provides personalized experiences when users visit entertainment facilities. This system is primarily composed of three components: a server, a terminal, and a user.

[0144] server:

[0145] The server, acting as a central data processing unit, collects user behavior data and generates personalized visit plans based on it. Furthermore, the server is equipped with an emotion engine that recognizes emotions from the user's facial expressions and voice data, and incorporates this information into the visit plan to provide more personalized suggestions. The server also analyzes congestion data and waiting times in real time and transmits necessary information to the user's device.

[0146] Terminal:

[0147] The device functions as a smartphone or AR-enabled device, presenting the user with personalized visit plans and local information provided by a server. The device uses the user's location information to provide real-time navigation and, when approaching specific attractions, offers visual information tailored to the user's emotional state. For example, if the user appears to be enjoying themselves, the device will suggest information and attractions that further enhance that enjoyable experience.

[0148] User:

[0149] Users can review suggested visit plans through the application and receive real-time guidance and emotion-appropriate attraction information while moving around the facility. After the visit, users enter feedback into the app, and this data is sent to the server to be used to improve future visit suggestions. Data obtained through emotion recognition is also collected as part of the feedback and used for analysis.

[0150] Specific example:

[0151] For example, if a user is moved while watching a dolphin show at an aquarium, the server will acquire that emotional data and prioritize suggesting animal interaction programs in subsequent visit plans. During the show, the device will provide AR-based information about the dolphins' behavior that the user might find interesting, enabling a deeper experience.

[0152] As described above, the present invention aims to enhance visits to entertainment facilities by providing personalized experiences that take into account the user's emotions.

[0153] The following describes the processing flow.

[0154] Step 1:

[0155] Users create a profile by launching the application, entering personal information, and selecting attractions and themes of interest to register.

[0156] Step 2:

[0157] The server generates a profile in the database based on the user's registration information and selected interests, and creates a personalized visit plan.

[0158] Step 3:

[0159] The terminal displays the visit plan sent from the server on the user's screen, allowing for visual confirmation. The user reviews the proposed plan and makes adjustments as needed.

[0160] Step 4:

[0161] On the day of the visit, the device acquires the user's location information via GPS and sends it to the server. The server calculates the latest congestion and wait time data and updates the device with the optimal attraction route.

[0162] Step 5:

[0163] The device uses AR functionality to visually provide information about attractions or animals specified by the user. An emotion engine recognizes the user's facial expressions and voice, and sends emotion data to a server.

[0164] Step 6:

[0165] The server analyzes emotional data and further adjusts the visit plan based on the user's current emotions. If the user is enjoying themselves, it prioritizes suggesting attractions that offer a similar experience.

[0166] Step 7:

[0167] Users navigate through attractions guided by their devices, receiving real-time information and emotionally adaptive visuals along the way. This allows them to make the most of their experience.

[0168] Step 8:

[0169] After their visit, users access the app to provide feedback on their experience. The emotion data recognized by the emotion engine is also sent to the server.

[0170] Step 9:

[0171] The server analyzes the collected feedback and sentiment data and uses a generative AI algorithm to make adjustments to improve the accuracy of suggestions for the next user visit.

[0172] (Example 2)

[0173] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0174] Modern entertainment facilities are required to provide personalized experiences for their users. However, existing systems only offer generic plans based on behavioral data, making it difficult to provide experiences that reflect users' emotions and real-time circumstances. Furthermore, there is a lack of effective ways to utilize user feedback, which makes continuous system improvement difficult.

[0175] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0176] In this invention, the server includes information processing means for analyzing behavioral and emotional information to generate a personalized action plan based on the user's interests and emotions; display means for providing visual information to the user using augmented reality technology; and data processing means for analyzing and presenting congestion status and waiting times in real time. This enables the provision of a personalized experience tailored to the user's emotions and circumstances, and allows for continuous improvement of the system based on feedback.

[0177] "Behavioral information" refers to data related to the user's actions, such as their location, travel history, and records of attractions they visited.

[0178] "Emotional information" refers to data about the emotional state analyzed from the user's facial expressions and voice.

[0179] An "individualized action plan" refers to a visit plan optimized for each individual user based on behavioral and emotional information.

[0180] "Information processing means" refers to software and hardware configured to collect and analyze behavioral and emotional information.

[0181] Augmented reality technology refers to technologies that integrate digital information into the real world and present it visually to users.

[0182] "Display means" refers to devices and software that utilize augmented reality technology to visually present digital information to users.

[0183] "Data processing means" refers to software or hardware for collecting and analyzing congestion status and waiting times in real time.

[0184] "Feedback" refers to information such as opinions and impressions that users provide after experiencing something.

[0185] "System improvement" refers to analyzing collected feedback and taking corrective or additional actions to improve the quality of the service.

[0186] This invention is a system that provides personalized experiences based on the user's behavior and emotions. This system mainly consists of three components: a server, a terminal, and a user.

[0187] server

[0188] The server functions as a central data processing unit. It collects behavioral and emotional information and uses this data to generate visit plans using an AI model. The collected data is analyzed through an emotion engine, which analyzes emotions from facial expressions and voice, and is used to create plans tailored to the user's experience. The server also analyzes congestion and wait times in real time and provides necessary information to the terminal. For example, if a user shows interest in multiple attractions, it can determine their priority and create an optimized plan.

[0189] terminal

[0190] The device functions as a smartphone or augmented reality device. It visually presents users with personalized visit plans and real-time crowd information transmitted from a server. When a user approaches a specific attraction, the device uses AR technology to provide additional information relevant to that location. For example, while watching a dolphin show at an aquarium, it can display detailed information about the dolphins' behavior in real time, based on the user's emotional state.

[0191] User

[0192] Users can review the suggested visit plan via their device and use it as a guide. After experiencing each attraction, they enter feedback into the application. This feedback data is sent to the server and used to provide an even more improved, personalized experience on their next visit.

[0193] Specific examples and prompt statements

[0194] For example, if a user rides a roller coaster at a theme park and expresses excitement, the server can prioritize suggesting similar thrilling attractions as the next one to visit. An example of a prompt in this case might be, "Please suggest a suitable next attraction after the roller coaster experience."

[0195] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0196] Step 1:

[0197] The server collects user behavior and emotional information. Inputs include user location data, visit history, facial expression data, and voice data. The server acquires this data using sensor technology and recognition software and stores the information in a cloud database. The output at this stage is an analyzable behavior and emotional dataset.

[0198] Step 2:

[0199] The server performs sentiment analysis based on the collected data. The input data is sentiment information obtained in the previous stage. The server uses an emotion engine to analyze facial expressions and voice parameters to determine the user's emotional state (e.g., joy, surprise, boredom, etc.). The output is an individualized sentiment profile. This profile is used to personalize the visit plan.

[0200] Step 3:

[0201] The server generates the next visit plan using a generative AI model. The input consists of existing behavioral data, emotional profiles, and real-time congestion data. Based on this, the generative AI model performs data processing and predictive calculations to output a visit plan optimized for each user. For example, it might suggest thrilling attractions during less crowded times.

[0202] Step 4:

[0203] The terminal displays the visit plan received from the server to the user. The input consists of the generated visit plan data and local information. The terminal displays this visually and provides detailed explanations using AR technology. Output is generated that includes navigation assistance, such as additional information about nearby attractions.

[0204] Step 5:

[0205] Users review the provided visit plan using a terminal and visit attractions according to the instructions. As feedback, users register their impressions and newly acquired interests on the terminal. This feedback data is sent to a server to be used to optimize future visit plans. This feedback output serves as valuable input for the user's next visit.

[0206] Step 6:

[0207] The server analyzes user feedback and continuously collected sentiment data to improve the entire system. Inputs include feedback data, sentiment history, and visit history. Based on this, an information processing program performs analysis, resulting in further improvements to the accuracy of action plans and overall system service quality.

[0208] (Application Example 2)

[0209] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0210] There is a need for systems that allow users to receive not only suggestions based on their interests in entertainment facilities and living spaces, but also more personalized experiences that take their emotions into account. However, conventional technologies have been insufficient in terms of individualization that takes emotions into account, making it difficult to achieve highly satisfying suggestions.

[0211] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0212] In this invention, the server includes information processing means for analyzing behavioral data to generate personalized visit plans based on the user's interests, display means for providing visual information to the user using virtual reality technology, and emotion analysis means for acquiring emotion data and making suggestions based on emotion recognition. This makes it possible to provide a personalized experience that responds to the user's emotions.

[0213] "Behavioral data" refers to information about a user's actions, such as their location and movement history within a facility.

[0214] An "individualized visit plan" refers to a facility visit schedule and suggestions that are personalized and optimized based on the user's interests and feelings.

[0215] "Information processing means" refers to computers and related equipment used to analyze user behavioral data and emotional data, and to generate visit plans based on the results.

[0216] "Virtual reality technology" refers to technology that combines information from the real world with digital information to provide users with visually rich information.

[0217] "Display means" refers to devices and technologies used to visually present information to users.

[0218] "Data processing means for analyzing and presenting congestion status and waiting times in real time" refers to technology that acquires and analyzes information on the flow of people and waiting times within a facility and reports it to users immediately.

[0219] "Data correction means" refers to a function that adjusts system data based on user feedback to improve the accuracy of proposals and visit plans.

[0220] "Emotional analysis methods" refer to technologies that analyze facial expressions and voice data to understand the user's emotions and incorporate the results into the visit plan.

[0221] "Personalization methods" refer to functions that select and provide information and suggestions according to the user's emotions and interests.

[0222] The system for carrying out the present invention consists of three components: a server, a terminal, and a user. The server is the central component that collects user behavioral and emotional data and generates personalized visit plans. The hardware used here is a high-performance data processing server, and the software includes an emotion recognition algorithm (e.g., a cloud-based emotion analysis API).

[0223] The server analyzes behavioral data to generate a visit plan tailored to the user's interests. This process also considers the user's emotional data, and the experience is personalized through corresponding individualization methods. This plan is sent to the device and presented visually to the user. The device functions specifically as a smartphone or AR glasses, receiving data from the server and providing information using virtual reality technology.

[0224] As a concrete example, when a user is with a home robot, the robot acquires the user's facial expressions and voice in real time, sends them to a server, and performs emotion analysis. Based on the resulting data, if it determines that the user is relaxed, it suggests playing relaxing music and optimizes the environment.

[0225] An example of a prompt message could be, "Write a flow for a robot program that analyzes the facial expression of a user returning home in the evening and plays healing music if they appear tired." This makes it possible to provide an optimal experience tailored to the user's emotions.

[0226] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0227] Step 1:

[0228] The server receives location and behavioral data from the user's device. As input, it retrieves data on the user's current location and past behavioral history, and stores it in a database. This prepares the server to identify the user's interests and behavioral patterns.

[0229] Step 2:

[0230] The device uses sensors to acquire the user's facial expressions and voice data, and sends it to the server. Real-time facial and voice information of the user is acquired as input and passed to an emotion recognition algorithm. The server uses a generative AI model to analyze emotions and determine the user's current emotional state. The user's emotion data is obtained as output.

[0231] Step 3:

[0232] The server generates personalized visit plans using a generative AI model based on the acquired behavioral and emotional data. Congestion and waiting time data are also considered during the plan generation process. User behavioral and emotional data are used as input, and a visit plan tailored to the user is created as output.

[0233] Step 4:

[0234] The server sends the generated visit plan to the terminal, which then visually presents it to the user. The terminal uses virtual reality technology to visualize the plan in an easy-to-use way. It receives visit plan data sent from the server as input and generates information that is easy for the user to understand as output.

[0235] Step 5:

[0236] After a user visits the system, feedback is sent from the user's device to the server. The server collects the feedback and uses data correction mechanisms to improve the system. It receives user feedback data as input and generates data necessary for system optimization as output.

[0237] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0238] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0239] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0240] [Second Embodiment]

[0241] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0242] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0243] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0244] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0245] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0246] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0247] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0248] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0249] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0250] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0251] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0252] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0253] This invention implements a system that provides personalized experiences to users when they visit entertainment facilities. This system is mainly composed of a server, a terminal, and a user, and its embodiments are shown below.

[0254] server:

[0255] The server functions as a central information processing unit, analyzing behavioral data collected from users to generate personalized visit plans. This process takes into account the user's past behavioral history and interests to create the optimal attraction route. The server also manages real-time data on crowd levels and wait times, transmitting this information to the terminals.

[0256] Terminal:

[0257] The terminal functions as a smartphone or AR device, presenting users with visit plans and local information provided by a server. As users move around the facility, the terminal uses virtual reality technology to provide visual information about attractions and animals. For example, when a user sees an attraction through the terminal's camera, it displays a description of that attraction and the current wait time.

[0258] User:

[0259] Users can smoothly enjoy the facility by reviewing the visit plan suggested by the system based on their interests. By operating a terminal, users can receive real-time information and efficiently tour the facility. Furthermore, users can provide feedback after their visit, and the server uses this information to improve the system, thereby increasing the accuracy of suggestions for future visits.

[0260] Specific example:

[0261] For example, if a user prefers adventure-type attractions in an amusement park, the server will take this interest into account and suggest a schedule that includes a roller coaster first, followed by a visit to a haunted house. The device will display the wait time for the roller coaster and suggest other options if the wait is long. Furthermore, when approaching a haunted house via the device's camera, it will provide visual information about its background story and safety guidelines.

[0262] In summary, the present invention personalizes the user's experience when visiting entertainment facilities, enabling a more fulfilling visit.

[0263] The following describes the processing flow.

[0264] Step 1:

[0265] Users launch the application, enter their personal information and interests, and register.

[0266] Step 2:

[0267] The server receives user registration information, stores it in a database, and generates an interest-based profile.

[0268] Step 3:

[0269] The user selects the facility they plan to visit from the application and enters the date.

[0270] Step 4:

[0271] The server collects attraction information, operating hours, and crowd prediction data for the selected facility, and combines this with the user's profile to generate an optimal visit plan.

[0272] Step 5:

[0273] The terminal displays the visit plan received from the server on its screen, visually presenting it to the user.

[0274] Step 6:

[0275] The user reviews the proposed visit plan and makes adjustments as needed.

[0276] Step 7:

[0277] On the day of visit, the terminal uses the GPS function to obtain the user's location information and provides guidance on attractions based on the current location.

[0278] Step 8:

[0279] The server collects the waiting time data of each attraction in real time and transmits it to the terminal.

[0280] Step 9:

[0281] The terminal displays the visual information about the attractions around the user through the camera using AR technology and presents the congestion situation and background information.

[0282] Step 10:

[0283] The user efficiently moves around the attractions and facilities through the terminal and enjoys the experience.

[0284] Step 11:

[0285] After the visit, the user inputs feedback into the application and transmits the evaluation of the experience to the server.

[0286] Step 12:

[0287] The server analyzes the feedback from the user and adjusts the system to improve the accuracy of the next recommendation.

[0288] (Example 1)

[0289] Next, Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0290] Traditionally, when visiting entertainment facilities, it was difficult for visitors to plan their visits while fully considering their interests and the expected crowd levels, resulting in an inefficient enjoyment of the facilities. Furthermore, there were insufficient methods for effectively utilizing feedback to improve the visit experience. As a result, it was difficult to provide highly satisfying experiences tailored to individual interests.

[0291] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0292] In this invention, the server includes a storage means for accumulating user behavior history and generating a user profile, an information processing means for generating personalized visit plans using a generated AI model, and a display means for providing visual presentations using virtual reality technology. This makes it possible to provide personalized visit plans based on the user's interests and real-time congestion data.

[0293] "User activity history" refers to information about the user's past activities and visits to attractions within the facility.

[0294] A "user profile" is a collection of attribute information that reflects a user's interests and preferences, and is used to provide personalized services.

[0295] "Memory devices" refer to devices or methods for storing data long-term or short-term, and are responsible for storing user profiles and behavioral history.

[0296] A "generative AI model" is an artificial intelligence platform that analyzes user data and behavioral history to make predictions and suggestions based on specific objectives.

[0297] "Information processing means" refers to technical means for collecting, analyzing, and processing data to generate useful information.

[0298] "Virtual reality technology" is a technology that visually presents a virtual environment generated by a computer.

[0299] "Visual presentation" refers to a method of providing information to users visually, and specifically includes displays using screens or projectors.

[0300] "Real-time data" refers to data that is updated and provided in real time, such as information on congestion levels and waiting times.

[0301] "Data analysis" refers to statistical or computational methods used to extract useful information from collected data.

[0302] "Feedback" refers to the opinions and evaluations that users provide regarding a service or experience, and this information is used to improve the system.

[0303] This invention aims to realize a system that personalizes the user's visit experience in entertainment facilities, providing an efficient and highly satisfying experience. Specific embodiments are described below.

[0304] server:

[0305] The server utilizes a database management system and AI technology to memorize user behavior history and generate profiles. Specifically, it stores user behavior data in databases using MySQL or PostgreSQL. The generated AI model is built using artificial intelligence libraries such as TensorFlow and PyTorch, which generates personalized visit plans. The server also has the ability to collect and analyze real-time data on facility congestion and waiting times. This is achieved by using Apache Kafka to enable real-time data streaming.

[0306] Terminal:

[0307] The terminal functions as a smartphone or an AR device, presenting plans and local information to the user. The user can operate applications on the terminal to view visit plans and real-time information. This application is developed using Java or Kotlin for Android terminals and Swift for iOS terminals. The terminal uses AR technology to visually present information about each attraction. This utilizes ARCore for Android and ARKit for iOS.

[0308] User:

[0309] The user can efficiently enjoy the visit by touring the facilities according to the visit plan presented by the system. The user can use the terminal to check detailed information and congestion status of attractions and make appropriate judgments. Also, it is expected that the user will provide feedback through the terminal after the visit to improve the quality of future visit plans.

[0310] Specific example:

[0311] For example, if the user prefers adventure attractions, the server generates a plan to visit the roller coaster first and then the haunted house based on this preference. The terminal can display the waiting time and congestion status of the roller coaster in real time and make alternative proposals if necessary. When the user approaches the haunted house, relevant background stories and safety guidelines are visually provided through the terminal. For this plan generation, a prompt sentence such as "Propose a visit plan when going to the amusement park. The user wants to avoid crowded places where there is adventure." can be used.

[0312] With this system, the user can obtain a highly satisfactory visit experience based on personalized information.

[0313] The flow of specific processing in Example 1 will be described using FIG. 11.

[0314] Step 1:

[0315] The server collects user activity history data from the database. The input for this process is past facility visit records based on the user ID, and the output is a dataset of activity history. The server uses database queries to extract information about attractions the user has visited in the past and how frequently they were visited.

[0316] Step 2:

[0317] The server generates and stores user profiles using behavioral history. This profile includes information about the user's interests and tendencies. The input is behavioral history data obtained from step 1, and the output is the user's interest profile. The server applies a data analysis algorithm to update this profile to the latest state.

[0318] Step 3:

[0319] The server generates personalized visit plans using a generative AI model. Inputs are user profiles and real-time facility data (e.g., congestion information, wait times, etc.), and output is a personalized visit plan. The generative AI model uses machine learning techniques to perform calculations to provide the optimal attraction order that reflects the user's interests.

[0320] Step 4:

[0321] The server collects real-time data from sensors installed at various points within the facility and transmits it to the terminal. The input for this process is real-time data from IoT sensors, and the output is analyzed congestion status and waiting time information. The server uses stream processing to analyze this data in a timely manner and transfer it to the user's terminal.

[0322] Step 5:

[0323] The terminal displays visit plans and real-time information sent from the server to the user. Inputs are visit plan data and congestion information received from the server, while output is visual information presented to the user. The terminal uses a user interface to display attraction wait times and tourist information.

[0324] Step 6:

[0325] Users provide feedback on their experience at the facility via a terminal after their visit. The input consists of the user's subjective experience, while the output is feedback data sent to the server. Users register their ratings and opinions using the feedback form on the terminal.

[0326] Step 7:

[0327] The server analyzes feedback collected from users and uses it to improve the next visit plan. The input to this process is feedback data, and the output is updated generative AI model parameters. The server performs data analysis and retrains the AI ​​model to improve the accuracy of the suggestions.

[0328] (Application Example 1)

[0329] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0330] Traditional tourist information systems required users to research a large amount of information in advance and select destinations that matched their interests. This presented a problem in that selecting appropriate tourist spots and creating efficient travel plans was difficult, especially in unfamiliar areas. Furthermore, congestion and unexpected waiting times on the day of visit often prevented visitors from sticking to their plans. Improving these conditions and enhancing the tourist experience is crucial.

[0331] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0332] In this invention, the server includes means for analyzing behavioral history and preference data to generate personalized visit routes based on the user's interests, means for providing visual information to the user using augmented reality technology, and means for analyzing congestion and waiting times in real time and providing that information. As a result, users can efficiently visit tourist destinations based on personalized visit plans, and flexible plan changes can be made in response to real-time conditions.

[0333] "Activity history" refers to a record of various actions and movements that a user has taken up to that point.

[0334] "Preference data" refers to data that indicates a user's interests and preferences, and is information collected based on past choices and hobbies.

[0335] A "personalized itinerary" is an optimal route for sightseeing or visiting that is specially created taking into account the user's specific interests and behavioral history.

[0336] Augmented reality technology is a technology that overlays digital information onto the real world, thereby improving the user's visual experience.

[0337] A "means for analyzing congestion and waiting times in real time" refers to a system that enables the immediate analysis of the current level of congestion and waiting times at a location and provides this information to users.

[0338] A "server" is a central information processing device that processes and stores data via a network and provides analysis results to users.

[0339] A "terminal device" is a device that a user can carry and use, and whose role is to receive information from a server and present it to the user.

[0340] "Data improvement methods" refer to data processing techniques that collect feedback from users and contribute to improving system performance and the accuracy of suggestions.

[0341] "Route generation means for dynamically adjusting visit routes between facilities in the tourism sector" refers to a technology that generates and modifies the optimal visit route in real time according to the user's situation and the facility's situation.

[0342] This invention provides a system that personalizes the tourism experience based on the user's preferences and behavioral history. Specifically, the server collects and analyzes past behavioral history and interest data provided by the user. The software used in this process includes Apache Hadoop for managing and analyzing big data, and Scikit-learn for analyzing data using machine learning algorithms. This generates a travel route tailored to the user's specific preferences.

[0343] Simultaneously, the server collects real-time data on congestion levels and waiting times at tourist destinations, and immediately displays this information to users. This allows users to check whether their pre-planned schedule is appropriate for the current situation at the destination.

[0344] The terminal plays a role in providing users with visual information using augmented reality technology. Specifically, the terminal combines the user's location information with information provided by the server to display detailed information and visual guides about facilities that interest the user on the screen. This process utilizes AR devices (such as Microsoft HoloLens) to leverage augmented reality technology. This makes it possible to directly overlay information about visited locations and attractions onto the user's field of view.

[0345] Furthermore, users are required to provide feedback after their visit. This feedback is sent to the server and used by data refinement tools to improve the experience on subsequent visits. Through this iterative process, the AI ​​model continuously learns, improving the accuracy of suggestions and providing a more personalized user experience.

[0346] For example, if a user is interested in history, the server will take that interest into account and suggest routes to historically significant museums and landmarks. Based on this, the terminal can display real-time congestion information and inform tourists of the best time to visit.

[0347] An example of a prompt for a generative AI model might be a command such as, "Analyze the user's history and suggest the optimal visiting route for them using an AR device." This prompt allows the AI ​​to provide personalized sightseeing plans in real time.

[0348] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0349] Step 1:

[0350] The server receives user behavior history and preference data. Input data includes past visits, activities, and categories of interest. Based on this, Apache Hadoop is used for big data management and analysis to generate user preference profiles. The output is an individualized preference profile.

[0351] Step 2:

[0352] The server uses the generated preference profile to create the optimal travel route. Scikit-learn is used to determine the priority of potential destinations. The input is the preference profile and available tourist destination information. The output is a prioritized list of destinations and the travel route.

[0353] Step 3:

[0354] The server collects real-time congestion and wait time data for tourist destinations. This uses real-time data from an external API. The input is the data from the API response, which is analyzed to obtain congestion and wait time information. The output is the latest congestion and wait time information.

[0355] Step 4:

[0356] The terminal receives visit route and congestion information provided by the server and presents it to the user. This process uses AR technology to visually display the information. The terminal's input is data from the server, and its output is the presentation of visual information to the user. Specifically, the terminal activates an AR device (e.g., Microsoft HoloLens) and virtually overlays tourist information onto it.

[0357] Step 5:

[0358] Users follow a suggested itinerary displayed on their device while sightseeing. During the tour, they can directly input their satisfaction level at each tourist spot. This enables real-time feedback. Input consists of user ratings and feedback, while output is their satisfaction data.

[0359] Step 6:

[0360] The server collects feedback from users and uses the data to improve the accuracy of future suggestions. Based on the collected data, the generating AI model is fed into a feedback loop to improve the accuracy of the suggestions. The input is user feedback data, and the output is an updated database and an improved AI model.

[0361] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0362] This invention combines an emotion engine with a system that provides personalized experiences when users visit entertainment facilities. This system is primarily composed of three components: a server, a terminal, and a user.

[0363] server:

[0364] The server, acting as a central data processing unit, collects user behavior data and generates personalized visit plans based on it. Furthermore, the server is equipped with an emotion engine that recognizes emotions from the user's facial expressions and voice data, and incorporates this information into the visit plan to provide more personalized suggestions. The server also analyzes congestion data and waiting times in real time and transmits necessary information to the user's device.

[0365] Terminal:

[0366] The device functions as a smartphone or AR-enabled device, presenting the user with personalized visit plans and local information provided by a server. The device uses the user's location information to provide real-time navigation and, when approaching specific attractions, offers visual information tailored to the user's emotional state. For example, if the user appears to be enjoying themselves, the device will suggest information and attractions that further enhance that enjoyable experience.

[0367] User:

[0368] Users can review suggested visit plans through the application and receive real-time guidance and emotion-appropriate attraction information while moving around the facility. After the visit, users enter feedback into the app, and this data is sent to the server to be used to improve future visit suggestions. Data obtained through emotion recognition is also collected as part of the feedback and used for analysis.

[0369] Specific example:

[0370] For example, if a user is moved while watching a dolphin show at an aquarium, the server will acquire that emotional data and prioritize suggesting animal interaction programs in subsequent visit plans. During the show, the device will provide AR-based information about the dolphins' behavior that the user might find interesting, enabling a deeper experience.

[0371] As described above, the present invention aims to enhance visits to entertainment facilities by providing personalized experiences that take into account the user's emotions.

[0372] The following describes the processing flow.

[0373] Step 1:

[0374] Users create a profile by launching the application, entering personal information, and selecting attractions and themes of interest to register.

[0375] Step 2:

[0376] The server generates a profile in the database based on the user's registration information and selected interests, and creates a personalized visit plan.

[0377] Step 3:

[0378] The terminal displays the visit plan sent from the server on the user's screen, allowing for visual confirmation. The user reviews the proposed plan and makes adjustments as needed.

[0379] Step 4:

[0380] On the day of the visit, the device acquires the user's location information via GPS and sends it to the server. The server calculates the latest congestion and wait time data and updates the device with the optimal attraction route.

[0381] Step 5:

[0382] The device uses AR functionality to visually provide information about attractions or animals specified by the user. An emotion engine recognizes the user's facial expressions and voice, and sends emotion data to a server.

[0383] Step 6:

[0384] The server analyzes emotional data and further adjusts the visit plan based on the user's current emotions. If the user is enjoying themselves, it prioritizes suggesting attractions that offer a similar experience.

[0385] Step 7:

[0386] Users navigate through attractions guided by their devices, receiving real-time information and emotionally adaptive visuals along the way. This allows them to make the most of their experience.

[0387] Step 8:

[0388] After their visit, users access the app to provide feedback on their experience. The emotion data recognized by the emotion engine is also sent to the server.

[0389] Step 9:

[0390] The server analyzes the collected feedback and sentiment data and uses a generative AI algorithm to make adjustments to improve the accuracy of suggestions for the next user visit.

[0391] (Example 2)

[0392] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0393] Modern entertainment facilities are required to provide personalized experiences for their users. However, existing systems only offer generic plans based on behavioral data, making it difficult to provide experiences that reflect users' emotions and real-time circumstances. Furthermore, there is a lack of effective ways to utilize user feedback, which makes continuous system improvement difficult.

[0394] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0395] In this invention, the server includes information processing means for analyzing behavioral and emotional information to generate a personalized action plan based on the user's interests and emotions; display means for providing visual information to the user using augmented reality technology; and data processing means for analyzing and presenting congestion status and waiting times in real time. This enables the provision of a personalized experience tailored to the user's emotions and circumstances, and allows for continuous improvement of the system based on feedback.

[0396] "Behavioral information" refers to data related to the user's actions, such as their location, travel history, and records of attractions they visited.

[0397] "Emotional information" refers to data about the emotional state analyzed from the user's facial expressions and voice.

[0398] An "individualized action plan" refers to a visit plan optimized for each individual user based on behavioral and emotional information.

[0399] "Information processing means" refers to software and hardware configured to collect and analyze behavioral and emotional information.

[0400] Augmented reality technology refers to technologies that integrate digital information into the real world and present it visually to users.

[0401] "Display means" refers to devices and software that utilize augmented reality technology to visually present digital information to users.

[0402] "Data processing means" refers to software or hardware for collecting and analyzing congestion status and waiting times in real time.

[0403] "Feedback" refers to information such as opinions and impressions that users provide after experiencing something.

[0404] "System improvement" refers to analyzing collected feedback and taking corrective or additional actions to improve the quality of the service.

[0405] This invention is a system that provides personalized experiences based on the user's behavior and emotions. This system mainly consists of three components: a server, a terminal, and a user.

[0406] server

[0407] The server functions as a central data processing unit. It collects behavioral and emotional information and uses this data to generate visit plans using an AI model. The collected data is analyzed through an emotion engine, which analyzes emotions from facial expressions and voice, and is used to create plans tailored to the user's experience. The server also analyzes congestion and wait times in real time and provides necessary information to the terminal. For example, if a user shows interest in multiple attractions, it can determine their priority and create an optimized plan.

[0408] terminal

[0409] The device functions as a smartphone or augmented reality device. It visually presents users with personalized visit plans and real-time crowd information transmitted from a server. When a user approaches a specific attraction, the device uses AR technology to provide additional information relevant to that location. For example, while watching a dolphin show at an aquarium, it can display detailed information about the dolphins' behavior in real time, based on the user's emotional state.

[0410] User

[0411] Users can review the suggested visit plan via their device and use it as a guide. After experiencing each attraction, they enter feedback into the application. This feedback data is sent to the server and used to provide an even more improved, personalized experience on their next visit.

[0412] Specific examples and prompt statements

[0413] For example, if a user rides a roller coaster at a theme park and expresses excitement, the server can prioritize suggesting similar thrilling attractions as the next one to visit. An example of a prompt in this case might be, "Please suggest a suitable next attraction after the roller coaster experience."

[0414] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0415] Step 1:

[0416] The server collects user behavior and emotional information. Inputs include user location data, visit history, facial expression data, and voice data. The server acquires this data using sensor technology and recognition software and stores the information in a cloud database. The output at this stage is an analyzable behavior and emotional dataset.

[0417] Step 2:

[0418] The server performs sentiment analysis based on the collected data. The input data is sentiment information obtained in the previous stage. The server uses an emotion engine to analyze facial expressions and voice parameters to determine the user's emotional state (e.g., joy, surprise, boredom, etc.). The output is an individualized sentiment profile. This profile is used to personalize the visit plan.

[0419] Step 3:

[0420] The server generates the next visit plan using a generative AI model. The input consists of existing behavioral data, emotional profiles, and real-time congestion data. Based on this, the generative AI model performs data processing and predictive calculations to output a visit plan optimized for each user. For example, it might suggest thrilling attractions during less crowded times.

[0421] Step 4:

[0422] The terminal displays the visit plan received from the server to the user. The input consists of the generated visit plan data and local information. The terminal displays this visually and provides detailed explanations using AR technology. Output is generated that includes navigation assistance, such as additional information about nearby attractions.

[0423] Step 5:

[0424] Users review the provided visit plan using a terminal and visit attractions according to the instructions. As feedback, users register their impressions and newly acquired interests on the terminal. This feedback data is sent to a server to be used to optimize future visit plans. This feedback output serves as valuable input for the user's next visit.

[0425] Step 6:

[0426] The server analyzes user feedback and continuously collected sentiment data to improve the entire system. Inputs include feedback data, sentiment history, and visit history. Based on this, an information processing program performs analysis, resulting in further improvements to the accuracy of action plans and overall system service quality.

[0427] (Application Example 2)

[0428] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 as the "terminal".

[0429] There is a need for systems that allow users to receive not only suggestions based on their interests in entertainment facilities and living spaces, but also more personalized experiences that take their emotions into account. However, conventional technologies have been insufficient in terms of individualization that takes emotions into account, making it difficult to achieve highly satisfying suggestions.

[0430] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0431] In this invention, the server includes information processing means for analyzing behavioral data to generate personalized visit plans based on the user's interests, display means for providing visual information to the user using virtual reality technology, and emotion analysis means for acquiring emotion data and making suggestions based on emotion recognition. This makes it possible to provide a personalized experience that responds to the user's emotions.

[0432] "Behavioral data" refers to information about a user's actions, such as their location and movement history within a facility.

[0433] An "individualized visit plan" refers to a facility visit schedule and suggestions that are personalized and optimized based on the user's interests and feelings.

[0434] "Information processing means" refers to computers and related equipment used to analyze user behavioral data and emotional data, and to generate visit plans based on the results.

[0435] "Virtual reality technology" refers to technology that combines information from the real world with digital information to provide users with visually rich information.

[0436] "Display means" refers to devices and technologies used to visually present information to users.

[0437] "Data processing means for analyzing and presenting congestion status and waiting times in real time" refers to technology that acquires and analyzes information on the flow of people and waiting times within a facility and reports it to users immediately.

[0438] "Data correction means" refers to a function that adjusts system data based on user feedback to improve the accuracy of proposals and visit plans.

[0439] "Emotional analysis methods" refer to technologies that analyze facial expressions and voice data to understand the user's emotions and incorporate the results into the visit plan.

[0440] "Personalization methods" refer to functions that select and provide information and suggestions according to the user's emotions and interests.

[0441] The system for carrying out the present invention consists of three components: a server, a terminal, and a user. The server is the central component that collects user behavioral and emotional data and generates personalized visit plans. The hardware used here is a high-performance data processing server, and the software includes an emotion recognition algorithm (e.g., a cloud-based emotion analysis API).

[0442] The server analyzes behavioral data to generate a visit plan tailored to the user's interests. This process also considers the user's emotional data, and the experience is personalized through corresponding individualization methods. This plan is sent to the device and presented visually to the user. The device functions specifically as a smartphone or AR glasses, receiving data from the server and providing information using virtual reality technology.

[0443] As a concrete example, when a user is with a home robot, the robot acquires the user's facial expressions and voice in real time, sends them to a server, and performs emotion analysis. Based on the resulting data, if it determines that the user is relaxed, it suggests playing relaxing music and optimizes the environment.

[0444] An example of a prompt message could be, "Write a flow for a robot program that analyzes the facial expression of a user returning home in the evening and plays healing music if they appear tired." This makes it possible to provide an optimal experience tailored to the user's emotions.

[0445] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0446] Step 1:

[0447] The server receives location and behavioral data from the user's device. As input, it retrieves data on the user's current location and past behavioral history, and stores it in a database. This prepares the server to identify the user's interests and behavioral patterns.

[0448] Step 2:

[0449] The device uses sensors to acquire the user's facial expressions and voice data, and sends it to the server. Real-time facial and voice information of the user is acquired as input and passed to an emotion recognition algorithm. The server uses a generative AI model to analyze emotions and determine the user's current emotional state. The user's emotion data is obtained as output.

[0450] Step 3:

[0451] The server generates personalized visit plans using a generative AI model based on the acquired behavioral and emotional data. Congestion and waiting time data are also considered during the plan generation process. User behavioral and emotional data are used as input, and a visit plan tailored to the user is created as output.

[0452] Step 4:

[0453] The server sends the generated visit plan to the terminal, which then visually presents it to the user. The terminal uses virtual reality technology to visualize the plan in an easy-to-use way. It receives visit plan data sent from the server as input and generates information that is easy for the user to understand as output.

[0454] Step 5:

[0455] After a user visits the system, feedback is sent from the user's device to the server. The server collects the feedback and uses data correction mechanisms to improve the system. It receives user feedback data as input and generates data necessary for system optimization as output.

[0456] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0457] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0458] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0459] [Third Embodiment]

[0460] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0461] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0462] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0463] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0464] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0465] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0466] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0467] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0468] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0469] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0470] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0471] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0472] This invention implements a system that provides personalized experiences to users when they visit entertainment facilities. This system is mainly composed of a server, a terminal, and a user, and its embodiments are shown below.

[0473] server:

[0474] The server functions as a central information processing unit, analyzing behavioral data collected from users to generate personalized visit plans. This process takes into account the user's past behavioral history and interests to create the optimal attraction route. The server also manages real-time data on crowd levels and wait times, transmitting this information to the terminals.

[0475] Terminal:

[0476] The terminal functions as a smartphone or AR device, presenting users with visit plans and local information provided by a server. As users move around the facility, the terminal uses virtual reality technology to provide visual information about attractions and animals. For example, when a user sees an attraction through the terminal's camera, it displays a description of that attraction and the current wait time.

[0477] User:

[0478] Users can smoothly enjoy the facility by reviewing the visit plan suggested by the system based on their interests. By operating a terminal, users can receive real-time information and efficiently tour the facility. Furthermore, users can provide feedback after their visit, and the server uses this information to improve the system, thereby increasing the accuracy of suggestions for future visits.

[0479] Specific example:

[0480] For example, if a user prefers adventure-type attractions in an amusement park, the server will take this interest into account and suggest a schedule that includes a roller coaster first, followed by a visit to a haunted house. The device will display the wait time for the roller coaster and suggest other options if the wait is long. Furthermore, when approaching a haunted house via the device's camera, it will provide visual information about its background story and safety guidelines.

[0481] In summary, the present invention personalizes the user's experience when visiting entertainment facilities, enabling a more fulfilling visit.

[0482] The following describes the processing flow.

[0483] Step 1:

[0484] Users launch the application, enter their personal information and interests, and register.

[0485] Step 2:

[0486] The server receives user registration information, stores it in a database, and generates an interest-based profile.

[0487] Step 3:

[0488] The user selects the facility they plan to visit from the application and enters the date.

[0489] Step 4:

[0490] The server collects attraction information, operating hours, and crowd prediction data for the selected facility, and combines this with the user's profile to generate an optimal visit plan.

[0491] Step 5:

[0492] The terminal displays the visit plan received from the server on its screen, visually presenting it to the user.

[0493] Step 6:

[0494] The user reviews the proposed visit plan and makes adjustments as needed.

[0495] Step 7:

[0496] On the day of the visit, the device uses GPS functionality to obtain the user's location information and provides guidance to attractions based on their current location.

[0497] Step 8:

[0498] The server collects real-time wait time data for each attraction and sends it to the terminal.

[0499] Step 9:

[0500] The device uses augmented reality (AR) technology to display visual information about attractions around the user via its camera, showing crowd levels and background information.

[0501] Step 10:

[0502] Users can efficiently navigate attractions and facilities through their devices, enjoying the experience.

[0503] Step 11:

[0504] After visiting, users enter feedback into the application and send their evaluation of the experience to the server.

[0505] Step 12:

[0506] The server analyzes user feedback and adjusts the system to improve the accuracy of future suggestions.

[0507] (Example 1)

[0508] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0509] Traditionally, when visiting entertainment facilities, it was difficult for visitors to plan their visits while fully considering their interests and the expected crowd levels, resulting in an inefficient enjoyment of the facilities. Furthermore, there were insufficient methods for effectively utilizing feedback to improve the visit experience. As a result, it was difficult to provide highly satisfying experiences tailored to individual interests.

[0510] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0511] In this invention, the server includes a storage means for accumulating user behavior history and generating a user profile, an information processing means for generating personalized visit plans using a generated AI model, and a display means for providing visual presentations using virtual reality technology. This makes it possible to provide personalized visit plans based on the user's interests and real-time congestion data.

[0512] "User activity history" refers to information about the user's past activities and visits to attractions within the facility.

[0513] A "user profile" is a collection of attribute information that reflects a user's interests and preferences, and is used to provide personalized services.

[0514] "Memory devices" refer to devices or methods for storing data long-term or short-term, and are responsible for storing user profiles and behavioral history.

[0515] A "generative AI model" is an artificial intelligence platform that analyzes user data and behavioral history to make predictions and suggestions based on specific objectives.

[0516] "Information processing means" refers to technical means for collecting, analyzing, and processing data to generate useful information.

[0517] "Virtual reality technology" is a technology that visually presents a virtual environment generated by a computer.

[0518] "Visual presentation" refers to a method of providing information to users visually, and specifically includes displays using screens or projectors.

[0519] "Real-time data" refers to data that is updated and provided in real time, such as information on congestion levels and waiting times.

[0520] "Data analysis" refers to statistical or computational methods used to extract useful information from collected data.

[0521] "Feedback" refers to the opinions and evaluations that users provide regarding a service or experience, and this information is used to improve the system.

[0522] This invention aims to realize a system that personalizes the user's visit experience in entertainment facilities, providing an efficient and highly satisfying experience. Specific embodiments are described below.

[0523] server:

[0524] The server utilizes a database management system and AI technology to memorize user behavior history and generate profiles. Specifically, it stores user behavior data in databases using MySQL or PostgreSQL. The generated AI model is built using artificial intelligence libraries such as TensorFlow and PyTorch, which generates personalized visit plans. The server also has the ability to collect and analyze real-time data on facility congestion and waiting times. This is achieved by using Apache Kafka to enable real-time data streaming.

[0525] Terminal:

[0526] The device functions as a smartphone or AR device, presenting users with plans and local information. Users can operate applications on the device to check visit plans and real-time information. This application is developed using Java or Kotlin for Android devices and Swift for iOS devices. The device uses AR technology to visually present information about each attraction. This utilizes ARCore for Android and ARKit for iOS.

[0527] User:

[0528] Users can efficiently enjoy the facilities by following the visit plan provided by the system. They can use their devices to check detailed information and crowd conditions for attractions, allowing them to make informed decisions. Furthermore, it is expected that users will provide feedback via their devices after their visit, improving the quality of future visit plans.

[0529] Specific example:

[0530] For example, if a user prefers adventure attractions, the server can generate a plan based on this preference, visiting a roller coaster first, followed by a haunted house. The terminal can display the roller coaster's wait time and crowd status in real time and offer alternative suggestions as needed. As the user approaches the haunted house, relevant background stories and safety guidelines are visually provided through the terminal. This plan generation can use a prompt such as, "Suggest a visit plan for a trip to the amusement park. The user likes adventure and wants to avoid crowded places."

[0531] This system allows users to have a highly satisfying browsing experience based on personalized information.

[0532] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0533] Step 1:

[0534] The server collects user activity history data from the database. The input for this process is past facility visit records based on the user ID, and the output is a dataset of activity history. The server uses database queries to extract information about attractions the user has visited in the past and how frequently they were visited.

[0535] Step 2:

[0536] The server generates and stores user profiles using behavioral history. This profile includes information about the user's interests and tendencies. The input is behavioral history data obtained from step 1, and the output is the user's interest profile. The server applies a data analysis algorithm to update this profile to the latest state.

[0537] Step 3:

[0538] The server generates personalized visit plans using a generative AI model. Inputs are user profiles and real-time facility data (e.g., congestion information, wait times, etc.), and output is a personalized visit plan. The generative AI model uses machine learning techniques to perform calculations to provide the optimal attraction order that reflects the user's interests.

[0539] Step 4:

[0540] The server collects real-time data from sensors installed at various points within the facility and transmits it to the terminal. The input for this process is real-time data from IoT sensors, and the output is analyzed congestion status and waiting time information. The server uses stream processing to analyze this data in a timely manner and transfer it to the user's terminal.

[0541] Step 5:

[0542] The terminal displays visit plans and real-time information sent from the server to the user. Inputs are visit plan data and congestion information received from the server, while output is visual information presented to the user. The terminal uses a user interface to display attraction wait times and tourist information.

[0543] Step 6:

[0544] Users provide feedback on their experience at the facility via a terminal after their visit. The input consists of the user's subjective experience, while the output is feedback data sent to the server. Users register their ratings and opinions using the feedback form on the terminal.

[0545] Step 7:

[0546] The server analyzes feedback collected from users and uses it to improve the next visit plan. The input to this process is feedback data, and the output is updated generative AI model parameters. The server performs data analysis and retrains the AI ​​model to improve the accuracy of the suggestions.

[0547] (Application Example 1)

[0548] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0549] Traditional tourist information systems required users to research a large amount of information in advance and select destinations that matched their interests. This presented a problem in that selecting appropriate tourist spots and creating efficient travel plans was difficult, especially in unfamiliar areas. Furthermore, congestion and unexpected waiting times on the day of visit often prevented visitors from sticking to their plans. Improving these conditions and enhancing the tourist experience is crucial.

[0550] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0551] In this invention, the server includes means for analyzing behavioral history and preference data to generate personalized visit routes based on the user's interests, means for providing visual information to the user using augmented reality technology, and means for analyzing congestion and waiting times in real time and providing that information. As a result, users can efficiently visit tourist destinations based on personalized visit plans, and flexible plan changes can be made in response to real-time conditions.

[0552] "Activity history" refers to a record of various actions and movements that a user has taken up to that point.

[0553] "Preference data" refers to data that indicates a user's interests and preferences, and is information collected based on past choices and hobbies.

[0554] A "personalized itinerary" is an optimal route for sightseeing or visiting that is specially created taking into account the user's specific interests and behavioral history.

[0555] Augmented reality technology is a technology that overlays digital information onto the real world, thereby improving the user's visual experience.

[0556] A "means for analyzing congestion and waiting times in real time" refers to a system that enables the immediate analysis of the current level of congestion and waiting times at a location and provides this information to users.

[0557] A "server" is a central information processing device that processes and stores data via a network and provides analysis results to users.

[0558] A "terminal device" is a device that a user can carry and use, and whose role is to receive information from a server and present it to the user.

[0559] "Data improvement methods" refer to data processing techniques that collect feedback from users and contribute to improving system performance and the accuracy of suggestions.

[0560] "Route generation means for dynamically adjusting visit routes between facilities in the tourism sector" refers to a technology that generates and modifies the optimal visit route in real time according to the user's situation and the facility's situation.

[0561] This invention provides a system that personalizes the tourism experience based on the user's preferences and behavioral history. Specifically, the server collects and analyzes past behavioral history and interest data provided by the user. The software used in this process includes Apache Hadoop for managing and analyzing big data, and Scikit-learn for analyzing data using machine learning algorithms. This generates a travel route tailored to the user's specific preferences.

[0562] Simultaneously, the server collects real-time data on congestion levels and waiting times at tourist destinations, and immediately displays this information to users. This allows users to check whether their pre-planned schedule is appropriate for the current situation at the destination.

[0563] The terminal plays a role in providing users with visual information using augmented reality technology. Specifically, the terminal combines the user's location information with information provided by the server to display detailed information and visual guides about facilities that interest the user on the screen. This process utilizes AR devices (such as Microsoft HoloLens) to leverage augmented reality technology. This makes it possible to directly overlay information about visited locations and attractions onto the user's field of view.

[0564] Furthermore, users are required to provide feedback after their visit. This feedback is sent to the server and used by data refinement tools to improve the experience on subsequent visits. Through this iterative process, the AI ​​model continuously learns, improving the accuracy of suggestions and providing a more personalized user experience.

[0565] For example, if a user is interested in history, the server will take that interest into account and suggest routes to historically significant museums and landmarks. Based on this, the terminal can display real-time congestion information and inform tourists of the best time to visit.

[0566] An example of a prompt for a generative AI model might be a command such as, "Analyze the user's history and suggest the optimal visiting route for them using an AR device." This prompt allows the AI ​​to provide personalized sightseeing plans in real time.

[0567] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0568] Step 1:

[0569] The server receives user behavior history and preference data. Input data includes past visits, activities, and categories of interest. Based on this, Apache Hadoop is used for big data management and analysis to generate user preference profiles. The output is an individualized preference profile.

[0570] Step 2:

[0571] The server uses the generated preference profile to create the optimal travel route. Scikit-learn is used to determine the priority of potential destinations. The input is the preference profile and available tourist destination information. The output is a prioritized list of destinations and the travel route.

[0572] Step 3:

[0573] The server collects real-time congestion and wait time data for tourist destinations. This uses real-time data from an external API. The input is the data from the API response, which is analyzed to obtain congestion and wait time information. The output is the latest congestion and wait time information.

[0574] Step 4:

[0575] The terminal receives visit route and congestion information provided by the server and presents it to the user. This process uses AR technology to visually display the information. The terminal's input is data from the server, and its output is the presentation of visual information to the user. Specifically, the terminal activates an AR device (e.g., Microsoft HoloLens) and virtually overlays tourist information onto it.

[0576] Step 5:

[0577] Users follow a suggested itinerary displayed on their device while sightseeing. During the tour, they can directly input their satisfaction level at each tourist spot. This enables real-time feedback. Input consists of user ratings and feedback, while output is their satisfaction data.

[0578] Step 6:

[0579] The server collects feedback from users and uses the data to improve the accuracy of future suggestions. Based on the collected data, the generating AI model is fed into a feedback loop to improve the accuracy of the suggestions. The input is user feedback data, and the output is an updated database and an improved AI model.

[0580] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0581] This invention combines an emotion engine with a system that provides personalized experiences when users visit entertainment facilities. This system is primarily composed of three components: a server, a terminal, and a user.

[0582] server:

[0583] The server, acting as a central data processing unit, collects user behavior data and generates personalized visit plans based on it. Furthermore, the server is equipped with an emotion engine that recognizes emotions from the user's facial expressions and voice data, and incorporates this information into the visit plan to provide more personalized suggestions. The server also analyzes congestion data and waiting times in real time and transmits necessary information to the user's device.

[0584] Terminal:

[0585] The device functions as a smartphone or AR-enabled device, presenting the user with personalized visit plans and local information provided by a server. The device uses the user's location information to provide real-time navigation and, when approaching specific attractions, offers visual information tailored to the user's emotional state. For example, if the user appears to be enjoying themselves, the device will suggest information and attractions that further enhance that enjoyable experience.

[0586] User:

[0587] Users can review suggested visit plans through the application and receive real-time guidance and emotion-appropriate attraction information while moving around the facility. After the visit, users enter feedback into the app, and this data is sent to the server to be used to improve future visit suggestions. Data obtained through emotion recognition is also collected as part of the feedback and used for analysis.

[0588] Specific example:

[0589] For example, if a user is moved while watching a dolphin show at an aquarium, the server will acquire that emotional data and prioritize suggesting animal interaction programs in subsequent visit plans. During the show, the device will provide AR-based information about the dolphins' behavior that the user might find interesting, enabling a deeper experience.

[0590] As described above, the present invention aims to enhance visits to entertainment facilities by providing personalized experiences that take into account the user's emotions.

[0591] The following describes the processing flow.

[0592] Step 1:

[0593] Users create a profile by launching the application, entering personal information, and selecting attractions and themes of interest to register.

[0594] Step 2:

[0595] The server generates a profile in the database based on the user's registration information and selected interests, and creates a personalized visit plan.

[0596] Step 3:

[0597] The terminal displays the visit plan sent from the server on the user's screen, allowing for visual confirmation. The user reviews the proposed plan and makes adjustments as needed.

[0598] Step 4:

[0599] On the day of the visit, the device acquires the user's location information via GPS and sends it to the server. The server calculates the latest congestion and wait time data and updates the device with the optimal attraction route.

[0600] Step 5:

[0601] The device uses AR functionality to visually provide information about attractions or animals specified by the user. An emotion engine recognizes the user's facial expressions and voice, and sends emotion data to a server.

[0602] Step 6:

[0603] The server analyzes emotional data and further adjusts the visit plan based on the user's current emotions. If the user is enjoying themselves, it prioritizes suggesting attractions that offer a similar experience.

[0604] Step 7:

[0605] Users navigate through attractions guided by their devices, receiving real-time information and emotionally adaptive visuals along the way. This allows them to make the most of their experience.

[0606] Step 8:

[0607] After their visit, users access the app to provide feedback on their experience. The emotion data recognized by the emotion engine is also sent to the server.

[0608] Step 9:

[0609] The server analyzes the collected feedback and sentiment data and uses a generative AI algorithm to make adjustments to improve the accuracy of suggestions for the next user visit.

[0610] (Example 2)

[0611] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0612] Modern entertainment facilities are required to provide personalized experiences for their users. However, existing systems only offer generic plans based on behavioral data, making it difficult to provide experiences that reflect users' emotions and real-time circumstances. Furthermore, there is a lack of effective ways to utilize user feedback, which makes continuous system improvement difficult.

[0613] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0614] In this invention, the server includes information processing means for analyzing behavioral and emotional information to generate a personalized action plan based on the user's interests and emotions; display means for providing visual information to the user using augmented reality technology; and data processing means for analyzing and presenting congestion status and waiting times in real time. This enables the provision of a personalized experience tailored to the user's emotions and circumstances, and allows for continuous improvement of the system based on feedback.

[0615] "Behavioral information" refers to data related to the user's actions, such as their location, travel history, and records of attractions they visited.

[0616] "Emotional information" refers to data about the emotional state analyzed from the user's facial expressions and voice.

[0617] An "individualized action plan" refers to a visit plan optimized for each individual user based on behavioral and emotional information.

[0618] "Information processing means" refers to software and hardware configured to collect and analyze behavioral and emotional information.

[0619] Augmented reality technology refers to technologies that integrate digital information into the real world and present it visually to users.

[0620] "Display means" refers to devices and software that utilize augmented reality technology to visually present digital information to users.

[0621] "Data processing means" refers to software or hardware for collecting and analyzing congestion status and waiting times in real time.

[0622] "Feedback" refers to information such as opinions and impressions that users provide after experiencing something.

[0623] "System improvement" refers to analyzing collected feedback and taking corrective or additional actions to improve the quality of the service.

[0624] This invention is a system that provides personalized experiences based on the user's behavior and emotions. This system mainly consists of three components: a server, a terminal, and a user.

[0625] server

[0626] The server functions as a central data processing unit. It collects behavioral and emotional information and uses this data to generate visit plans using an AI model. The collected data is analyzed through an emotion engine, which analyzes emotions from facial expressions and voice, and is used to create plans tailored to the user's experience. The server also analyzes congestion and wait times in real time and provides necessary information to the terminal. For example, if a user shows interest in multiple attractions, it can determine their priority and create an optimized plan.

[0627] terminal

[0628] The device functions as a smartphone or augmented reality device. It visually presents users with personalized visit plans and real-time crowd information transmitted from a server. When a user approaches a specific attraction, the device uses AR technology to provide additional information relevant to that location. For example, while watching a dolphin show at an aquarium, it can display detailed information about the dolphins' behavior in real time, based on the user's emotional state.

[0629] User

[0630] Users can review the suggested visit plan via their device and use it as a guide. After experiencing each attraction, they enter feedback into the application. This feedback data is sent to the server and used to provide an even more improved, personalized experience on their next visit.

[0631] Specific examples and prompt statements

[0632] For example, if a user rides a roller coaster at a theme park and expresses excitement, the server can prioritize suggesting similar thrilling attractions as the next one to visit. An example of a prompt in this case might be, "Please suggest a suitable next attraction after the roller coaster experience."

[0633] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0634] Step 1:

[0635] The server collects user behavior and emotional information. Inputs include user location data, visit history, facial expression data, and voice data. The server acquires this data using sensor technology and recognition software and stores the information in a cloud database. The output at this stage is an analyzable behavior and emotional dataset.

[0636] Step 2:

[0637] The server performs sentiment analysis based on the collected data. The input data is sentiment information obtained in the previous stage. The server uses an emotion engine to analyze facial expressions and voice parameters to determine the user's emotional state (e.g., joy, surprise, boredom, etc.). The output is an individualized sentiment profile. This profile is used to personalize the visit plan.

[0638] Step 3:

[0639] The server generates the next visit plan using a generative AI model. The input consists of existing behavioral data, emotional profiles, and real-time congestion data. Based on this, the generative AI model performs data processing and predictive calculations to output a visit plan optimized for each user. For example, it might suggest thrilling attractions during less crowded times.

[0640] Step 4:

[0641] The terminal displays the visit plan received from the server to the user. The input consists of the generated visit plan data and local information. The terminal displays this visually and provides detailed explanations using AR technology. Output is generated that includes navigation assistance, such as additional information about nearby attractions.

[0642] Step 5:

[0643] Users review the provided visit plan using a terminal and visit attractions according to the instructions. As feedback, users register their impressions and newly acquired interests on the terminal. This feedback data is sent to a server to be used to optimize future visit plans. This feedback output serves as valuable input for the user's next visit.

[0644] Step 6:

[0645] The server analyzes user feedback and continuously collected sentiment data to improve the entire system. Inputs include feedback data, sentiment history, and visit history. Based on this, an information processing program performs analysis, resulting in further improvements to the accuracy of action plans and overall system service quality.

[0646] (Application Example 2)

[0647] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0648] There is a need for systems that allow users to receive not only suggestions based on their interests in entertainment facilities and living spaces, but also more personalized experiences that take their emotions into account. However, conventional technologies have been insufficient in terms of individualization that takes emotions into account, making it difficult to achieve highly satisfying suggestions.

[0649] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0650] In this invention, the server includes information processing means for analyzing behavioral data to generate personalized visit plans based on the user's interests, display means for providing visual information to the user using virtual reality technology, and emotion analysis means for acquiring emotion data and making suggestions based on emotion recognition. This makes it possible to provide a personalized experience that responds to the user's emotions.

[0651] "Behavioral data" refers to information about a user's actions, such as their location and movement history within a facility.

[0652] An "individualized visit plan" refers to a facility visit schedule and suggestions that are personalized and optimized based on the user's interests and feelings.

[0653] "Information processing means" refers to computers and related equipment used to analyze user behavioral data and emotional data, and to generate visit plans based on the results.

[0654] "Virtual reality technology" refers to technology that combines information from the real world with digital information to provide users with visually rich information.

[0655] "Display means" refers to devices and technologies used to visually present information to users.

[0656] "Data processing means for analyzing and presenting congestion status and waiting times in real time" refers to technology that acquires and analyzes information on the flow of people and waiting times within a facility and reports it to users immediately.

[0657] "Data correction means" refers to a function that adjusts system data based on user feedback to improve the accuracy of proposals and visit plans.

[0658] "Emotional analysis methods" refer to technologies that analyze facial expressions and voice data to understand the user's emotions and incorporate the results into the visit plan.

[0659] "Personalization methods" refer to functions that select and provide information and suggestions according to the user's emotions and interests.

[0660] The system for carrying out the present invention consists of three components: a server, a terminal, and a user. The server is the central component that collects user behavioral and emotional data and generates personalized visit plans. The hardware used here is a high-performance data processing server, and the software includes an emotion recognition algorithm (e.g., a cloud-based emotion analysis API).

[0661] The server analyzes behavioral data to generate a visit plan tailored to the user's interests. This process also considers the user's emotional data, and the experience is personalized through corresponding individualization methods. This plan is sent to the device and presented visually to the user. The device functions specifically as a smartphone or AR glasses, receiving data from the server and providing information using virtual reality technology.

[0662] As a concrete example, when a user is with a home robot, the robot acquires the user's facial expressions and voice in real time, sends them to a server, and performs emotion analysis. Based on the resulting data, if it determines that the user is relaxed, it suggests playing relaxing music and optimizes the environment.

[0663] An example of a prompt message could be, "Write a flow for a robot program that analyzes the facial expression of a user returning home in the evening and plays healing music if they appear tired." This makes it possible to provide an optimal experience tailored to the user's emotions.

[0664] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0665] Step 1:

[0666] The server receives location and behavioral data from the user's device. As input, it retrieves data on the user's current location and past behavioral history, and stores it in a database. This prepares the server to identify the user's interests and behavioral patterns.

[0667] Step 2:

[0668] The device uses sensors to acquire the user's facial expressions and voice data, and sends it to the server. Real-time facial and voice information of the user is acquired as input and passed to an emotion recognition algorithm. The server uses a generative AI model to analyze emotions and determine the user's current emotional state. The user's emotion data is obtained as output.

[0669] Step 3:

[0670] The server generates personalized visit plans using a generative AI model based on the acquired behavioral and emotional data. Congestion and waiting time data are also considered during the plan generation process. User behavioral and emotional data are used as input, and a visit plan tailored to the user is created as output.

[0671] Step 4:

[0672] The server sends the generated visit plan to the terminal, which then visually presents it to the user. The terminal uses virtual reality technology to visualize the plan in an easy-to-use way. It receives visit plan data sent from the server as input and generates information that is easy for the user to understand as output.

[0673] Step 5:

[0674] After a user visits the system, feedback is sent from the user's device to the server. The server collects the feedback and uses data correction mechanisms to improve the system. It receives user feedback data as input and generates data necessary for system optimization as output.

[0675] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0676] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0677] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0678] [Fourth Embodiment]

[0679] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0680] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0681] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0682] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0683] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0684] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0685] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0686] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0687] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0688] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0689] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0690] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0691] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0692] This invention implements a system that provides personalized experiences to users when they visit entertainment facilities. This system is mainly composed of a server, a terminal, and a user, and its embodiments are shown below.

[0693] server:

[0694] The server functions as a central information processing unit, analyzing behavioral data collected from users to generate personalized visit plans. This process takes into account the user's past behavioral history and interests to create the optimal attraction route. The server also manages real-time data on crowd levels and wait times, transmitting this information to the terminals.

[0695] Terminal:

[0696] The terminal functions as a smartphone or AR device, presenting users with visit plans and local information provided by a server. As users move around the facility, the terminal uses virtual reality technology to provide visual information about attractions and animals. For example, when a user sees an attraction through the terminal's camera, it displays a description of that attraction and the current wait time.

[0697] User:

[0698] Users can smoothly enjoy the facility by reviewing the visit plan suggested by the system based on their interests. By operating a terminal, users can receive real-time information and efficiently tour the facility. Furthermore, users can provide feedback after their visit, and the server uses this information to improve the system, thereby increasing the accuracy of suggestions for future visits.

[0699] Specific example:

[0700] For example, if a user prefers adventure-type attractions in an amusement park, the server will take this interest into account and suggest a schedule that includes a roller coaster first, followed by a visit to a haunted house. The device will display the wait time for the roller coaster and suggest other options if the wait is long. Furthermore, when approaching a haunted house via the device's camera, it will provide visual information about its background story and safety guidelines.

[0701] In summary, the present invention personalizes the user's experience when visiting entertainment facilities, enabling a more fulfilling visit.

[0702] The following describes the processing flow.

[0703] Step 1:

[0704] Users launch the application, enter their personal information and interests, and register.

[0705] Step 2:

[0706] The server receives user registration information, stores it in a database, and generates an interest-based profile.

[0707] Step 3:

[0708] The user selects the facility they plan to visit from the application and enters the date.

[0709] Step 4:

[0710] The server collects attraction information, operating hours, and crowd prediction data for the selected facility, and combines this with the user's profile to generate an optimal visit plan.

[0711] Step 5:

[0712] The terminal displays the visit plan received from the server on its screen, visually presenting it to the user.

[0713] Step 6:

[0714] The user reviews the proposed visit plan and makes adjustments as needed.

[0715] Step 7:

[0716] On the day of the visit, the device uses GPS functionality to obtain the user's location information and provides guidance to attractions based on their current location.

[0717] Step 8:

[0718] The server collects real-time wait time data for each attraction and sends it to the terminal.

[0719] Step 9:

[0720] The device uses augmented reality (AR) technology to display visual information about attractions around the user via its camera, showing crowd levels and background information.

[0721] Step 10:

[0722] Users can efficiently navigate attractions and facilities through their devices, enjoying the experience.

[0723] Step 11:

[0724] After visiting, users enter feedback into the application and send their evaluation of the experience to the server.

[0725] Step 12:

[0726] The server analyzes user feedback and adjusts the system to improve the accuracy of future suggestions.

[0727] (Example 1)

[0728] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0729] Traditionally, when visiting entertainment facilities, it was difficult for visitors to plan their visits while fully considering their interests and the expected crowd levels, resulting in an inefficient enjoyment of the facilities. Furthermore, there were insufficient methods for effectively utilizing feedback to improve the visit experience. As a result, it was difficult to provide highly satisfying experiences tailored to individual interests.

[0730] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0731] In this invention, the server includes a storage means for accumulating user behavior history and generating a user profile, an information processing means for generating personalized visit plans using a generated AI model, and a display means for providing visual presentations using virtual reality technology. This makes it possible to provide personalized visit plans based on the user's interests and real-time congestion data.

[0732] "User activity history" refers to information about the user's past activities and visits to attractions within the facility.

[0733] A "user profile" is a collection of attribute information that reflects a user's interests and preferences, and is used to provide personalized services.

[0734] "Memory devices" refer to devices or methods for storing data long-term or short-term, and are responsible for storing user profiles and behavioral history.

[0735] A "generative AI model" is an artificial intelligence platform that analyzes user data and behavioral history to make predictions and suggestions based on specific objectives.

[0736] "Information processing means" refers to technical means for collecting, analyzing, and processing data to generate useful information.

[0737] "Virtual reality technology" is a technology that visually presents a virtual environment generated by a computer.

[0738] "Visual presentation" refers to a method of providing information to users visually, and specifically includes displays using screens or projectors.

[0739] "Real-time data" refers to data that is updated and provided in real time, such as information on congestion levels and waiting times.

[0740] "Data analysis" refers to statistical or computational methods used to extract useful information from collected data.

[0741] "Feedback" refers to the opinions and evaluations that users provide regarding a service or experience, and this information is used to improve the system.

[0742] This invention aims to realize a system that personalizes the user's visit experience in entertainment facilities, providing an efficient and highly satisfying experience. Specific embodiments are described below.

[0743] server:

[0744] The server utilizes a database management system and AI technology to memorize user behavior history and generate profiles. Specifically, it stores user behavior data in databases using MySQL or PostgreSQL. The generated AI model is built using artificial intelligence libraries such as TensorFlow and PyTorch, which generates personalized visit plans. The server also has the ability to collect and analyze real-time data on facility congestion and waiting times. This is achieved by using Apache Kafka to enable real-time data streaming.

[0745] Terminal:

[0746] The device functions as a smartphone or AR device, presenting users with plans and local information. Users can operate applications on the device to check visit plans and real-time information. This application is developed using Java or Kotlin for Android devices and Swift for iOS devices. The device uses AR technology to visually present information about each attraction. This utilizes ARCore for Android and ARKit for iOS.

[0747] User:

[0748] Users can efficiently enjoy the facilities by following the visit plan provided by the system. They can use their devices to check detailed information and crowd conditions for attractions, allowing them to make informed decisions. Furthermore, it is expected that users will provide feedback via their devices after their visit, improving the quality of future visit plans.

[0749] Specific example:

[0750] For example, if a user prefers adventure attractions, the server can generate a plan based on this preference, visiting a roller coaster first, followed by a haunted house. The terminal can display the roller coaster's wait time and crowd status in real time and offer alternative suggestions as needed. As the user approaches the haunted house, relevant background stories and safety guidelines are visually provided through the terminal. This plan generation can use a prompt such as, "Suggest a visit plan for a trip to the amusement park. The user likes adventure and wants to avoid crowded places."

[0751] This system allows users to have a highly satisfying browsing experience based on personalized information.

[0752] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0753] Step 1:

[0754] The server collects user activity history data from the database. The input for this process is past facility visit records based on the user ID, and the output is a dataset of activity history. The server uses database queries to extract information about attractions the user has visited in the past and how frequently they were visited.

[0755] Step 2:

[0756] The server generates and stores user profiles using behavioral history. This profile includes information about the user's interests and tendencies. The input is behavioral history data obtained from step 1, and the output is the user's interest profile. The server applies a data analysis algorithm to update this profile to the latest state.

[0757] Step 3:

[0758] The server generates personalized visit plans using a generative AI model. Inputs are user profiles and real-time facility data (e.g., congestion information, wait times, etc.), and output is a personalized visit plan. The generative AI model uses machine learning techniques to perform calculations to provide the optimal attraction order that reflects the user's interests.

[0759] Step 4:

[0760] The server collects real-time data from sensors installed at various points within the facility and transmits it to the terminal. The input for this process is real-time data from IoT sensors, and the output is analyzed congestion status and waiting time information. The server uses stream processing to analyze this data in a timely manner and transfer it to the user's terminal.

[0761] Step 5:

[0762] The terminal displays visit plans and real-time information sent from the server to the user. Inputs are visit plan data and congestion information received from the server, while output is visual information presented to the user. The terminal uses a user interface to display attraction wait times and tourist information.

[0763] Step 6:

[0764] Users provide feedback on their experience at the facility via a terminal after their visit. The input consists of the user's subjective experience, while the output is feedback data sent to the server. Users register their ratings and opinions using the feedback form on the terminal.

[0765] Step 7:

[0766] The server analyzes feedback collected from users and uses it to improve the next visit plan. The input to this process is feedback data, and the output is updated generative AI model parameters. The server performs data analysis and retrains the AI ​​model to improve the accuracy of the suggestions.

[0767] (Application Example 1)

[0768] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0769] Traditional tourist information systems required users to research a large amount of information in advance and select destinations that matched their interests. This presented a problem in that selecting appropriate tourist spots and creating efficient travel plans was difficult, especially in unfamiliar areas. Furthermore, congestion and unexpected waiting times on the day of visit often prevented visitors from sticking to their plans. Improving these conditions and enhancing the tourist experience is crucial.

[0770] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0771] In this invention, the server includes means for analyzing behavioral history and preference data to generate personalized visit routes based on the user's interests, means for providing visual information to the user using augmented reality technology, and means for analyzing congestion and waiting times in real time and providing that information. As a result, users can efficiently visit tourist destinations based on personalized visit plans, and flexible plan changes can be made in response to real-time conditions.

[0772] "Activity history" refers to a record of various actions and movements that a user has taken up to that point.

[0773] "Preference data" refers to data that indicates a user's interests and preferences, and is information collected based on past choices and hobbies.

[0774] A "personalized itinerary" is an optimal route for sightseeing or visiting that is specially created taking into account the user's specific interests and behavioral history.

[0775] Augmented reality technology is a technology that overlays digital information onto the real world, thereby improving the user's visual experience.

[0776] A "means for analyzing congestion and waiting times in real time" refers to a system that enables the immediate analysis of the current level of congestion and waiting times at a location and provides this information to users.

[0777] A "server" is a central information processing device that processes and stores data via a network and provides analysis results to users.

[0778] A "terminal device" is a device that a user can carry and use, and whose role is to receive information from a server and present it to the user.

[0779] "Data improvement methods" refer to data processing techniques that collect feedback from users and contribute to improving system performance and the accuracy of suggestions.

[0780] "Route generation means for dynamically adjusting visit routes between facilities in the tourism sector" refers to a technology that generates and modifies the optimal visit route in real time according to the user's situation and the facility's situation.

[0781] This invention provides a system that personalizes the tourism experience based on the user's preferences and behavioral history. Specifically, the server collects and analyzes past behavioral history and interest data provided by the user. The software used in this process includes Apache Hadoop for managing and analyzing big data, and Scikit-learn for analyzing data using machine learning algorithms. This generates a travel route tailored to the user's specific preferences.

[0782] Simultaneously, the server collects real-time data on congestion levels and waiting times at tourist destinations, and immediately displays this information to users. This allows users to check whether their pre-planned schedule is appropriate for the current situation at the destination.

[0783] The terminal plays a role in providing users with visual information using augmented reality technology. Specifically, the terminal combines the user's location information with information provided by the server to display detailed information and visual guides about facilities that interest the user on the screen. This process utilizes AR devices (such as Microsoft HoloLens) to leverage augmented reality technology. This makes it possible to directly overlay information about visited locations and attractions onto the user's field of view.

[0784] Furthermore, users are required to provide feedback after their visit. This feedback is sent to the server and used by data refinement tools to improve the experience on subsequent visits. Through this iterative process, the AI ​​model continuously learns, improving the accuracy of suggestions and providing a more personalized user experience.

[0785] For example, if a user is interested in history, the server will take that interest into account and suggest routes to historically significant museums and landmarks. Based on this, the terminal can display real-time congestion information and inform tourists of the best time to visit.

[0786] An example of a prompt for a generative AI model might be a command such as, "Analyze the user's history and suggest the optimal visiting route for them using an AR device." This prompt allows the AI ​​to provide personalized sightseeing plans in real time.

[0787] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0788] Step 1:

[0789] The server receives user behavior history and preference data. Input data includes past visits, activities, and categories of interest. Based on this, Apache Hadoop is used for big data management and analysis to generate user preference profiles. The output is an individualized preference profile.

[0790] Step 2:

[0791] The server uses the generated preference profile to create the optimal travel route. Scikit-learn is used to determine the priority of potential destinations. The input is the preference profile and available tourist destination information. The output is a prioritized list of destinations and the travel route.

[0792] Step 3:

[0793] The server collects real-time congestion and wait time data for tourist destinations. This uses real-time data from an external API. The input is the data from the API response, which is analyzed to obtain congestion and wait time information. The output is the latest congestion and wait time information.

[0794] Step 4:

[0795] The terminal receives visit route and congestion information provided by the server and presents it to the user. This process uses AR technology to visually display the information. The terminal's input is data from the server, and its output is the presentation of visual information to the user. Specifically, the terminal activates an AR device (e.g., Microsoft HoloLens) and virtually overlays tourist information onto it.

[0796] Step 5:

[0797] Users follow a suggested itinerary displayed on their device while sightseeing. During the tour, they can directly input their satisfaction level at each tourist spot. This enables real-time feedback. Input consists of user ratings and feedback, while output is their satisfaction data.

[0798] Step 6:

[0799] The server collects feedback from users and uses the data to improve the accuracy of future suggestions. Based on the collected data, the generating AI model is fed into a feedback loop to improve the accuracy of the suggestions. The input is user feedback data, and the output is an updated database and an improved AI model.

[0800] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0801] This invention combines an emotion engine with a system that provides personalized experiences when users visit entertainment facilities. This system is primarily composed of three components: a server, a terminal, and a user.

[0802] server:

[0803] The server, acting as a central data processing unit, collects user behavior data and generates personalized visit plans based on it. Furthermore, the server is equipped with an emotion engine that recognizes emotions from the user's facial expressions and voice data, and incorporates this information into the visit plan to provide more personalized suggestions. The server also analyzes congestion data and waiting times in real time and transmits necessary information to the user's device.

[0804] Terminal:

[0805] The device functions as a smartphone or AR-enabled device, presenting the user with personalized visit plans and local information provided by a server. The device uses the user's location information to provide real-time navigation and, when approaching specific attractions, offers visual information tailored to the user's emotional state. For example, if the user appears to be enjoying themselves, the device will suggest information and attractions that further enhance that enjoyable experience.

[0806] User:

[0807] Users can review suggested visit plans through the application and receive real-time guidance and emotion-appropriate attraction information while moving around the facility. After the visit, users enter feedback into the app, and this data is sent to the server to be used to improve future visit suggestions. Data obtained through emotion recognition is also collected as part of the feedback and used for analysis.

[0808] Specific example:

[0809] For example, if a user is moved while watching a dolphin show at an aquarium, the server will acquire that emotional data and prioritize suggesting animal interaction programs in subsequent visit plans. During the show, the device will provide AR-based information about the dolphins' behavior that the user might find interesting, enabling a deeper experience.

[0810] As described above, the present invention aims to enhance visits to entertainment facilities by providing personalized experiences that take into account the user's emotions.

[0811] The following describes the processing flow.

[0812] Step 1:

[0813] Users create a profile by launching the application, entering personal information, and selecting attractions and themes of interest to register.

[0814] Step 2:

[0815] The server generates a profile in the database based on the user's registration information and selected interests, and creates a personalized visit plan.

[0816] Step 3:

[0817] The terminal displays the visit plan sent from the server on the user's screen, allowing for visual confirmation. The user reviews the proposed plan and makes adjustments as needed.

[0818] Step 4:

[0819] On the day of the visit, the device acquires the user's location information via GPS and sends it to the server. The server calculates the latest congestion and wait time data and updates the device with the optimal attraction route.

[0820] Step 5:

[0821] The device uses AR functionality to visually provide information about attractions or animals specified by the user. An emotion engine recognizes the user's facial expressions and voice, and sends emotion data to a server.

[0822] Step 6:

[0823] The server analyzes emotional data and further adjusts the visit plan based on the user's current emotions. If the user is enjoying themselves, it prioritizes suggesting attractions that offer a similar experience.

[0824] Step 7:

[0825] Users navigate through attractions guided by their devices, receiving real-time information and emotionally adaptive visuals along the way. This allows them to make the most of their experience.

[0826] Step 8:

[0827] After their visit, users access the app to provide feedback on their experience. The emotion data recognized by the emotion engine is also sent to the server.

[0828] Step 9:

[0829] The server analyzes the collected feedback and sentiment data and uses a generative AI algorithm to make adjustments to improve the accuracy of suggestions for the next user visit.

[0830] (Example 2)

[0831] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0832] Modern entertainment facilities are required to provide personalized experiences for their users. However, existing systems only offer generic plans based on behavioral data, making it difficult to provide experiences that reflect users' emotions and real-time circumstances. Furthermore, there is a lack of effective ways to utilize user feedback, which makes continuous system improvement difficult.

[0833] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0834] In this invention, the server includes information processing means for analyzing behavioral and emotional information to generate a personalized action plan based on the user's interests and emotions; display means for providing visual information to the user using augmented reality technology; and data processing means for analyzing and presenting congestion status and waiting times in real time. This enables the provision of a personalized experience tailored to the user's emotions and circumstances, and allows for continuous improvement of the system based on feedback.

[0835] "Behavioral information" refers to data related to the user's actions, such as their location, travel history, and records of attractions they visited.

[0836] "Emotional information" refers to data about the emotional state analyzed from the user's facial expressions and voice.

[0837] An "individualized action plan" refers to a visit plan optimized for each individual user based on behavioral and emotional information.

[0838] "Information processing means" refers to software and hardware configured to collect and analyze behavioral and emotional information.

[0839] Augmented reality technology refers to technologies that integrate digital information into the real world and present it visually to users.

[0840] "Display means" refers to devices and software that utilize augmented reality technology to visually present digital information to users.

[0841] "Data processing means" refers to software or hardware for collecting and analyzing congestion status and waiting times in real time.

[0842] "Feedback" refers to information such as opinions and impressions that users provide after experiencing something.

[0843] "System improvement" refers to analyzing collected feedback and taking corrective or additional actions to improve the quality of the service.

[0844] This invention is a system that provides personalized experiences based on the user's behavior and emotions. This system mainly consists of three components: a server, a terminal, and a user.

[0845] server

[0846] The server functions as a central data processing unit. It collects behavioral and emotional information and uses this data to generate visit plans using an AI model. The collected data is analyzed through an emotion engine, which analyzes emotions from facial expressions and voice, and is used to create plans tailored to the user's experience. The server also analyzes congestion and wait times in real time and provides necessary information to the terminal. For example, if a user shows interest in multiple attractions, it can determine their priority and create an optimized plan.

[0847] terminal

[0848] The device functions as a smartphone or augmented reality device. It visually presents users with personalized visit plans and real-time crowd information transmitted from a server. When a user approaches a specific attraction, the device uses AR technology to provide additional information relevant to that location. For example, while watching a dolphin show at an aquarium, it can display detailed information about the dolphins' behavior in real time, based on the user's emotional state.

[0849] User

[0850] Users can review the suggested visit plan via their device and use it as a guide. After experiencing each attraction, they enter feedback into the application. This feedback data is sent to the server and used to provide an even more improved, personalized experience on their next visit.

[0851] Specific examples and prompt statements

[0852] For example, if a user rides a roller coaster at a theme park and expresses excitement, the server can prioritize suggesting similar thrilling attractions as the next one to visit. An example of a prompt in this case might be, "Please suggest a suitable next attraction after the roller coaster experience."

[0853] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0854] Step 1:

[0855] The server collects user behavior and emotional information. Inputs include user location data, visit history, facial expression data, and voice data. The server acquires this data using sensor technology and recognition software and stores the information in a cloud database. The output at this stage is an analyzable behavior and emotional dataset.

[0856] Step 2:

[0857] The server performs sentiment analysis based on the collected data. The input data is sentiment information obtained in the previous stage. The server uses an emotion engine to analyze facial expressions and voice parameters to determine the user's emotional state (e.g., joy, surprise, boredom, etc.). The output is an individualized sentiment profile. This profile is used to personalize the visit plan.

[0858] Step 3:

[0859] The server generates the next visit plan using a generative AI model. The input consists of existing behavioral data, emotional profiles, and real-time congestion data. Based on this, the generative AI model performs data processing and predictive calculations to output a visit plan optimized for each user. For example, it might suggest thrilling attractions during less crowded times.

[0860] Step 4:

[0861] The terminal displays the visit plan received from the server to the user. The input consists of the generated visit plan data and local information. The terminal displays this visually and provides detailed explanations using AR technology. Output is generated that includes navigation assistance, such as additional information about nearby attractions.

[0862] Step 5:

[0863] Users review the provided visit plan using a terminal and visit attractions according to the instructions. As feedback, users register their impressions and newly acquired interests on the terminal. This feedback data is sent to a server to be used to optimize future visit plans. This feedback output serves as valuable input for the user's next visit.

[0864] Step 6:

[0865] The server analyzes user feedback and continuously collected sentiment data to improve the entire system. Inputs include feedback data, sentiment history, and visit history. Based on this, an information processing program performs analysis, resulting in further improvements to the accuracy of action plans and overall system service quality.

[0866] (Application Example 2)

[0867] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0868] There is a need for systems that allow users to receive not only suggestions based on their interests in entertainment facilities and living spaces, but also more personalized experiences that take their emotions into account. However, conventional technologies have been insufficient in terms of individualization that takes emotions into account, making it difficult to achieve highly satisfying suggestions.

[0869] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0870] In this invention, the server includes information processing means for analyzing behavioral data to generate personalized visit plans based on the user's interests, display means for providing visual information to the user using virtual reality technology, and emotion analysis means for acquiring emotion data and making suggestions based on emotion recognition. This makes it possible to provide a personalized experience that responds to the user's emotions.

[0871] "Behavioral data" refers to information about a user's actions, such as their location and movement history within a facility.

[0872] An "individualized visit plan" refers to a facility visit schedule and suggestions that are personalized and optimized based on the user's interests and feelings.

[0873] "Information processing means" refers to computers and related equipment used to analyze user behavioral data and emotional data, and to generate visit plans based on the results.

[0874] "Virtual reality technology" refers to technology that combines information from the real world with digital information to provide users with visually rich information.

[0875] "Display means" refers to devices and technologies used to visually present information to users.

[0876] "Data processing means for analyzing and presenting congestion status and waiting times in real time" refers to technology that acquires and analyzes information on the flow of people and waiting times within a facility and reports it to users immediately.

[0877] "Data correction means" refers to a function that adjusts system data based on user feedback to improve the accuracy of proposals and visit plans.

[0878] "Emotional analysis methods" refer to technologies that analyze facial expressions and voice data to understand the user's emotions and incorporate the results into the visit plan.

[0879] "Personalization methods" refer to functions that select and provide information and suggestions according to the user's emotions and interests.

[0880] The system for carrying out the present invention consists of three components: a server, a terminal, and a user. The server is the central component that collects user behavioral and emotional data and generates personalized visit plans. The hardware used here is a high-performance data processing server, and the software includes an emotion recognition algorithm (e.g., a cloud-based emotion analysis API).

[0881] The server analyzes behavioral data to generate a visit plan tailored to the user's interests. This process also considers the user's emotional data, and the experience is personalized through corresponding individualization methods. This plan is sent to the device and presented visually to the user. The device functions specifically as a smartphone or AR glasses, receiving data from the server and providing information using virtual reality technology.

[0882] As a concrete example, when a user is with a home robot, the robot acquires the user's facial expressions and voice in real time, sends them to a server, and performs emotion analysis. Based on the resulting data, if it determines that the user is relaxed, it suggests playing relaxing music and optimizes the environment.

[0883] An example of a prompt message could be, "Write a flow for a robot program that analyzes the facial expression of a user returning home in the evening and plays healing music if they appear tired." This makes it possible to provide an optimal experience tailored to the user's emotions.

[0884] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0885] Step 1:

[0886] The server receives location and behavioral data from the user's device. As input, it retrieves data on the user's current location and past behavioral history, and stores it in a database. This prepares the server to identify the user's interests and behavioral patterns.

[0887] Step 2:

[0888] The device uses sensors to acquire the user's facial expressions and voice data, and sends it to the server. Real-time facial and voice information of the user is acquired as input and passed to an emotion recognition algorithm. The server uses a generative AI model to analyze emotions and determine the user's current emotional state. The user's emotion data is obtained as output.

[0889] Step 3:

[0890] The server generates personalized visit plans using a generative AI model based on the acquired behavioral and emotional data. Congestion and waiting time data are also considered during the plan generation process. User behavioral and emotional data are used as input, and a visit plan tailored to the user is created as output.

[0891] Step 4:

[0892] The server sends the generated visit plan to the terminal, which then visually presents it to the user. The terminal uses virtual reality technology to visualize the plan in an easy-to-use way. It receives visit plan data sent from the server as input and generates information that is easy for the user to understand as output.

[0893] Step 5:

[0894] After a user visits the system, feedback is sent from the user's device to the server. The server collects the feedback and uses data correction mechanisms to improve the system. It receives user feedback data as input and generates data necessary for system optimization as output.

[0895] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0896] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0897] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0898] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0899] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0900] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0901] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0902] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0903] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0904] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0905] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0906] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0907] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0908] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0909] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0910] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0911] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0912] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0913] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0914] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0915] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0916] The following is further disclosed regarding the embodiments described above.

[0917] (Claim 1)

[0918] An information processing means that analyzes behavioral data to generate personalized visit plans based on the user's interests,

[0919] A display means that provides visual information to users using virtual reality technology,

[0920] A data processing method that analyzes and presents congestion status and waiting times in real time,

[0921] A data correction method that collects feedback and uses it to improve the system,

[0922] A system that includes this.

[0923] (Claim 2)

[0924] The system according to claim 1, comprising a storage means for generating a profile based on the user's usage history and improving the accuracy of suggesting visit plans.

[0925] (Claim 3)

[0926] The system according to claim 1, including an information processing program for analyzing feedback data and optimizing the content of the next proposal.

[0927] "Example 1"

[0928] (Claim 1)

[0929] A storage means for accumulating user behavior history and generating user profiles,

[0930] Information processing means that generates an individualized visit plan using a generated AI model based on that profile,

[0931] A data processing system that monitors local congestion and waiting times in real time, analyzes and presents this data,

[0932] A display means that provides visual presentations using virtual reality technology,

[0933] A data correction method that collects user feedback and performs data analysis to improve system performance,

[0934] A system that includes this.

[0935] (Claim 2)

[0936] The system according to claim 1, comprising an information processing program that analyzes environmental data acquired in real time and adjusts the visit plan.

[0937] (Claim 3)

[0938] The system according to claim 1, which has a function to retrain the generated AI model using feedback data to improve the accuracy of the next proposal.

[0939] "Application Example 1"

[0940] (Claim 1)

[0941] A server means that analyzes behavioral history and preference data to generate personalized visit routes based on the user's interests,

[0942] A terminal device that provides visual information to users using augmented reality technology,

[0943] A data processing means that analyzes congestion status and waiting times in real time and provides that information,

[0944] A data improvement method that collects feedback and uses it to improve the quality of the system,

[0945] A route generation means for dynamically adjusting visit routes between facilities in the tourism sector,

[0946] A system that includes this.

[0947] (Claim 2)

[0948] The system according to claim 1, comprising recording means for generating a profile based on user interest data and improving the accuracy of suggested visit routes.

[0949] (Claim 3)

[0950] The system according to claim 1, comprising an information processing program for analyzing feedback data and optimizing the next proposal.

[0951] "Example 2 of combining an emotion engine"

[0952] (Claim 1)

[0953] An information processing means that analyzes behavioral and emotional information to generate an individualized action plan based on the user's interests and emotions,

[0954] A display means that provides visual information to users using augmented reality technology,

[0955] A data processing method that analyzes and presents congestion status and waiting times in real time,

[0956] A data correction method that collects user feedback and emotion recognition data and uses it to improve the system,

[0957] A system that includes this.

[0958] (Claim 2)

[0959] The system according to claim 1, comprising a storage means for generating a profile based on the user's usage history and emotional data, and for improving the accuracy of suggesting action plans.

[0960] (Claim 3)

[0961] The system according to claim 1, comprising an information processing program for analyzing feedback data and emotion recognition data to optimize the content of the next proposal.

[0962] "Application example 2 when combining with an emotional engine"

[0963] (Claim 1)

[0964] An information processing means that analyzes behavioral data to generate personalized visit plans based on the user's interests,

[0965] A display means that provides visual information to users using virtual reality technology,

[0966] A data processing method that analyzes and presents congestion status and waiting times in real time,

[0967] A data correction method that collects feedback and uses it to improve the system,

[0968] A means for analyzing emotions to acquire emotional data and make suggestions based on emotion recognition,

[0969] Personalization methods for selecting and providing entertainment information according to the user's emotions,

[0970] A system that includes this.

[0971] (Claim 2)

[0972] The system according to claim 1, comprising a storage means for generating a profile based on the user's usage history and improving the accuracy of suggesting visit plans.

[0973] (Claim 3)

[0974] The system according to claim 1, including an information processing program for analyzing feedback data and optimizing the content of the next proposal. [Explanation of symbols]

[0975] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. An information processing means that analyzes behavioral data to generate personalized visit plans based on the user's interests, A display means that provides visual information to users using virtual reality technology, A data processing method that analyzes and presents congestion status and waiting times in real time, A data correction method that collects feedback and uses it to improve the system, A system that includes this.

2. The system according to claim 1, comprising a storage means for generating a profile based on the user's usage history and improving the accuracy of suggesting visit plans.

3. The system according to claim 1, including an information processing program for analyzing feedback data and optimizing the content of the next proposal.

Images