system

Abstract

We provide the system. [Solution] A means of acquiring data to collect visitor behavior information, A generation means that analyzes the aforementioned behavioral information and generates activity routes and recommended information optimized for the visitor, Information presentation means that provides the aforementioned recommended information to visitors in real time, A visualization means that presents visual information using augmented reality technology, A home navigation system that suggests optimized activity routes based on the interests and schedules of residents within the home, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] When visitors visit amusement parks or aquariums, they hope for an efficient and enjoyable experience. However, currently, real-time individual information provision and route management within the facilities of the visited places are insufficient, making it difficult to provide an optimal experience based on the interests of the visitors. In addition, there is a lack of means to visually provide prediction of waiting time, avoidance of congestion, and detailed explanations about attractions, which are factors leading to a decrease in the satisfaction of the visitors. To solve these problems, a new system that realizes personalized information provision based on the interests and behaviors of visitors is required.

Means for Solving the Problems

[0005] This invention collects visitor behavioral information, and a generative model analyzes this information to generate optimized attraction routes and recommended information in real time. This information is provided to the visitor's terminal via an information display means, and further visual information about attractions and animals is presented using augmented reality technology. As a result, visitors can obtain efficient route guidance and detailed information tailored to their interests, allowing them to enjoy a more fulfilling visit experience. It also includes a navigation function based on the visitor's current location to support congestion avoidance and timely movement.

[0006] "Data acquisition means" refers to a device or method for collecting visitor behavior information and location information.

[0007] A "generative modeling system" is a mechanism that analyzes collected data to generate optimized attraction routes and recommended information for visitors.

[0008] An "information presentation means" is a means of providing generated recommendation information to visitors in real time.

[0009] A "visualization method" is a system that uses augmented reality technology to present visitors with detailed visual information about attractions and animals.

[0010] A "navigation function" is a feature that provides the optimal travel route based on the visitor's current location.

[0011] "Personalized settings" are settings that individually adjust the information and recommendations provided based on the visitor's interests and preferences. [Brief explanation of the drawing]

[0012] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

Embodiments for Carrying Out the Invention

[0013] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0014] First, the language used in the following description will be explained.

[0015] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0016] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0017] In the following embodiments, the numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0018] In the following embodiments, the numbered communication I / F (Interface) is an interface that includes a communication processor and an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark), etc.

[0019] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0020] [First Embodiment]

[0021] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0022] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0023] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0024] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0025] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0026] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0027] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0028] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0029] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0030] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0031] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0032] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0033] This invention relates to a system for providing visitors with an optimized experience in amusement parks and aquariums. The following is a description of the configuration and operation based on an embodiment of this system.

[0034] First, the user installs a dedicated application on their smartphone. When starting to use the app, the user sets their preferred attractions and animals. The device sends this setting information to the server, which then forms the user's profile.

[0035] As visitors move around the facility, the device collects location information and user behavior data in real time. This includes information such as which attractions they visited and how long they stayed in each area. The device continuously transmits this information to the server.

[0036] The server analyzes user behavior patterns and interests based on the received data. A generative model uses this data to create real-time recommended routes for attractions and event information tailored to each visitor. These recommendations are sent from the server to the terminal and presented to the user through the user interface.

[0037] Furthermore, the device uses augmented reality technology to provide detailed visual information about the attractions and animals on display at the visited location. For example, by holding a smartphone over a specific animal, users can visually understand information and the animal's ecology through a 3D model.

[0038] As a result, users can follow attraction routes optimized to their interests, reduce wait times, and efficiently explore the park. Furthermore, the feedback gained through this experience is stored on the server, helping to improve the accuracy of future system suggestions.

[0039] This system aims to enhance visitor satisfaction by providing visitors with enjoyable, efficient, and interest-based information. Furthermore, its efficient traffic flow management technology contributes to the smooth operation of the facility itself.

[0040] The following describes the processing flow.

[0041] Step 1:

[0042] Users install the app on their smartphones and enter information about their interests and preferences. This includes selecting attractions, animals, and other things they are interested in.

[0043] Step 2:

[0044] The device collects information entered by the user and transmits it to a server via the internet. This creates a user profile on the server.

[0045] Step 3:

[0046] Once a user begins visiting a facility, the device uses GPS and internal sensors to collect the user's location and behavioral data in real time. This information shows which areas the user has visited and which attractions they are enjoying.

[0047] Step 4:

[0048] The device periodically sends collected location and behavioral data to the server. This data transmission is processed appropriately to ensure that user data is anonymized.

[0049] Step 5:

[0050] The server analyzes the received data and generates recommendations for optimal attraction routes and events based on the user's visit patterns and interests. Machine learning algorithms are applied to this analysis.

[0051] Step 6:

[0052] The server sends the generated recommendations to the device. This information includes recommended attractions near the current location and routes with shorter wait times.

[0053] Step 7:

[0054] The device displays the received information to the user within the app. It also uses AR technology to overlay detailed visual information about attractions and animals onto the real world.

[0055] Step 8:

[0056] Users navigate the park and enjoy the attractions based on the information provided. They can also input their satisfaction level and feedback into the app.

[0057] Step 9:

[0058] The server receives feedback from users and uses it as data to improve the generative model. This allows for more accurate suggestions to be provided on subsequent visits.

[0059] (Example 1)

[0060] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0061] Ensuring visitors have an efficient and satisfying facility experience tailored to their interests is a challenging problem. Traditional facility guide systems lack the ability to provide personalized experiences based on individual visitors' interests and behavioral patterns, often forcing visitors to navigate unplanned routes in crowded environments. As a result, visitors are unable to fully appreciate the facility's attractions. Furthermore, managing visitor flow becomes difficult for facilities, leading to decreased overall operational efficiency.

[0062] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0063] In this invention, the server includes information input means for visitors to input their interests, data acquisition means for collecting visitor location information and behavioral data in real time, and generative model means for generating attraction routes and recommendation information optimized for visitors using a generative AI model. As a result, visitors can receive optimal attraction routes based on their interests in real time, enabling an efficient and satisfying facility experience. Furthermore, by using this system, the facility can manage visitor flow more effectively and improve overall operational efficiency.

[0064] "Information input means" refers to equipment or interfaces that allow visitors to input their interests and utilize that information in their experience at the facility.

[0065] A "profile creation method" refers to a technique or system for organizing and constructing information related to individual visitors based on their interests and behavioral history.

[0066] "Data acquisition means" refers to technologies and devices for collecting location information and behavioral data of visitors within a facility.

[0067] A "generative modeling system" is a mechanism that analyzes visitors' interests and behavioral patterns and uses a generative AI model to create optimal attraction routes and recommended information.

[0068] "Information presentation means" refers to methods and devices for providing visitors with real-time generated attraction routes and recommended information.

[0069] "Visualization means" refers to technologies that use augmented reality to provide visitors with visual information about attractions and objects.

[0070] A "feedback accumulation method" is a system used to record opinions and evaluations about the experience received from visitors, in order to improve the accuracy of future suggestions.

[0071] This invention is a system for providing visitors with a facility experience tailored to their interests, and is realized through the interaction of a server, terminals, and users. The following is a detailed description of the system based on this invention.

[0072] First, users install a dedicated application on their smartphones. Based on the facility they are visiting, users set their preferred attractions and animals within the app. For example, when a user visits an aquarium, they can set dolphins as the animal they want to see. This setting information is sent to the server via the device.

[0073] The terminal is the user's smartphone, which uses GPS and sensor technology to collect visitor location information and behavioral data in real time. This data forms the basis for analyzing which attractions were visited and how long visitors stayed in each area during their visit. The collected data is continuously transmitted from the terminal to the server.

[0074] The server receives configuration information and behavioral data sent from the terminal and forms a user profile based on this information. Next, the server uses a generative AI model to analyze the user's behavioral patterns and interests and generate appropriate attraction routes and event information for the visitor. The AI ​​model processes prompts such as, "Suggest the optimal attraction route based on the user's interests. The user is interested in dolphins."

[0075] The generated recommendation information is sent from the server to the device and presented to the user in real time. Furthermore, the device utilizes augmented reality (AR) technology; when a visitor points their smartphone at an attraction or animal, it displays a 3D model and visual information about that object. This allows users to have a visually enriched experience.

[0076] Furthermore, feedback provided by users through the app after their experience is stored on the server. This feedback information is used to improve services for future visitors.

[0077] In this way, the system provides personalized attraction experiences tailored to visitors' interests, thereby increasing the operational efficiency of the facility.

[0078] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0079] Step 1:

[0080] Users input their interests using a dedicated app installed on their smartphones. Within the app, users select items of interest from a list of attractions and animals and set their level of interest. This input information is then used as foundational data for a subsequent recommendation algorithm.

[0081] Step 2:

[0082] The terminal sends the user's input of interest information to the server. The entered data (attractions and animals of interest) is transferred to the server via a communication protocol. The server receives this information and creates a user profile.

[0083] Step 3:

[0084] As users move around the facility, the device uses GPS and sensors to acquire the user's location and behavioral data. For example, it collects information such as which attractions they visited, when they visited them, and how long they stayed there. This data is continuously transmitted to the server.

[0085] Step 4:

[0086] The server analyzes behavioral data received from the terminal. It then uses a generative AI model to identify user behavior patterns and generate recommended routes based on those patterns. This process uses prompts such as, "Please suggest the optimal attraction route based on the user's interests." The analyzed data is used to determine the recommendations.

[0087] Step 5:

[0088] The server generates recommendation information, which is then sent to the terminal and presented to the user. The terminal receives this information and displays it to the user in real time through the user interface. The user can then follow this information to follow an optimized attraction route.

[0089] Step 6:

[0090] The device utilizes augmented reality (AR) technology, allowing visitors to display 3D models and visual information by holding up their smartphones when they visit an attraction. This enables users to obtain visually enriched information on the spot.

[0091] Step 7:

[0092] Users provide feedback through the app after their experience. This feedback includes aspects such as satisfaction with the visited attraction and the time spent there. The device sends this feedback to a server, and the accumulated data is used by the system to improve the accuracy of suggestions for future visits.

[0093] (Application Example 1)

[0094] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0095] In modern families, diverse family members often have different interests and schedules, making it challenging to efficiently plan activities and activities that satisfy everyone. In particular, there is a need for ways to optimize family events and daily schedule adjustments, enabling each family member to spend their time more fulfilling.

[0096] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0097] In this invention, the server includes data acquisition means for collecting visitor behavior information, generation means for analyzing the behavior information and generating activity routes and recommendation information optimized for the visitor, visualization means for presenting visual information using augmented reality technology, and home guidance means for suggesting activity routes optimized based on the interests and schedules of residents in the home. This makes it possible to automatically suggest an optimal activity schedule tailored to the interests of each family member.

[0098] "Data acquisition means" refers to a device or process for collecting visitor behavioral information and interest information.

[0099] "Generation methods" refer to the process of analyzing collected behavioral information to create activity routes and recommendation information optimized for visitors.

[0100] "Information presentation means" refers to a device or process for providing generated recommendation information to visitors in real time.

[0101] "Visualization means" refers to a device or process that uses augmented reality technology to provide visual information to visitors.

[0102] A "household guidance system" is a mechanism for suggesting optimized activity routes and events based on the interests and schedules of residents within a household.

[0103] In the system realizing this invention, the server collects visitor behavior information through data acquisition means. The data acquisition means receives information from sensors and smart devices installed in the home and transmits it to a cloud environment. As hardware, motion sensors and voice recognition devices are used. As software, machine learning APIs such as Google® Cloud ML and AWS® ML are used for data analysis.

[0104] Based on the analyzed data, the server uses a generation mechanism to create activity routes optimized for the individual interests of each family member. The generation mechanism analyzes the behavioral patterns of each visitor using the collected data and forms an interest profile. As a result, it generates basic information for the home guidance mechanism to suggest the most suitable events and activities.

[0105] The terminal provides users with real-time suggested recommendations via information presentation devices. Visualization devices use augmented reality technology to display suggested information clearly to the user. If necessary, the terminal displays visual information on displays installed in each room of the family, providing direct interactive guidance.

[0106] For example, when planning weekend activities, the server analyzes interest profiles and recommends a treasure hunt game in the garden. It takes into account family members' schedules and weather data to suggest the optimal time. Using a generative AI model, it provides detailed recommendations with prompts like: "Based on the current interests of each family member, please suggest the best activity schedule and event information for tomorrow. Please consider individual interests and include new recipes and activities."

[0107] This system allows users to enjoy optimized activity routes tailored to their individual interests, enabling them to spend more fulfilling time at home.

[0108] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0109] Step 1:

[0110] The server collects location and behavioral data of family members from sensors and smart devices within the home via data acquisition methods. The input is raw data from sensors, and the output is a dataset containing location and behavioral information. Data processing involves filtering out redundant data and extracting useful data.

[0111] Step 2:

[0112] The server uses a generation mechanism to analyze the collected data, examining the behavioral patterns and interest profiles of each member. The input is the dataset obtained in step 1, and the output is the interest profile of each member. As a data calculation, a machine learning model is applied to perform specific operations to extract features of the behavioral patterns.

[0113] Step 3:

[0114] The server proposes optimal activity routes and events based on the generated interest profiles. The input is the interest profiles obtained in step 2, and the output is the proposed activity routes and event information. As a data calculation, it uses a generative AI model to generate diverse suggestions based on prompt sentences.

[0115] Step 4:

[0116] The terminal presents suggested information received from the server to the user through an information display mechanism. The input is suggested information from the server, and the output is information visible to the user. Specific operations include the process of displaying text and visual data on the display.

[0117] Step 5:

[0118] The user selects and executes an activity based on the displayed information. The input is the activity route and event information, and the output is the execution of the selected activity. The user's specific action involves selecting a suggested activity and performing the corresponding actions.

[0119] Step 6:

[0120] The server collects feedback on user selections and actions and adds it to a dataset. The input is user feedback, and the output is the updated dataset. Specifically, the feedback is stored in a database and used to optimize future suggestions.

[0121] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0122] This invention relates to a system that provides visitors with an optimized experience in amusement parks and aquariums, and in particular incorporates an emotion engine that recognizes the user's emotions and dynamically adjusts the suggested content. The following is a description of the configuration and operation based on an embodiment of this system.

[0123] Users install a dedicated application on their smartphones and begin using the app. First, the visitor's basic interests and preferences are entered into the app and sent to the server via the device. This process creates a user profile on the server.

[0124] During a visit within the facility, the terminal continuously collects user movement and behavior data in real time. This information is periodically sent to a server, recording the user's movement route and time spent in each location. In addition, the terminal is equipped with an emotion engine that recognizes the user's emotions, using the camera and microphone to analyze the user's facial expressions and voice to evaluate their emotional state.

[0125] The server analyzes visitor behavioral and emotional data and uses generative models to generate optimal attraction routes and recommendations. Furthermore, it dynamically adjusts recommendations based on the emotional state obtained by the emotion engine, providing information and content that matches the visitor's emotions. For example, if a user is excited, it recommends thrilling attractions; if they are relaxed, it suggests leisurely exhibits.

[0126] The terminal provides users with pre-configured information received from the server. This includes elements that use augmented reality technology to visualize detailed information about attractions and exhibits, providing visitors with an interactive and immersive experience.

[0127] This system allows visitors to enjoy experiences tailored to their individual interests and emotional states, leading to increased satisfaction. Furthermore, user feedback is fed back to the server, which is used for further optimization. This enables facilities to achieve efficient management and high visitor satisfaction.

[0128] The following describes the processing flow.

[0129] Step 1:

[0130] Users install the app on their smartphones and enter information about their interests and preferences. For example, users can select their favorite attractions or animals.

[0131] Step 2:

[0132] The device sends the information entered by the user to the server. This information is stored on the server as part of the user profile and used for future personalized suggestions.

[0133] Step 3:

[0134] Once a user begins visiting a facility, the device uses its built-in GPS and sensors to acquire the user's location and behavioral data in real time. This allows the device to understand which attractions the user is visiting.

[0135] Step 4:

[0136] The emotion engine built into the device uses the device's camera and microphone to analyze the user's facial expressions and voice, and evaluate their emotional state. For example, it can detect whether the user is smiling or whether there is excitement in their voice tone.

[0137] Step 5:

[0138] The device continuously transmits location information, behavioral data, and emotional data to the server. The transmitted data is anonymized and processed in a way that protects privacy.

[0139] Step 6:

[0140] The server comprehensively analyzes the acquired data and uses a generative model to generate recommended attraction routes and event information for visitors. This process also includes adjustments based on the user's emotional state.

[0141] Step 7:

[0142] The server sends back adjusted recommendations to the device. This information includes details about attractions and exhibits tailored to the user's current emotional state.

[0143] Step 8:

[0144] The device displays the received information to the user within the app. Furthermore, it uses AR technology to display detailed information about specific attractions or animals in 3D, enhancing the user experience.

[0145] Step 9:

[0146] Users explore the facility using the information provided and enjoy the attractions and exhibits. Users can also provide feedback through the app about their experiences and emotional changes during their visit.

[0147] Step 10:

[0148] The server stores user feedback in a database and uses it to make improvements that enhance the overall accuracy of the system's suggestions. This allows for more effective customization on subsequent visits.

[0149] (Example 2)

[0150] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0151] In amusement parks, aquariums, and other entertainment facilities, there is a problem in that it is difficult to provide visitors with an experience optimized based on their individual interests and emotional states. Furthermore, guiding visitors in real time on how to navigate the facility and increasing their satisfaction is also a challenge. There is a need for systems that can solve these problems and improve the visitor experience.

[0152] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0153] In this invention, the server includes data acquisition means for collecting visitor behavioral and emotional information; generation model means for analyzing the behavioral and emotional information, generating an optimized attraction route and recommendation information for the visitor, and dynamically adjusting it according to the user's emotional state; information presentation means for providing the recommendation information to the visitor in real time and further presenting visual information using augmented reality technology; and feedback collection means for receiving feedback from visitors and using it to improve the accuracy of the analysis results. As a result, visitors can enjoy an optimal experience based on their emotions and interests, and make more meaningful use of the facility.

[0154] "Data acquisition means" refers to the general term for devices and software used to collect visitor behavioral and emotional information.

[0155] "Generative model means" refers to a process and technology for analyzing collected behavioral and emotional information, generating attraction routes and recommendation information optimized for visitors, and dynamically adjusting the information according to the user's emotional state.

[0156] "Information presentation means" refers to devices or methods that provide generated recommendation information to visitors in real time and present visual information using augmented reality technology.

[0157] A "feedback collection method" refers to a function or mechanism used to obtain opinions and evaluation information from visitors and to improve the accuracy of analysis results.

[0158] Augmented reality technology is a technology that overlays digital information onto the real world environment, providing users with a visually enhanced experience.

[0159] The present invention is a system for providing visitors with an optimal experience at amusement facilities such as amusement parks and aquariums. This system includes data acquisition means, generative model means, information presentation means, and feedback collection means.

[0160] First, the terminal uses mobile devices such as smartphones to acquire data. The terminal is equipped with a camera and microphone, which are used to analyze visitors' facial expressions and voices to acquire emotional information. In addition, GPS and Bluetooth technology are used to collect behavioral information such as visitors' location and length of stay.

[0161] The server receives behavioral and emotional information sent from the terminal and analyzes it using a generative AI model. This generative model generates attraction routes and content optimized for the visitor, dynamically adjusting recommendations based on their emotional state. For example, if the user is excited, it recommends thrilling events; if relaxed, it suggests quiet exhibits.

[0162] The generated recommendation information is sent back to the device. The device uses augmented reality technology to display detailed information about the recommended attractions and exhibits overlaid on the visitor's field of view, providing an interactive and immersive experience.

[0163] Users can provide feedback on their experience within the app after their visit. This feedback is then collected and sent to the server via a feedback collection system. The server uses the collected feedback to improve its analysis algorithms, enabling it to provide an even more refined experience on subsequent visits.

[0164] As a concrete example, a possible prompt message for a user visiting an aquarium might be: "A woman in her 20s is currently visiting the aquarium and appears relaxed. Please recommend the next attraction that would be best suited for her." Based on this prompt message, a generative AI model can generate and provide personalized recommendations for the user.

[0165] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0166] Step 1:

[0167] Users install a dedicated app on their smartphones and enter basic information about their interests and preferences at the start. This includes settings such as attraction categories and whether they prefer quiet exhibits. The entered information is sent to a server via the device. This data allows the server to create a visitor profile, which forms the basis for a personalized experience.

[0168] Step 2:

[0169] The device collects real-time data on the user's movements and actions during their visit to the facility. Specifically, it uses GPS and Bluetooth technology to track the user's current location and movement patterns. Furthermore, it uses the camera and microphone built into the device to analyze the user's facial expressions and voice to extract emotional data. This data is periodically transmitted to a server.

[0170] Step 3:

[0171] The server uses a generative AI model to analyze behavioral and emotional information received from the terminal. Based on the input data, it infers the user's current mood and generates the optimal attraction route accordingly. For example, if the server determines that the user is excited, thrilling attractions will be recommended. This information is dynamically adjusted, and the optimal route suggestion is generated as the output result.

[0172] Step 4:

[0173] The recommendation information generated by the AI ​​model is sent from the server to the terminal. The terminal then uses augmented reality technology to overlay detailed information about attractions and exhibits onto the user's field of view. This allows visitors to receive real-time visual information about their next destination and have an interactive experience.

[0174] Step 5:

[0175] After completing their experience at a facility, users can provide feedback on their satisfaction level and areas for improvement within the app. This feedback is sent to a server to help optimize the experience further. The server analyzes this feedback and uses it to improve the accuracy of its analysis algorithms. The final output is a collection of data to further enhance the user's experience on their next visit.

[0176] (Application Example 2)

[0177] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0178] In urban areas, it is difficult for individual citizens and visitors to obtain the optimal experiences and information according to their emotional state and interests at any given time. Therefore, there is a need to analyze visitors' emotions in real time and provide information and routes tailored accordingly. Furthermore, there is a challenge in achieving a more immersive experience by providing visual information using augmented reality technology.

[0179] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0180] In this invention, the server includes data acquisition means for collecting visitor behavior information and emotional states, generation model means for analyzing the behavior information and emotional data and generating optimized information and recommended content, and information presentation means for providing the recommended information to visitors in real time and making adjustments based on their emotions. As a result, visitors can receive experiences and navigation instructions that match their emotions and interests at the time, and by utilizing visual information from augmented reality technology, they can enrich their activities in urban areas.

[0181] A "visitor" is an individual who visits a specific facility or urban area for the purpose of using the system.

[0182] "Behavioral information" refers to data about visitors' activities acquired by the system, such as their travel routes and the amount of time they spend inside the facility.

[0183] "Emotional state" refers to information that indicates a visitor's temporary mental state, analyzed from their facial expressions and voice.

[0184] "Generative modeling means" refers to methods and functions for calculating the most suitable information and content for visitors based on acquired behavioral and emotional data.

[0185] "Information presentation means" refers to methods and technologies for providing visitors with optimized information in real time.

[0186] A "visualization method" refers to a system that uses augmented reality technology to visually present information to visitors.

[0187] Augmented reality technology is a technique that overlays digital information onto real-world scenes.

[0188] This invention is a system that collects visitor behavior information and emotional state and provides an optimized experience based on this information. The server receives data from the visitor's smartphone or wearable device and collects behavioral and emotional data. This data is acquired by the camera and microphone built into the smartphone and analyzed using OpenCV or the Google Cloud Speech-to-Text API.

[0189] The server runs a generative AI model based on the collected data, suggesting attractions, events, and spots suitable for visitors. The generated information is transmitted to the device in real time and adjusted according to the visitor's emotions. Augmented reality technologies such as AR.js are used to present information intuitively.

[0190] The device displays the received optimization information to the visitor in an easy-to-understand manner, providing an experience tailored to their emotional state. For example, if a desire for tranquility is detected, recreational spots such as parks or quiet cafes are suggested. These suggestions are displayed on the screen as digital information alongside real-world scenery, allowing visitors to intuitively navigate to those locations.

[0191] For example, if a visitor is strolling through a city and feels relaxed, the server will suggest quiet and peaceful spots, and the device will guide them there using augmented reality technology. The input prompt for the generating AI model would be: "Generate content to suggest the best recommended spots for when the user is relaxing. Include information about places where they can have a pleasant afternoon."

[0192] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0193] Step 1:

[0194] When a user arrives at their destination, the device uses its camera and microphone to collect visitor behavioral and emotional data. The input is the user's video and audio, while the output is the analyzed behavioral and emotional state. The camera captures facial expressions, OpenCV is used to analyze emotions, and the microphone records audio, which is then converted to text using the Google Cloud Speech-to-Text API for emotion analysis.

[0195] Step 2:

[0196] The device sends the collected data to the server. The server updates the visitor behavior database based on this data. The input is the behavioral information and emotional state analyzed in the previous stage, and the output is the updated database. Specifically, the data is uploaded to the server in real time via a secure communication method.

[0197] Step 3:

[0198] The server runs a generative AI model and analyzes behavioral and emotional data. This generates information and content optimized for the visitor. The input is behavioral data and emotional states stored on the server, and the output is recommended attractions and content. Prompts are used to instruct the generative model to generate appropriate information.

[0199] Step 4:

[0200] The server sends the generated recommendation information back to the device in real time. This provides a list of recommended activities and attractions. The input is the recommendation information output by the generation AI model, and the output is the data packets sent to the device.

[0201] Step 5:

[0202] The device visualizes the received information using augmented reality technology and provides it to the user. This allows the user to intuitively understand where to go next or what events to attend. The input is recommendation information received from the server, and the output is AR content as visual data. AR.js is used to overlay digital information onto the real-world landscape in real time.

[0203] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0204] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0205] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0206] [Second Embodiment]

[0207] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0208] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0209] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0210] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0211] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0212] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0213] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0214] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0215] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0216] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0217] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0218] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0219] This invention relates to a system for providing visitors with an optimized experience in amusement parks and aquariums. The following is a description of the configuration and operation based on an embodiment of this system.

[0220] First, the user installs a dedicated application on their smartphone. When starting to use the app, the user sets their preferred attractions and animals. The device sends this setting information to the server, which then forms the user's profile.

[0221] As visitors move around the facility, the device collects location information and user behavior data in real time. This includes information such as which attractions they visited and how long they stayed in each area. The device continuously transmits this information to the server.

[0222] The server analyzes user behavior patterns and interests based on the received data. A generative model uses this data to create real-time recommended routes for attractions and event information tailored to each visitor. These recommendations are sent from the server to the terminal and presented to the user through the user interface.

[0223] Furthermore, the device uses augmented reality technology to provide detailed visual information about the attractions and animals on display at the visited location. For example, by holding a smartphone over a specific animal, users can visually understand information and the animal's ecology through a 3D model.

[0224] As a result, users can follow attraction routes optimized to their interests, reduce wait times, and efficiently explore the park. Furthermore, the feedback gained through this experience is stored on the server, helping to improve the accuracy of future system suggestions.

[0225] This system aims to enhance visitor satisfaction by providing visitors with enjoyable, efficient, and interest-based information. Furthermore, its efficient traffic flow management technology contributes to the smooth operation of the facility itself.

[0226] The following describes the processing flow.

[0227] Step 1:

[0228] Users install the app on their smartphones and enter information about their interests and preferences. This includes selecting attractions, animals, and other things they are interested in.

[0229] Step 2:

[0230] The device collects information entered by the user and transmits it to a server via the internet. This creates a user profile on the server.

[0231] Step 3:

[0232] Once a user begins visiting a facility, the device uses GPS and internal sensors to collect the user's location and behavioral data in real time. This information shows which areas the user has visited and which attractions they are enjoying.

[0233] Step 4:

[0234] The device periodically sends collected location and behavioral data to the server. This data transmission is processed appropriately to ensure that user data is anonymized.

[0235] Step 5:

[0236] The server analyzes the received data and generates recommendations for optimal attraction routes and events based on the user's visit patterns and interests. Machine learning algorithms are applied to this analysis.

[0237] Step 6:

[0238] The server sends the generated recommendations to the device. This information includes recommended attractions near the current location and routes with shorter wait times.

[0239] Step 7:

[0240] The device displays the received information to the user within the app. It also uses AR technology to overlay detailed visual information about attractions and animals onto the real world.

[0241] Step 8:

[0242] Users navigate the park and enjoy the attractions based on the information provided. They can also input their satisfaction level and feedback into the app.

[0243] Step 9:

[0244] The server receives feedback from users and uses it as data to improve the generative model. This allows for more accurate suggestions to be provided on subsequent visits.

[0245] (Example 1)

[0246] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0247] Ensuring visitors have an efficient and satisfying facility experience tailored to their interests is a challenging problem. Traditional facility guide systems lack the ability to provide personalized experiences based on individual visitors' interests and behavioral patterns, often forcing visitors to navigate unplanned routes in crowded environments. As a result, visitors are unable to fully appreciate the facility's attractions. Furthermore, managing visitor flow becomes difficult for facilities, leading to decreased overall operational efficiency.

[0248] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0249] In this invention, the server includes information input means for visitors to input their interests, data acquisition means for collecting visitor location information and behavioral data in real time, and generative model means for generating attraction routes and recommendation information optimized for visitors using a generative AI model. As a result, visitors can receive optimal attraction routes based on their interests in real time, enabling an efficient and satisfying facility experience. Furthermore, by using this system, the facility can manage visitor flow more effectively and improve overall operational efficiency.

[0250] "Information input means" refers to equipment or interfaces that allow visitors to input their interests and utilize that information in their experience at the facility.

[0251] A "profile creation method" refers to a technique or system for organizing and constructing information related to individual visitors based on their interests and behavioral history.

[0252] "Data acquisition means" refers to technologies and devices for collecting location information and behavioral data of visitors within a facility.

[0253] A "generative modeling system" is a mechanism that analyzes visitors' interests and behavioral patterns and uses a generative AI model to create optimal attraction routes and recommended information.

[0254] "Information presentation means" refers to methods and devices for providing visitors with real-time generated attraction routes and recommended information.

[0255] "Visualization means" refers to technologies that use augmented reality to provide visitors with visual information about attractions and objects.

[0256] A "feedback accumulation method" is a system used to record opinions and evaluations about the experience received from visitors, in order to improve the accuracy of future suggestions.

[0257] This invention is a system for providing visitors with a facility experience tailored to their interests, and is realized through the interaction of a server, terminals, and users. The following is a detailed description of the system based on this invention.

[0258] First, users install a dedicated application on their smartphones. Based on the facility they are visiting, users set their preferred attractions and animals within the app. For example, when a user visits an aquarium, they can set dolphins as the animal they want to see. This setting information is sent to the server via the device.

[0259] The terminal is the user's smartphone, which uses GPS and sensor technology to collect visitor location information and behavioral data in real time. This data forms the basis for analyzing which attractions were visited and how long visitors stayed in each area during their visit. The collected data is continuously transmitted from the terminal to the server.

[0260] The server receives configuration information and behavioral data sent from the terminal and forms a user profile based on this information. Next, the server uses a generative AI model to analyze the user's behavioral patterns and interests and generate appropriate attraction routes and event information for the visitor. The AI ​​model processes prompts such as, "Suggest the optimal attraction route based on the user's interests. The user is interested in dolphins."

[0261] The generated recommendation information is sent from the server to the device and presented to the user in real time. Furthermore, the device utilizes augmented reality (AR) technology; when a visitor points their smartphone at an attraction or animal, it displays a 3D model and visual information about that object. This allows users to have a visually enriched experience.

[0262] Furthermore, feedback provided by users through the app after their experience is stored on the server. This feedback information is used to improve services for future visitors.

[0263] In this way, the system provides personalized attraction experiences tailored to visitors' interests, thereby increasing the operational efficiency of the facility.

[0264] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0265] Step 1:

[0266] Users input their interests using a dedicated app installed on their smartphones. Within the app, users select items of interest from a list of attractions and animals and set their level of interest. This input information is then used as foundational data for a subsequent recommendation algorithm.

[0267] Step 2:

[0268] The terminal sends the user's input of interest information to the server. The entered data (attractions and animals of interest) is transferred to the server via a communication protocol. The server receives this information and creates a user profile.

[0269] Step 3:

[0270] As users move around the facility, the device uses GPS and sensors to acquire the user's location and behavioral data. For example, it collects information such as which attractions they visited, when they visited them, and how long they stayed there. This data is continuously transmitted to the server.

[0271] Step 4:

[0272] The server analyzes behavioral data received from the terminal. It then uses a generative AI model to identify user behavior patterns and generate recommended routes based on those patterns. This process uses prompts such as, "Please suggest the optimal attraction route based on the user's interests." The analyzed data is used to determine the recommendations.

[0273] Step 5:

[0274] The server generates recommendation information, which is then sent to the terminal and presented to the user. The terminal receives this information and displays it to the user in real time through the user interface. The user can then follow this information to follow an optimized attraction route.

[0275] Step 6:

[0276] The device utilizes augmented reality (AR) technology, allowing visitors to display 3D models and visual information by holding up their smartphones when they visit an attraction. This enables users to obtain visually enriched information on the spot.

[0277] Step 7:

[0278] Users provide feedback through the app after their experience. This feedback includes aspects such as satisfaction with the visited attraction and the time spent there. The device sends this feedback to a server, and the accumulated data is used by the system to improve the accuracy of suggestions for future visits.

[0279] (Application Example 1)

[0280] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0281] In modern families, since various family members have different interests and schedules, it is a difficult task to efficiently plan activities and schedules that can satisfy all family members. In particular, there is a need for means to optimize events and daily schedule adjustments within the family so that each member can spend a more fulfilling time.

[0282] The specific processing by the specific processing unit 290 of the data processing apparatus 12 in Application Example 1 is realized by the following respective means.

[0283] In this invention, the server includes a data acquisition means for collecting the behavior information of visitors, a generation means for analyzing the behavior information and generating an activity route and recommended information optimized for the visitors, a visualization means for presenting visual information using augmented reality technology, and a home guidance means for proposing an activity route optimized based on the interests and schedules of the residents in the home. Thereby, it becomes possible to automatically propose an optimal activity schedule according to the interests of individual family members.

[0284] The "data acquisition means" is a device or process for collecting the behavior information and interest information of visitors.

[0285] The "generation means" is a process for analyzing the collected behavior information and creating an activity route and recommended information optimized for the visitors.

[0286] The "information presentation means" is a device or process for providing the generated recommended information to the visitors in real time.

[0287] The "visualization means" is a device or process for providing visual information to the visitors using augmented reality technology.

[0288] The "home guidance means" is a mechanism for proposing an optimized activity route and events based on the interests and schedules of the residents in the home.

[0289] In the system realizing this invention, the server collects visitor behavior information through data acquisition means. The data acquisition means receives information from sensors and smart devices installed in the home and transmits it to a cloud environment. As hardware, motion sensors and voice recognition devices are used. As software, machine learning APIs such as Google Cloud ML and AWS ML are used for data analysis.

[0290] Based on the analyzed data, the server uses a generation mechanism to create activity routes optimized for the individual interests of each family member. The generation mechanism analyzes the behavioral patterns of each visitor using the collected data and forms an interest profile. As a result, it generates basic information for the home guidance mechanism to suggest the most suitable events and activities.

[0291] The terminal provides users with real-time suggested recommendations via information presentation devices. Visualization devices use augmented reality technology to display suggested information clearly to the user. If necessary, the terminal displays visual information on displays installed in each room of the family, providing direct interactive guidance.

[0292] For example, when planning weekend activities, the server analyzes interest profiles and recommends a treasure hunt game in the garden. It takes into account family members' schedules and weather data to suggest the optimal time. Using a generative AI model, it provides detailed recommendations with prompts like: "Based on the current interests of each family member, please suggest the best activity schedule and event information for tomorrow. Please consider individual interests and include new recipes and activities."

[0293] This system allows users to enjoy optimized activity routes tailored to their individual interests, enabling them to spend more fulfilling time at home.

[0294] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0295] Step 1:

[0296] The server collects location and behavioral data of family members from sensors and smart devices within the home via data acquisition methods. The input is raw data from sensors, and the output is a dataset containing location and behavioral information. Data processing involves filtering out redundant data and extracting useful data.

[0297] Step 2:

[0298] The server uses a generation mechanism to analyze the collected data, examining the behavioral patterns and interest profiles of each member. The input is the dataset obtained in step 1, and the output is the interest profile of each member. As a data calculation, a machine learning model is applied to perform specific operations to extract features of the behavioral patterns.

[0299] Step 3:

[0300] The server proposes optimal activity routes and events based on the generated interest profiles. The input is the interest profiles obtained in step 2, and the output is the proposed activity routes and event information. As a data calculation, it uses a generative AI model to generate diverse suggestions based on prompt sentences.

[0301] Step 4:

[0302] The terminal presents suggested information received from the server to the user through an information display mechanism. The input is suggested information from the server, and the output is information visible to the user. Specific operations include the process of displaying text and visual data on the display.

[0303] Step 5:

[0304] The user selects and executes activities based on the displayed information. The input is the activity route and event information, and the output is the execution of the selected activity. As a specific action of the user, the proposed activity is selected and the actions following it are executed.

[0305] Step 6:

[0306] The server collects feedback on the user's selection and executed activities and adds it to the dataset. The input is the user's feedback, and the output is the updated dataset. As a specific action, the feedback is stored in the database and utilized for the optimization of the next proposal.

[0307] Furthermore, an emotion engine for estimating the user's emotion may be combined. That is, the specific processing unit 290 may estimate the user's emotion using the emotion recognition model 59 and perform specific processing using the user's emotion.

[0308] The present invention relates to a system that provides an optimized experience for visitors in amusement parks and aquariums, and particularly incorporates an emotion engine that recognizes the user's emotion and dynamically adjusts the proposed content. The following is an explanation of the configuration and operation based on an embodiment of this system.

[0309] The user installs a dedicated application on the smartphone and starts using the app. First, the basic interests and preferences of the visitor are input into the app and transmitted to the server through the terminal. Through this process, the user's profile is formed on the server.

[0310] During the visit inside the facility, the terminal continuously collects the user's movement and action data in real time. This information is periodically transmitted to the server, and the user's movement route and stay time are recorded. In addition, the terminal is equipped with an emotion engine that recognizes the user's emotion, and analyzes the user's expression and voice using a camera and a microphone to evaluate the emotional state.

[0311] The server analyzes visitor behavioral and emotional data and uses generative models to generate optimal attraction routes and recommendations. Furthermore, it dynamically adjusts recommendations based on the emotional state obtained by the emotion engine, providing information and content that matches the visitor's emotions. For example, if a user is excited, it recommends thrilling attractions; if they are relaxed, it suggests leisurely exhibits.

[0312] The terminal provides users with pre-configured information received from the server. This includes elements that use augmented reality technology to visualize detailed information about attractions and exhibits, providing visitors with an interactive and immersive experience.

[0313] This system allows visitors to enjoy experiences tailored to their individual interests and emotional states, leading to increased satisfaction. Furthermore, user feedback is fed back to the server, which is used for further optimization. This enables facilities to achieve efficient management and high visitor satisfaction.

[0314] The following describes the processing flow.

[0315] Step 1:

[0316] Users install the app on their smartphones and enter information about their interests and preferences. For example, users can select their favorite attractions or animals.

[0317] Step 2:

[0318] The device sends the information entered by the user to the server. This information is stored on the server as part of the user profile and used for future personalized suggestions.

[0319] Step 3:

[0320] Once a user begins visiting a facility, the device uses its built-in GPS and sensors to acquire the user's location and behavioral data in real time. This allows the device to understand which attractions the user is visiting.

[0321] Step 4:

[0322] The emotion engine built into the device uses the device's camera and microphone to analyze the user's facial expressions and voice, and evaluate their emotional state. For example, it can detect whether the user is smiling or whether there is excitement in their voice tone.

[0323] Step 5:

[0324] The device continuously transmits location information, behavioral data, and emotional data to the server. The transmitted data is anonymized and processed in a way that protects privacy.

[0325] Step 6:

[0326] The server comprehensively analyzes the acquired data and uses a generative model to generate recommended attraction routes and event information for visitors. This process also includes adjustments based on the user's emotional state.

[0327] Step 7:

[0328] The server sends back adjusted recommendations to the device. This information includes details about attractions and exhibits tailored to the user's current emotional state.

[0329] Step 8:

[0330] The device displays the received information to the user within the app. Furthermore, it uses AR technology to display detailed information about specific attractions or animals in 3D, enhancing the user experience.

[0331] Step 9:

[0332] Users explore the facility using the information provided and enjoy the attractions and exhibits. Users can also provide feedback through the app about their experiences and emotional changes during their visit.

[0333] Step 10:

[0334] The server stores user feedback in a database and uses it to make improvements that enhance the overall accuracy of the system's suggestions. This allows for more effective customization on subsequent visits.

[0335] (Example 2)

[0336] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0337] In amusement parks, aquariums, and other entertainment facilities, there is a problem in that it is difficult to provide visitors with an experience optimized based on their individual interests and emotional states. Furthermore, guiding visitors in real time on how to navigate the facility and increasing their satisfaction is also a challenge. There is a need for systems that can solve these problems and improve the visitor experience.

[0338] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0339] In this invention, the server includes data acquisition means for collecting visitor behavioral and emotional information; generation model means for analyzing the behavioral and emotional information, generating an optimized attraction route and recommendation information for the visitor, and dynamically adjusting it according to the user's emotional state; information presentation means for providing the recommendation information to the visitor in real time and further presenting visual information using augmented reality technology; and feedback collection means for receiving feedback from visitors and using it to improve the accuracy of the analysis results. As a result, visitors can enjoy an optimal experience based on their emotions and interests, and make more meaningful use of the facility.

[0340] "Data acquisition means" refers to the general term for devices and software used to collect visitor behavioral and emotional information.

[0341] "Generative model means" refers to a process and technology for analyzing collected behavioral and emotional information, generating attraction routes and recommendation information optimized for visitors, and dynamically adjusting the information according to the user's emotional state.

[0342] "Information presentation means" refers to devices or methods that provide generated recommendation information to visitors in real time and present visual information using augmented reality technology.

[0343] A "feedback collection method" refers to a function or mechanism used to obtain opinions and evaluation information from visitors and to improve the accuracy of analysis results.

[0344] Augmented reality technology is a technology that overlays digital information onto the real world environment, providing users with a visually enhanced experience.

[0345] The present invention is a system for providing visitors with an optimal experience at amusement facilities such as amusement parks and aquariums. This system includes data acquisition means, generative model means, information presentation means, and feedback collection means.

[0346] First, the terminal uses mobile devices such as smartphones to acquire data. The terminal is equipped with a camera and microphone, which are used to analyze visitors' facial expressions and voices to acquire emotional information. In addition, GPS and Bluetooth technology are used to collect behavioral information such as visitors' location and length of stay.

[0347] The server receives behavioral and emotional information sent from the terminal and analyzes it using a generative AI model. This generative model generates attraction routes and content optimized for the visitor, dynamically adjusting recommendations based on their emotional state. For example, if the user is excited, it recommends thrilling events; if relaxed, it suggests quiet exhibits.

[0348] The generated recommendation information is sent back to the device. The device uses augmented reality technology to display detailed information about the recommended attractions and exhibits overlaid on the visitor's field of view, providing an interactive and immersive experience.

[0349] Users can provide feedback on their experience within the app after their visit. This feedback is then collected and sent to the server via a feedback collection system. The server uses the collected feedback to improve its analysis algorithms, enabling it to provide an even more refined experience on subsequent visits.

[0350] As a concrete example, a possible prompt message for a user visiting an aquarium might be: "A woman in her 20s is currently visiting the aquarium and appears relaxed. Please recommend the next attraction that would be best suited for her." Based on this prompt message, a generative AI model can generate and provide personalized recommendations for the user.

[0351] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0352] Step 1:

[0353] Users install a dedicated app on their smartphones and enter basic information about their interests and preferences at the start. This includes settings such as attraction categories and whether they prefer quiet exhibits. The entered information is sent to a server via the device. This data allows the server to create a visitor profile, which forms the basis for a personalized experience.

[0354] Step 2:

[0355] The device collects real-time data on the user's movements and actions during their visit to the facility. Specifically, it uses GPS and Bluetooth technology to track the user's current location and movement patterns. Furthermore, it uses the camera and microphone built into the device to analyze the user's facial expressions and voice to extract emotional data. This data is periodically transmitted to a server.

[0356] Step 3:

[0357] The server uses a generative AI model to analyze behavioral and emotional information received from the terminal. Based on the input data, it infers the user's current mood and generates the optimal attraction route accordingly. For example, if the server determines that the user is excited, thrilling attractions will be recommended. This information is dynamically adjusted, and the optimal route suggestion is generated as the output result.

[0358] Step 4:

[0359] The recommendation information generated by the AI ​​model is sent from the server to the terminal. The terminal then uses augmented reality technology to overlay detailed information about attractions and exhibits onto the user's field of view. This allows visitors to receive real-time visual information about their next destination and have an interactive experience.

[0360] Step 5:

[0361] After completing their experience at a facility, users can provide feedback on their satisfaction level and areas for improvement within the app. This feedback is sent to a server to help optimize the experience further. The server analyzes this feedback and uses it to improve the accuracy of its analysis algorithms. The final output is a collection of data to further enhance the user's experience on their next visit.

[0362] (Application Example 2)

[0363] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 as the "terminal".

[0364] In urban areas, it is difficult for individual citizens and visitors to obtain the optimal experiences and information according to their emotional state and interests at any given time. Therefore, there is a need to analyze visitors' emotions in real time and provide information and routes tailored accordingly. Furthermore, there is a challenge in achieving a more immersive experience by providing visual information using augmented reality technology.

[0365] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0366] In this invention, the server includes data acquisition means for collecting visitor behavior information and emotional states, generation model means for analyzing the behavior information and emotional data and generating optimized information and recommended content, and information presentation means for providing the recommended information to visitors in real time and making adjustments based on their emotions. As a result, visitors can receive experiences and navigation instructions that match their emotions and interests at the time, and by utilizing visual information from augmented reality technology, they can enrich their activities in urban areas.

[0367] A "visitor" is an individual who visits a specific facility or urban area for the purpose of using the system.

[0368] "Behavioral information" refers to data about visitors' activities acquired by the system, such as their travel routes and the amount of time they spend inside the facility.

[0369] "Emotional state" refers to information that indicates a visitor's temporary mental state, analyzed from their facial expressions and voice.

[0370] "Generative modeling means" refers to methods and functions for calculating the most suitable information and content for visitors based on acquired behavioral and emotional data.

[0371] "Information presentation means" refers to methods and technologies for providing visitors with optimized information in real time.

[0372] A "visualization method" refers to a system that uses augmented reality technology to visually present information to visitors.

[0373] Augmented reality technology is a technique that overlays digital information onto real-world scenes.

[0374] This invention is a system that collects visitor behavior information and emotional state and provides an optimized experience based on this information. The server receives data from the visitor's smartphone or wearable device and collects behavioral and emotional data. This data is acquired by the camera and microphone built into the smartphone and analyzed using OpenCV or the Google Cloud Speech-to-Text API.

[0375] The server runs a generative AI model based on the collected data, suggesting attractions, events, and spots suitable for visitors. The generated information is transmitted to the device in real time and adjusted according to the visitor's emotions. Augmented reality technologies such as AR.js are used to present information intuitively.

[0376] The device displays the received optimization information to the visitor in an easy-to-understand manner, providing an experience tailored to their emotional state. For example, if a desire for tranquility is detected, recreational spots such as parks or quiet cafes are suggested. These suggestions are displayed on the screen as digital information alongside real-world scenery, allowing visitors to intuitively navigate to those locations.

[0377] For example, if a visitor is strolling through a city and feels relaxed, the server will suggest quiet and peaceful spots, and the device will guide them there using augmented reality technology. The input prompt for the generating AI model would be: "Generate content to suggest the best recommended spots for when the user is relaxing. Include information about places where they can have a pleasant afternoon."

[0378] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0379] Step 1:

[0380] When a user arrives at their destination, the device uses its camera and microphone to collect visitor behavioral and emotional data. The input is the user's video and audio, while the output is the analyzed behavioral and emotional state. The camera captures facial expressions, OpenCV is used to analyze emotions, and the microphone records audio, which is then converted to text using the Google Cloud Speech-to-Text API for emotion analysis.

[0381] Step 2:

[0382] The device sends the collected data to the server. The server updates the visitor behavior database based on this data. The input is the behavioral information and emotional state analyzed in the previous stage, and the output is the updated database. Specifically, the data is uploaded to the server in real time via a secure communication method.

[0383] Step 3:

[0384] The server runs a generative AI model and analyzes behavioral and emotional data. This generates information and content optimized for the visitor. The input is behavioral data and emotional states stored on the server, and the output is recommended attractions and content. Prompts are used to instruct the generative model to generate appropriate information.

[0385] Step 4:

[0386] The server sends the generated recommendation information back to the device in real time. This provides a list of recommended activities and attractions. The input is the recommendation information output by the generation AI model, and the output is the data packets sent to the device.

[0387] Step 5:

[0388] The device visualizes the received information using augmented reality technology and provides it to the user. This allows the user to intuitively understand where to go next or what events to attend. The input is recommendation information received from the server, and the output is AR content as visual data. AR.js is used to overlay digital information onto the real-world landscape in real time.

[0389] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0390] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0391] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0392] [Third Embodiment]

[0393] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0394] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0395] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0396] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0397] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0398] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0399] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0400] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0401] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0402] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0403] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0404] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0405] This invention relates to a system for providing visitors with an optimized experience in amusement parks and aquariums. The following is a description of the configuration and operation based on an embodiment of this system.

[0406] First, the user installs a dedicated application on their smartphone. When starting to use the app, the user sets their preferred attractions and animals. The device sends this setting information to the server, which then forms the user's profile.

[0407] As visitors move around the facility, the device collects location information and user behavior data in real time. This includes information such as which attractions they visited and how long they stayed in each area. The device continuously transmits this information to the server.

[0408] The server analyzes user behavior patterns and interests based on the received data. A generative model uses this data to create real-time recommended routes for attractions and event information tailored to each visitor. These recommendations are sent from the server to the terminal and presented to the user through the user interface.

[0409] Furthermore, the device uses augmented reality technology to provide detailed visual information about the attractions and animals on display at the visited location. For example, by holding a smartphone over a specific animal, users can visually understand information and the animal's ecology through a 3D model.

[0410] As a result, users can follow attraction routes optimized to their interests, reduce wait times, and efficiently explore the park. Furthermore, the feedback gained through this experience is stored on the server, helping to improve the accuracy of future system suggestions.

[0411] This system aims to enhance visitor satisfaction by providing visitors with enjoyable, efficient, and interest-based information. Furthermore, its efficient traffic flow management technology contributes to the smooth operation of the facility itself.

[0412] The following describes the processing flow.

[0413] Step 1:

[0414] Users install the app on their smartphones and enter information about their interests and preferences. This includes selecting attractions, animals, and other things they are interested in.

[0415] Step 2:

[0416] The device collects information entered by the user and transmits it to a server via the internet. This creates a user profile on the server.

[0417] Step 3:

[0418] Once a user begins visiting a facility, the device uses GPS and internal sensors to collect the user's location and behavioral data in real time. This information shows which areas the user has visited and which attractions they are enjoying.

[0419] Step 4:

[0420] The device periodically sends collected location and behavioral data to the server. This data transmission is processed appropriately to ensure that user data is anonymized.

[0421] Step 5:

[0422] The server analyzes the received data and generates recommendations for optimal attraction routes and events based on the user's visit patterns and interests. Machine learning algorithms are applied to this analysis.

[0423] Step 6:

[0424] The server sends the generated recommendations to the device. This information includes recommended attractions near the current location and routes with shorter wait times.

[0425] Step 7:

[0426] The device displays the received information to the user within the app. It also uses AR technology to overlay detailed visual information about attractions and animals onto the real world.

[0427] Step 8:

[0428] Users navigate the park and enjoy the attractions based on the information provided. They can also input their satisfaction level and feedback into the app.

[0429] Step 9:

[0430] The server receives feedback from users and uses it as data to improve the generative model. This allows for more accurate suggestions to be provided on subsequent visits.

[0431] (Example 1)

[0432] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0433] Ensuring visitors have an efficient and satisfying facility experience tailored to their interests is a challenging problem. Traditional facility guide systems lack the ability to provide personalized experiences based on individual visitors' interests and behavioral patterns, often forcing visitors to navigate unplanned routes in crowded environments. As a result, visitors are unable to fully appreciate the facility's attractions. Furthermore, managing visitor flow becomes difficult for facilities, leading to decreased overall operational efficiency.

[0434] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0435] In this invention, the server includes information input means for visitors to input their interests, data acquisition means for collecting visitor location information and behavioral data in real time, and generative model means for generating attraction routes and recommendation information optimized for visitors using a generative AI model. As a result, visitors can receive optimal attraction routes based on their interests in real time, enabling an efficient and satisfying facility experience. Furthermore, by using this system, the facility can manage visitor flow more effectively and improve overall operational efficiency.

[0436] "Information input means" refers to equipment or interfaces that allow visitors to input their interests and utilize that information in their experience at the facility.

[0437] A "profile creation method" refers to a technique or system for organizing and constructing information related to individual visitors based on their interests and behavioral history.

[0438] "Data acquisition means" refers to technologies and devices for collecting location information and behavioral data of visitors within a facility.

[0439] A "generative modeling system" is a mechanism that analyzes visitors' interests and behavioral patterns and uses a generative AI model to create optimal attraction routes and recommended information.

[0440] "Information presentation means" refers to methods and devices for providing visitors with real-time generated attraction routes and recommended information.

[0441] "Visualization means" refers to technologies that use augmented reality to provide visitors with visual information about attractions and objects.

[0442] A "feedback accumulation method" is a system used to record opinions and evaluations about the experience received from visitors, in order to improve the accuracy of future suggestions.

[0443] This invention is a system for providing visitors with a facility experience tailored to their interests, and is realized through the interaction of a server, terminals, and users. The following is a detailed description of the system based on this invention.

[0444] First, users install a dedicated application on their smartphones. Based on the facility they are visiting, users set their preferred attractions and animals within the app. For example, when a user visits an aquarium, they can set dolphins as the animal they want to see. This setting information is sent to the server via the device.

[0445] The terminal is the user's smartphone, which uses GPS and sensor technology to collect visitor location information and behavioral data in real time. This data forms the basis for analyzing which attractions were visited and how long visitors stayed in each area during their visit. The collected data is continuously transmitted from the terminal to the server.

[0446] The server receives configuration information and behavioral data sent from the terminal and forms a user profile based on this information. Next, the server uses a generative AI model to analyze the user's behavioral patterns and interests and generate appropriate attraction routes and event information for the visitor. The AI ​​model processes prompts such as, "Suggest the optimal attraction route based on the user's interests. The user is interested in dolphins."

[0447] The generated recommendation information is sent from the server to the device and presented to the user in real time. Furthermore, the device utilizes augmented reality (AR) technology; when a visitor points their smartphone at an attraction or animal, it displays a 3D model and visual information about that object. This allows users to have a visually enriched experience.

[0448] Furthermore, feedback provided by users through the app after their experience is stored on the server. This feedback information is used to improve services for future visitors.

[0449] In this way, the system provides personalized attraction experiences tailored to visitors' interests, thereby increasing the operational efficiency of the facility.

[0450] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0451] Step 1:

[0452] Users input their interests using a dedicated app installed on their smartphones. Within the app, users select items of interest from a list of attractions and animals and set their level of interest. This input information is then used as foundational data for a subsequent recommendation algorithm.

[0453] Step 2:

[0454] The terminal sends the user's input of interest information to the server. The entered data (attractions and animals of interest) is transferred to the server via a communication protocol. The server receives this information and creates a user profile.

[0455] Step 3:

[0456] As users move around the facility, the device uses GPS and sensors to acquire the user's location and behavioral data. For example, it collects information such as which attractions they visited, when they visited them, and how long they stayed there. This data is continuously transmitted to the server.

[0457] Step 4:

[0458] The server analyzes behavioral data received from the terminal. It then uses a generative AI model to identify user behavior patterns and generate recommended routes based on those patterns. This process uses prompts such as, "Please suggest the optimal attraction route based on the user's interests." The analyzed data is used to determine the recommendations.

[0459] Step 5:

[0460] The server generates recommendation information, which is then sent to the terminal and presented to the user. The terminal receives this information and displays it to the user in real time through the user interface. The user can then follow this information to follow an optimized attraction route.

[0461] Step 6:

[0462] The device utilizes augmented reality (AR) technology, allowing visitors to display 3D models and visual information by holding up their smartphones when they visit an attraction. This enables users to obtain visually enriched information on the spot.

[0463] Step 7:

[0464] Users provide feedback through the app after their experience. This feedback includes aspects such as satisfaction with the visited attraction and the time spent there. The device sends this feedback to a server, and the accumulated data is used by the system to improve the accuracy of suggestions for future visits.

[0465] (Application Example 1)

[0466] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0467] In modern families, diverse family members often have different interests and schedules, making it challenging to efficiently plan activities and activities that satisfy everyone. In particular, there is a need for ways to optimize family events and daily schedule adjustments, enabling each family member to spend their time more fulfilling.

[0468] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0469] In this invention, the server includes data acquisition means for collecting visitor behavior information, generation means for analyzing the behavior information and generating activity routes and recommendation information optimized for the visitor, visualization means for presenting visual information using augmented reality technology, and home guidance means for suggesting activity routes optimized based on the interests and schedules of residents in the home. This makes it possible to automatically suggest an optimal activity schedule tailored to the interests of each family member.

[0470] "Data acquisition means" refers to a device or process for collecting visitor behavioral information and interest information.

[0471] "Generation methods" refer to the process of analyzing collected behavioral information to create activity routes and recommendation information optimized for visitors.

[0472] "Information presentation means" refers to a device or process for providing generated recommendation information to visitors in real time.

[0473] "Visualization means" refers to a device or process that uses augmented reality technology to provide visual information to visitors.

[0474] A "household guidance system" is a mechanism for suggesting optimized activity routes and events based on the interests and schedules of residents within a household.

[0475] In the system realizing this invention, the server collects visitor behavior information through data acquisition means. The data acquisition means receives information from sensors and smart devices installed in the home and transmits it to a cloud environment. As hardware, motion sensors and voice recognition devices are used. As software, machine learning APIs such as Google Cloud ML and AWS ML are used for data analysis.

[0476] Based on the analyzed data, the server uses a generation mechanism to create activity routes optimized for the individual interests of each family member. The generation mechanism analyzes the behavioral patterns of each visitor using the collected data and forms an interest profile. As a result, it generates basic information for the home guidance mechanism to suggest the most suitable events and activities.

[0477] The terminal provides users with real-time suggested recommendations via information presentation devices. Visualization devices use augmented reality technology to display suggested information clearly to the user. If necessary, the terminal displays visual information on displays installed in each room of the family, providing direct interactive guidance.

[0478] For example, when planning weekend activities, the server analyzes interest profiles and recommends a treasure hunt game in the garden. It takes into account family members' schedules and weather data to suggest the optimal time. Using a generative AI model, it provides detailed recommendations with prompts like: "Based on the current interests of each family member, please suggest the best activity schedule and event information for tomorrow. Please consider individual interests and include new recipes and activities."

[0479] This system allows users to enjoy optimized activity routes tailored to their individual interests, enabling them to spend more fulfilling time at home.

[0480] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0481] Step 1:

[0482] The server collects location and behavioral data of family members from sensors and smart devices within the home via data acquisition methods. The input is raw data from sensors, and the output is a dataset containing location and behavioral information. Data processing involves filtering out redundant data and extracting useful data.

[0483] Step 2:

[0484] The server uses a generation mechanism to analyze the collected data, examining the behavioral patterns and interest profiles of each member. The input is the dataset obtained in step 1, and the output is the interest profile of each member. As a data calculation, a machine learning model is applied to perform specific operations to extract features of the behavioral patterns.

[0485] Step 3:

[0486] The server proposes optimal activity routes and events based on the generated interest profiles. The input is the interest profiles obtained in step 2, and the output is the proposed activity routes and event information. As a data calculation, it uses a generative AI model to generate diverse suggestions based on prompt sentences.

[0487] Step 4:

[0488] The terminal presents suggested information received from the server to the user through an information display mechanism. The input is suggested information from the server, and the output is information visible to the user. Specific operations include the process of displaying text and visual data on the display.

[0489] Step 5:

[0490] The user selects and executes an activity based on the displayed information. The input is the activity route and event information, and the output is the execution of the selected activity. The user's specific action involves selecting a suggested activity and performing the corresponding actions.

[0491] Step 6:

[0492] The server collects feedback on user selections and actions and adds it to a dataset. The input is user feedback, and the output is the updated dataset. Specifically, the feedback is stored in a database and used to optimize future suggestions.

[0493] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0494] This invention relates to a system that provides visitors with an optimized experience in amusement parks and aquariums, and in particular incorporates an emotion engine that recognizes the user's emotions and dynamically adjusts the suggested content. The following is a description of the configuration and operation based on an embodiment of this system.

[0495] Users install a dedicated application on their smartphones and begin using the app. First, the visitor's basic interests and preferences are entered into the app and sent to the server via the device. This process creates a user profile on the server.

[0496] During a visit within the facility, the terminal continuously collects user movement and behavior data in real time. This information is periodically sent to a server, recording the user's movement route and time spent in each location. In addition, the terminal is equipped with an emotion engine that recognizes the user's emotions, using the camera and microphone to analyze the user's facial expressions and voice to evaluate their emotional state.

[0497] The server analyzes visitor behavioral and emotional data and uses generative models to generate optimal attraction routes and recommendations. Furthermore, it dynamically adjusts recommendations based on the emotional state obtained by the emotion engine, providing information and content that matches the visitor's emotions. For example, if a user is excited, it recommends thrilling attractions; if they are relaxed, it suggests leisurely exhibits.

[0498] The terminal provides users with pre-configured information received from the server. This includes elements that use augmented reality technology to visualize detailed information about attractions and exhibits, providing visitors with an interactive and immersive experience.

[0499] This system allows visitors to enjoy experiences tailored to their individual interests and emotional states, leading to increased satisfaction. Furthermore, user feedback is fed back to the server, which is used for further optimization. This enables facilities to achieve efficient management and high visitor satisfaction.

[0500] The following describes the processing flow.

[0501] Step 1:

[0502] Users install the app on their smartphones and enter information about their interests and preferences. For example, users can select their favorite attractions or animals.

[0503] Step 2:

[0504] The device sends the information entered by the user to the server. This information is stored on the server as part of the user profile and used for future personalized suggestions.

[0505] Step 3:

[0506] Once a user begins visiting a facility, the device uses its built-in GPS and sensors to acquire the user's location and behavioral data in real time. This allows the device to understand which attractions the user is visiting.

[0507] Step 4:

[0508] The emotion engine built into the device uses the device's camera and microphone to analyze the user's facial expressions and voice, and evaluate their emotional state. For example, it can detect whether the user is smiling or whether there is excitement in their voice tone.

[0509] Step 5:

[0510] The device continuously transmits location information, behavioral data, and emotional data to the server. The transmitted data is anonymized and processed in a way that protects privacy.

[0511] Step 6:

[0512] The server comprehensively analyzes the acquired data and uses a generative model to generate recommended attraction routes and event information for visitors. This process also includes adjustments based on the user's emotional state.

[0513] Step 7:

[0514] The server sends back adjusted recommendations to the device. This information includes details about attractions and exhibits tailored to the user's current emotional state.

[0515] Step 8:

[0516] The device displays the received information to the user within the app. Furthermore, it uses AR technology to display detailed information about specific attractions or animals in 3D, enhancing the user experience.

[0517] Step 9:

[0518] Users explore the facility using the information provided and enjoy the attractions and exhibits. Users can also provide feedback through the app about their experiences and emotional changes during their visit.

[0519] Step 10:

[0520] The server stores user feedback in a database and uses it to make improvements that enhance the overall accuracy of the system's suggestions. This allows for more effective customization on subsequent visits.

[0521] (Example 2)

[0522] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0523] In amusement parks, aquariums, and other entertainment facilities, there is a problem in that it is difficult to provide visitors with an experience optimized based on their individual interests and emotional states. Furthermore, guiding visitors in real time on how to navigate the facility and increasing their satisfaction is also a challenge. There is a need for systems that can solve these problems and improve the visitor experience.

[0524] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0525] In this invention, the server includes data acquisition means for collecting visitor behavioral and emotional information; generation model means for analyzing the behavioral and emotional information, generating an optimized attraction route and recommendation information for the visitor, and dynamically adjusting it according to the user's emotional state; information presentation means for providing the recommendation information to the visitor in real time and further presenting visual information using augmented reality technology; and feedback collection means for receiving feedback from visitors and using it to improve the accuracy of the analysis results. As a result, visitors can enjoy an optimal experience based on their emotions and interests, and make more meaningful use of the facility.

[0526] "Data acquisition means" refers to the general term for devices and software used to collect visitor behavioral and emotional information.

[0527] "Generative model means" refers to a process and technology for analyzing collected behavioral and emotional information, generating attraction routes and recommendation information optimized for visitors, and dynamically adjusting the information according to the user's emotional state.

[0528] "Information presentation means" refers to devices or methods that provide generated recommendation information to visitors in real time and present visual information using augmented reality technology.

[0529] A "feedback collection method" refers to a function or mechanism used to obtain opinions and evaluation information from visitors and to improve the accuracy of analysis results.

[0530] Augmented reality technology is a technology that overlays digital information onto the real world environment, providing users with a visually enhanced experience.

[0531] The present invention is a system for providing visitors with an optimal experience at amusement facilities such as amusement parks and aquariums. This system includes data acquisition means, generative model means, information presentation means, and feedback collection means.

[0532] First, the terminal uses mobile devices such as smartphones to acquire data. The terminal is equipped with a camera and microphone, which are used to analyze visitors' facial expressions and voices to acquire emotional information. In addition, GPS and Bluetooth technology are used to collect behavioral information such as visitors' location and length of stay.

[0533] The server receives behavioral and emotional information sent from the terminal and analyzes it using a generative AI model. This generative model generates attraction routes and content optimized for the visitor, dynamically adjusting recommendations based on their emotional state. For example, if the user is excited, it recommends thrilling events; if relaxed, it suggests quiet exhibits.

[0534] The generated recommendation information is sent back to the device. The device uses augmented reality technology to display detailed information about the recommended attractions and exhibits overlaid on the visitor's field of view, providing an interactive and immersive experience.

[0535] Users can provide feedback on their experience within the app after their visit. This feedback is then collected and sent to the server via a feedback collection system. The server uses the collected feedback to improve its analysis algorithms, enabling it to provide an even more refined experience on subsequent visits.

[0536] As a concrete example, a possible prompt message for a user visiting an aquarium might be: "A woman in her 20s is currently visiting the aquarium and appears relaxed. Please recommend the next attraction that would be best suited for her." Based on this prompt message, a generative AI model can generate and provide personalized recommendations for the user.

[0537] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0538] Step 1:

[0539] Users install a dedicated app on their smartphones and enter basic information about their interests and preferences at the start. This includes settings such as attraction categories and whether they prefer quiet exhibits. The entered information is sent to a server via the device. This data allows the server to create a visitor profile, which forms the basis for a personalized experience.

[0540] Step 2:

[0541] The device collects real-time data on the user's movements and actions during their visit to the facility. Specifically, it uses GPS and Bluetooth technology to track the user's current location and movement patterns. Furthermore, it uses the camera and microphone built into the device to analyze the user's facial expressions and voice to extract emotional data. This data is periodically transmitted to a server.

[0542] Step 3:

[0543] The server uses a generative AI model to analyze behavioral and emotional information received from the terminal. Based on the input data, it infers the user's current mood and generates the optimal attraction route accordingly. For example, if the server determines that the user is excited, thrilling attractions will be recommended. This information is dynamically adjusted, and the optimal route suggestion is generated as the output result.

[0544] Step 4:

[0545] The recommendation information generated by the AI ​​model is sent from the server to the terminal. The terminal then uses augmented reality technology to overlay detailed information about attractions and exhibits onto the user's field of view. This allows visitors to receive real-time visual information about their next destination and have an interactive experience.

[0546] Step 5:

[0547] After completing their experience at a facility, users can provide feedback on their satisfaction level and areas for improvement within the app. This feedback is sent to a server to help optimize the experience further. The server analyzes this feedback and uses it to improve the accuracy of its analysis algorithms. The final output is a collection of data to further enhance the user's experience on their next visit.

[0548] (Application Example 2)

[0549] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0550] In urban areas, it is difficult for individual citizens and visitors to obtain the optimal experiences and information according to their emotional state and interests at any given time. Therefore, there is a need to analyze visitors' emotions in real time and provide information and routes tailored accordingly. Furthermore, there is a challenge in achieving a more immersive experience by providing visual information using augmented reality technology.

[0551] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0552] In this invention, the server includes data acquisition means for collecting visitor behavior information and emotional states, generation model means for analyzing the behavior information and emotional data and generating optimized information and recommended content, and information presentation means for providing the recommended information to visitors in real time and making adjustments based on their emotions. As a result, visitors can receive experiences and navigation instructions that match their emotions and interests at the time, and by utilizing visual information from augmented reality technology, they can enrich their activities in urban areas.

[0553] A "visitor" is an individual who visits a specific facility or urban area for the purpose of using the system.

[0554] "Behavioral information" refers to data about visitors' activities acquired by the system, such as their travel routes and the amount of time they spend inside the facility.

[0555] "Emotional state" refers to information that indicates a visitor's temporary mental state, analyzed from their facial expressions and voice.

[0556] "Generative modeling means" refers to methods and functions for calculating the most suitable information and content for visitors based on acquired behavioral and emotional data.

[0557] "Information presentation means" refers to methods and technologies for providing visitors with optimized information in real time.

[0558] A "visualization method" refers to a system that uses augmented reality technology to visually present information to visitors.

[0559] Augmented reality technology is a technique that overlays digital information onto real-world scenes.

[0560] This invention is a system that collects visitor behavior information and emotional state and provides an optimized experience based on this information. The server receives data from the visitor's smartphone or wearable device and collects behavioral and emotional data. This data is acquired by the camera and microphone built into the smartphone and analyzed using OpenCV or the Google Cloud Speech-to-Text API.

[0561] The server runs a generative AI model based on the collected data, suggesting attractions, events, and spots suitable for visitors. The generated information is transmitted to the device in real time and adjusted according to the visitor's emotions. Augmented reality technologies such as AR.js are used to present information intuitively.

[0562] The device displays the received optimization information to the visitor in an easy-to-understand manner, providing an experience tailored to their emotional state. For example, if a desire for tranquility is detected, recreational spots such as parks or quiet cafes are suggested. These suggestions are displayed on the screen as digital information alongside real-world scenery, allowing visitors to intuitively navigate to those locations.

[0563] For example, if a visitor is strolling through a city and feels relaxed, the server will suggest quiet and peaceful spots, and the device will guide them there using augmented reality technology. The input prompt for the generating AI model would be: "Generate content to suggest the best recommended spots for when the user is relaxing. Include information about places where they can have a pleasant afternoon."

[0564] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0565] Step 1:

[0566] When a user arrives at their destination, the device uses its camera and microphone to collect visitor behavioral and emotional data. The input is the user's video and audio, while the output is the analyzed behavioral and emotional state. The camera captures facial expressions, OpenCV is used to analyze emotions, and the microphone records audio, which is then converted to text using the Google Cloud Speech-to-Text API for emotion analysis.

[0567] Step 2:

[0568] The device sends the collected data to the server. The server updates the visitor behavior database based on this data. The input is the behavioral information and emotional state analyzed in the previous stage, and the output is the updated database. Specifically, the data is uploaded to the server in real time via a secure communication method.

[0569] Step 3:

[0570] The server runs a generative AI model and analyzes behavioral and emotional data. This generates information and content optimized for the visitor. The input is behavioral data and emotional states stored on the server, and the output is recommended attractions and content. Prompts are used to instruct the generative model to generate appropriate information.

[0571] Step 4:

[0572] The server sends the generated recommendation information back to the device in real time. This provides a list of recommended activities and attractions. The input is the recommendation information output by the generation AI model, and the output is the data packets sent to the device.

[0573] Step 5:

[0574] The device visualizes the received information using augmented reality technology and provides it to the user. This allows the user to intuitively understand where to go next or what events to attend. The input is recommendation information received from the server, and the output is AR content as visual data. AR.js is used to overlay digital information onto the real-world landscape in real time.

[0575] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0576] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0577] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0578] [Fourth Embodiment]

[0579] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0580] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0581] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0582] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0583] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0584] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0585] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0586] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0587] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0588] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0589] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0590] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0591] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0592] This invention relates to a system for providing visitors with an optimized experience in amusement parks and aquariums. The following is a description of the configuration and operation based on an embodiment of this system.

[0593] First, the user installs a dedicated application on their smartphone. When starting to use the app, the user sets their preferred attractions and animals. The device sends this setting information to the server, which then forms the user's profile.

[0594] As visitors move around the facility, the device collects location information and user behavior data in real time. This includes information such as which attractions they visited and how long they stayed in each area. The device continuously transmits this information to the server.

[0595] The server analyzes user behavior patterns and interests based on the received data. A generative model uses this data to create real-time recommended routes for attractions and event information tailored to each visitor. These recommendations are sent from the server to the terminal and presented to the user through the user interface.

[0596] Furthermore, the device uses augmented reality technology to provide detailed visual information about the attractions and animals on display at the visited location. For example, by holding a smartphone over a specific animal, users can visually understand information and the animal's ecology through a 3D model.

[0597] As a result, users can follow attraction routes optimized to their interests, reduce wait times, and efficiently explore the park. Furthermore, the feedback gained through this experience is stored on the server, helping to improve the accuracy of future system suggestions.

[0598] This system aims to enhance visitor satisfaction by providing visitors with enjoyable, efficient, and interest-based information. Furthermore, its efficient traffic flow management technology contributes to the smooth operation of the facility itself.

[0599] The following describes the processing flow.

[0600] Step 1:

[0601] Users install the app on their smartphones and enter information about their interests and preferences. This includes selecting attractions, animals, and other things they are interested in.

[0602] Step 2:

[0603] The device collects information entered by the user and transmits it to a server via the internet. This creates a user profile on the server.

[0604] Step 3:

[0605] Once a user begins visiting a facility, the device uses GPS and internal sensors to collect the user's location and behavioral data in real time. This information shows which areas the user has visited and which attractions they are enjoying.

[0606] Step 4:

[0607] The device periodically sends collected location and behavioral data to the server. This data transmission is processed appropriately to ensure that user data is anonymized.

[0608] Step 5:

[0609] The server analyzes the received data and generates recommendations for optimal attraction routes and events based on the user's visit patterns and interests. Machine learning algorithms are applied to this analysis.

[0610] Step 6:

[0611] The server sends the generated recommendations to the device. This information includes recommended attractions near the current location and routes with shorter wait times.

[0612] Step 7:

[0613] The device displays the received information to the user within the app. It also uses AR technology to overlay detailed visual information about attractions and animals onto the real world.

[0614] Step 8:

[0615] Users navigate the park and enjoy the attractions based on the information provided. They can also input their satisfaction level and feedback into the app.

[0616] Step 9:

[0617] The server receives feedback from users and uses it as data to improve the generative model. This allows for more accurate suggestions to be provided on subsequent visits.

[0618] (Example 1)

[0619] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0620] Ensuring visitors have an efficient and satisfying facility experience tailored to their interests is a challenging problem. Traditional facility guide systems lack the ability to provide personalized experiences based on individual visitors' interests and behavioral patterns, often forcing visitors to navigate unplanned routes in crowded environments. As a result, visitors are unable to fully appreciate the facility's attractions. Furthermore, managing visitor flow becomes difficult for facilities, leading to decreased overall operational efficiency.

[0621] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0622] In this invention, the server includes information input means for visitors to input their interests, data acquisition means for collecting visitor location information and behavioral data in real time, and generative model means for generating attraction routes and recommendation information optimized for visitors using a generative AI model. As a result, visitors can receive optimal attraction routes based on their interests in real time, enabling an efficient and satisfying facility experience. Furthermore, by using this system, the facility can manage visitor flow more effectively and improve overall operational efficiency.

[0623] "Information input means" refers to equipment or interfaces that allow visitors to input their interests and utilize that information in their experience at the facility.

[0624] A "profile creation method" refers to a technique or system for organizing and constructing information related to individual visitors based on their interests and behavioral history.

[0625] "Data acquisition means" refers to technologies and devices for collecting location information and behavioral data of visitors within a facility.

[0626] A "generative modeling system" is a mechanism that analyzes visitors' interests and behavioral patterns and uses a generative AI model to create optimal attraction routes and recommended information.

[0627] "Information presentation means" refers to methods and devices for providing visitors with real-time generated attraction routes and recommended information.

[0628] "Visualization means" refers to technologies that use augmented reality to provide visitors with visual information about attractions and objects.

[0629] A "feedback accumulation method" is a system used to record opinions and evaluations about the experience received from visitors, in order to improve the accuracy of future suggestions.

[0630] This invention is a system for providing visitors with a facility experience tailored to their interests, and is realized through the interaction of a server, terminals, and users. The following is a detailed description of the system based on this invention.

[0631] First, users install a dedicated application on their smartphones. Based on the facility they are visiting, users set their preferred attractions and animals within the app. For example, when a user visits an aquarium, they can set dolphins as the animal they want to see. This setting information is sent to the server via the device.

[0632] The terminal is the user's smartphone, which uses GPS and sensor technology to collect visitor location information and behavioral data in real time. This data forms the basis for analyzing which attractions were visited and how long visitors stayed in each area during their visit. The collected data is continuously transmitted from the terminal to the server.

[0633] The server receives configuration information and behavioral data sent from the terminal and forms a user profile based on this information. Next, the server uses a generative AI model to analyze the user's behavioral patterns and interests and generate appropriate attraction routes and event information for the visitor. The AI ​​model processes prompts such as, "Suggest the optimal attraction route based on the user's interests. The user is interested in dolphins."

[0634] The generated recommendation information is sent from the server to the device and presented to the user in real time. Furthermore, the device utilizes augmented reality (AR) technology; when a visitor points their smartphone at an attraction or animal, it displays a 3D model and visual information about that object. This allows users to have a visually enriched experience.

[0635] Furthermore, feedback provided by users through the app after their experience is stored on the server. This feedback information is used to improve services for future visitors.

[0636] In this way, the system provides personalized attraction experiences tailored to visitors' interests, thereby increasing the operational efficiency of the facility.

[0637] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0638] Step 1:

[0639] Users input their interests using a dedicated app installed on their smartphones. Within the app, users select items of interest from a list of attractions and animals and set their level of interest. This input information is then used as foundational data for a subsequent recommendation algorithm.

[0640] Step 2:

[0641] The terminal sends the user's input of interest information to the server. The entered data (attractions and animals of interest) is transferred to the server via a communication protocol. The server receives this information and creates a user profile.

[0642] Step 3:

[0643] As users move around the facility, the device uses GPS and sensors to acquire the user's location and behavioral data. For example, it collects information such as which attractions they visited, when they visited them, and how long they stayed there. This data is continuously transmitted to the server.

[0644] Step 4:

[0645] The server analyzes behavioral data received from the terminal. It then uses a generative AI model to identify user behavior patterns and generate recommended routes based on those patterns. This process uses prompts such as, "Please suggest the optimal attraction route based on the user's interests." The analyzed data is used to determine the recommendations.

[0646] Step 5:

[0647] The server generates recommendation information, which is then sent to the terminal and presented to the user. The terminal receives this information and displays it to the user in real time through the user interface. The user can then follow this information to follow an optimized attraction route.

[0648] Step 6:

[0649] The device utilizes augmented reality (AR) technology, allowing visitors to display 3D models and visual information by holding up their smartphones when they visit an attraction. This enables users to obtain visually enriched information on the spot.

[0650] Step 7:

[0651] Users provide feedback through the app after their experience. This feedback includes aspects such as satisfaction with the visited attraction and the time spent there. The device sends this feedback to a server, and the accumulated data is used by the system to improve the accuracy of suggestions for future visits.

[0652] (Application Example 1)

[0653] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0654] In modern families, diverse family members often have different interests and schedules, making it challenging to efficiently plan activities and activities that satisfy everyone. In particular, there is a need for ways to optimize family events and daily schedule adjustments, enabling each family member to spend their time more fulfilling.

[0655] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0656] In this invention, the server includes data acquisition means for collecting visitor behavior information, generation means for analyzing the behavior information and generating activity routes and recommendation information optimized for the visitor, visualization means for presenting visual information using augmented reality technology, and home guidance means for suggesting activity routes optimized based on the interests and schedules of residents in the home. This makes it possible to automatically suggest an optimal activity schedule tailored to the interests of each family member.

[0657] "Data acquisition means" refers to a device or process for collecting visitor behavioral information and interest information.

[0658] "Generation methods" refer to the process of analyzing collected behavioral information to create activity routes and recommendation information optimized for visitors.

[0659] "Information presentation means" refers to a device or process for providing generated recommendation information to visitors in real time.

[0660] "Visualization means" refers to a device or process that uses augmented reality technology to provide visual information to visitors.

[0661] A "household guidance system" is a mechanism for suggesting optimized activity routes and events based on the interests and schedules of residents within a household.

[0662] In the system realizing this invention, the server collects visitor behavior information through data acquisition means. The data acquisition means receives information from sensors and smart devices installed in the home and transmits it to a cloud environment. As hardware, motion sensors and voice recognition devices are used. As software, machine learning APIs such as Google Cloud ML and AWS ML are used for data analysis.

[0663] Based on the analyzed data, the server uses a generation mechanism to create activity routes optimized for the individual interests of each family member. The generation mechanism analyzes the behavioral patterns of each visitor using the collected data and forms an interest profile. As a result, it generates basic information for the home guidance mechanism to suggest the most suitable events and activities.

[0664] The terminal provides users with real-time suggested recommendations via information presentation devices. Visualization devices use augmented reality technology to display suggested information clearly to the user. If necessary, the terminal displays visual information on displays installed in each room of the family, providing direct interactive guidance.

[0665] For example, when planning weekend activities, the server analyzes interest profiles and recommends a treasure hunt game in the garden. It takes into account family members' schedules and weather data to suggest the optimal time. Using a generative AI model, it provides detailed recommendations with prompts like: "Based on the current interests of each family member, please suggest the best activity schedule and event information for tomorrow. Please consider individual interests and include new recipes and activities."

[0666] This system allows users to enjoy optimized activity routes tailored to their individual interests, enabling them to spend more fulfilling time at home.

[0667] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0668] Step 1:

[0669] The server collects location and behavioral data of family members from sensors and smart devices within the home via data acquisition methods. The input is raw data from sensors, and the output is a dataset containing location and behavioral information. Data processing involves filtering out redundant data and extracting useful data.

[0670] Step 2:

[0671] The server uses a generation mechanism to analyze the collected data, examining the behavioral patterns and interest profiles of each member. The input is the dataset obtained in step 1, and the output is the interest profile of each member. As a data calculation, a machine learning model is applied to perform specific operations to extract features of the behavioral patterns.

[0672] Step 3:

[0673] The server proposes optimal activity routes and events based on the generated interest profiles. The input is the interest profiles obtained in step 2, and the output is the proposed activity routes and event information. As a data calculation, it uses a generative AI model to generate diverse suggestions based on prompt sentences.

[0674] Step 4:

[0675] The terminal presents suggested information received from the server to the user through an information display mechanism. The input is suggested information from the server, and the output is information visible to the user. Specific operations include the process of displaying text and visual data on the display.

[0676] Step 5:

[0677] The user selects and executes an activity based on the displayed information. The input is the activity route and event information, and the output is the execution of the selected activity. The user's specific action involves selecting a suggested activity and performing the corresponding actions.

[0678] Step 6:

[0679] The server collects feedback on user selections and actions and adds it to a dataset. The input is user feedback, and the output is the updated dataset. Specifically, the feedback is stored in a database and used to optimize future suggestions.

[0680] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0681] This invention relates to a system that provides visitors with an optimized experience in amusement parks and aquariums, and in particular incorporates an emotion engine that recognizes the user's emotions and dynamically adjusts the suggested content. The following is a description of the configuration and operation based on an embodiment of this system.

[0682] Users install a dedicated application on their smartphones and begin using the app. First, the visitor's basic interests and preferences are entered into the app and sent to the server via the device. This process creates a user profile on the server.

[0683] During a visit within the facility, the terminal continuously collects user movement and behavior data in real time. This information is periodically sent to a server, recording the user's movement route and time spent in each location. In addition, the terminal is equipped with an emotion engine that recognizes the user's emotions, using the camera and microphone to analyze the user's facial expressions and voice to evaluate their emotional state.

[0684] The server analyzes visitor behavioral and emotional data and uses generative models to generate optimal attraction routes and recommendations. Furthermore, it dynamically adjusts recommendations based on the emotional state obtained by the emotion engine, providing information and content that matches the visitor's emotions. For example, if a user is excited, it recommends thrilling attractions; if they are relaxed, it suggests leisurely exhibits.

[0685] The terminal provides users with pre-configured information received from the server. This includes elements that use augmented reality technology to visualize detailed information about attractions and exhibits, providing visitors with an interactive and immersive experience.

[0686] This system allows visitors to enjoy experiences tailored to their individual interests and emotional states, leading to increased satisfaction. Furthermore, user feedback is fed back to the server, which is used for further optimization. This enables facilities to achieve efficient management and high visitor satisfaction.

[0687] The following describes the processing flow.

[0688] Step 1:

[0689] Users install the app on their smartphones and enter information about their interests and preferences. For example, users can select their favorite attractions or animals.

[0690] Step 2:

[0691] The device sends the information entered by the user to the server. This information is stored on the server as part of the user profile and used for future personalized suggestions.

[0692] Step 3:

[0693] Once a user begins visiting a facility, the device uses its built-in GPS and sensors to acquire the user's location and behavioral data in real time. This allows the device to understand which attractions the user is visiting.

[0694] Step 4:

[0695] The emotion engine built into the device uses the device's camera and microphone to analyze the user's facial expressions and voice, and evaluate their emotional state. For example, it can detect whether the user is smiling or whether there is excitement in their voice tone.

[0696] Step 5:

[0697] The device continuously transmits location information, behavioral data, and emotional data to the server. The transmitted data is anonymized and processed in a way that protects privacy.

[0698] Step 6:

[0699] The server comprehensively analyzes the acquired data and uses a generative model to generate recommended attraction routes and event information for visitors. This process also includes adjustments based on the user's emotional state.

[0700] Step 7:

[0701] The server sends back adjusted recommendations to the device. This information includes details about attractions and exhibits tailored to the user's current emotional state.

[0702] Step 8:

[0703] The device displays the received information to the user within the app. Furthermore, it uses AR technology to display detailed information about specific attractions or animals in 3D, enhancing the user experience.

[0704] Step 9:

[0705] Users explore the facility using the information provided and enjoy the attractions and exhibits. Users can also provide feedback through the app about their experiences and emotional changes during their visit.

[0706] Step 10:

[0707] The server stores user feedback in a database and uses it to make improvements that enhance the overall accuracy of the system's suggestions. This allows for more effective customization on subsequent visits.

[0708] (Example 2)

[0709] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0710] In amusement parks, aquariums, and other entertainment facilities, there is a problem in that it is difficult to provide visitors with an experience optimized based on their individual interests and emotional states. Furthermore, guiding visitors in real time on how to navigate the facility and increasing their satisfaction is also a challenge. There is a need for systems that can solve these problems and improve the visitor experience.

[0711] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0712] In this invention, the server includes data acquisition means for collecting visitor behavioral and emotional information; generation model means for analyzing the behavioral and emotional information, generating an optimized attraction route and recommendation information for the visitor, and dynamically adjusting it according to the user's emotional state; information presentation means for providing the recommendation information to the visitor in real time and further presenting visual information using augmented reality technology; and feedback collection means for receiving feedback from visitors and using it to improve the accuracy of the analysis results. As a result, visitors can enjoy an optimal experience based on their emotions and interests, and make more meaningful use of the facility.

[0713] "Data acquisition means" refers to the general term for devices and software used to collect visitor behavioral and emotional information.

[0714] "Generative model means" refers to a process and technology for analyzing collected behavioral and emotional information, generating attraction routes and recommendation information optimized for visitors, and dynamically adjusting the information according to the user's emotional state.

[0715] "Information presentation means" refers to devices or methods that provide generated recommendation information to visitors in real time and present visual information using augmented reality technology.

[0716] A "feedback collection method" refers to a function or mechanism used to obtain opinions and evaluation information from visitors and to improve the accuracy of analysis results.

[0717] Augmented reality technology is a technology that overlays digital information onto the real world environment, providing users with a visually enhanced experience.

[0718] The present invention is a system for providing visitors with an optimal experience at amusement facilities such as amusement parks and aquariums. This system includes data acquisition means, generative model means, information presentation means, and feedback collection means.

[0719] First, the terminal uses mobile devices such as smartphones to acquire data. The terminal is equipped with a camera and microphone, which are used to analyze visitors' facial expressions and voices to acquire emotional information. In addition, GPS and Bluetooth technology are used to collect behavioral information such as visitors' location and length of stay.

[0720] The server receives behavioral and emotional information sent from the terminal and analyzes it using a generative AI model. This generative model generates attraction routes and content optimized for the visitor, dynamically adjusting recommendations based on their emotional state. For example, if the user is excited, it recommends thrilling events; if relaxed, it suggests quiet exhibits.

[0721] The generated recommendation information is sent back to the device. The device uses augmented reality technology to display detailed information about the recommended attractions and exhibits overlaid on the visitor's field of view, providing an interactive and immersive experience.

[0722] Users can provide feedback on their experience within the app after their visit. This feedback is then collected and sent to the server via a feedback collection system. The server uses the collected feedback to improve its analysis algorithms, enabling it to provide an even more refined experience on subsequent visits.

[0723] As a concrete example, a possible prompt message for a user visiting an aquarium might be: "A woman in her 20s is currently visiting the aquarium and appears relaxed. Please recommend the next attraction that would be best suited for her." Based on this prompt message, a generative AI model can generate and provide personalized recommendations for the user.

[0724] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0725] Step 1:

[0726] Users install a dedicated app on their smartphones and enter basic information about their interests and preferences at the start. This includes settings such as attraction categories and whether they prefer quiet exhibits. The entered information is sent to a server via the device. This data allows the server to create a visitor profile, which forms the basis for a personalized experience.

[0727] Step 2:

[0728] The device collects real-time data on the user's movements and actions during their visit to the facility. Specifically, it uses GPS and Bluetooth technology to track the user's current location and movement patterns. Furthermore, it uses the camera and microphone built into the device to analyze the user's facial expressions and voice to extract emotional data. This data is periodically transmitted to a server.

[0729] Step 3:

[0730] The server uses a generative AI model to analyze behavioral and emotional information received from the terminal. Based on the input data, it infers the user's current mood and generates the optimal attraction route accordingly. For example, if the server determines that the user is excited, thrilling attractions will be recommended. This information is dynamically adjusted, and the optimal route suggestion is generated as the output result.

[0731] Step 4:

[0732] The recommendation information generated by the AI ​​model is sent from the server to the terminal. The terminal then uses augmented reality technology to overlay detailed information about attractions and exhibits onto the user's field of view. This allows visitors to receive real-time visual information about their next destination and have an interactive experience.

[0733] Step 5:

[0734] After completing their experience at a facility, users can provide feedback on their satisfaction level and areas for improvement within the app. This feedback is sent to a server to help optimize the experience further. The server analyzes this feedback and uses it to improve the accuracy of its analysis algorithms. The final output is a collection of data to further enhance the user's experience on their next visit.

[0735] (Application Example 2)

[0736] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0737] In urban areas, it is difficult for individual citizens and visitors to obtain the optimal experiences and information according to their emotional state and interests at any given time. Therefore, there is a need to analyze visitors' emotions in real time and provide information and routes tailored accordingly. Furthermore, there is a challenge in achieving a more immersive experience by providing visual information using augmented reality technology.

[0738] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0739] In this invention, the server includes data acquisition means for collecting visitor behavior information and emotional states, generation model means for analyzing the behavior information and emotional data and generating optimized information and recommended content, and information presentation means for providing the recommended information to visitors in real time and making adjustments based on their emotions. As a result, visitors can receive experiences and navigation instructions that match their emotions and interests at the time, and by utilizing visual information from augmented reality technology, they can enrich their activities in urban areas.

[0740] A "visitor" is an individual who visits a specific facility or urban area for the purpose of using the system.

[0741] "Behavioral information" refers to data about visitors' activities acquired by the system, such as their travel routes and the amount of time they spend inside the facility.

[0742] "Emotional state" refers to information that indicates a visitor's temporary mental state, analyzed from their facial expressions and voice.

[0743] "Generative modeling means" refers to methods and functions for calculating the most suitable information and content for visitors based on acquired behavioral and emotional data.

[0744] "Information presentation means" refers to methods and technologies for providing visitors with optimized information in real time.

[0745] A "visualization method" refers to a system that uses augmented reality technology to visually present information to visitors.

[0746] Augmented reality technology is a technique that overlays digital information onto real-world scenes.

[0747] This invention is a system that collects visitor behavior information and emotional state and provides an optimized experience based on this information. The server receives data from the visitor's smartphone or wearable device and collects behavioral and emotional data. This data is acquired by the camera and microphone built into the smartphone and analyzed using OpenCV or the Google Cloud Speech-to-Text API.

[0748] The server runs a generative AI model based on the collected data, suggesting attractions, events, and spots suitable for visitors. The generated information is transmitted to the device in real time and adjusted according to the visitor's emotions. Augmented reality technologies such as AR.js are used to present information intuitively.

[0749] The device displays the received optimization information to the visitor in an easy-to-understand manner, providing an experience tailored to their emotional state. For example, if a desire for tranquility is detected, recreational spots such as parks or quiet cafes are suggested. These suggestions are displayed on the screen as digital information alongside real-world scenery, allowing visitors to intuitively navigate to those locations.

[0750] For example, if a visitor is strolling through a city and feels relaxed, the server will suggest quiet and peaceful spots, and the device will guide them there using augmented reality technology. The input prompt for the generating AI model would be: "Generate content to suggest the best recommended spots for when the user is relaxing. Include information about places where they can have a pleasant afternoon."

[0751] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0752] Step 1:

[0753] When a user arrives at their destination, the device uses its camera and microphone to collect visitor behavioral and emotional data. The input is the user's video and audio, while the output is the analyzed behavioral and emotional state. The camera captures facial expressions, OpenCV is used to analyze emotions, and the microphone records audio, which is then converted to text using the Google Cloud Speech-to-Text API for emotion analysis.

[0754] Step 2:

[0755] The device sends the collected data to the server. The server updates the visitor behavior database based on this data. The input is the behavioral information and emotional state analyzed in the previous stage, and the output is the updated database. Specifically, the data is uploaded to the server in real time via a secure communication method.

[0756] Step 3:

[0757] The server runs a generative AI model and analyzes behavioral and emotional data. This generates information and content optimized for the visitor. The input is behavioral data and emotional states stored on the server, and the output is recommended attractions and content. Prompts are used to instruct the generative model to generate appropriate information.

[0758] Step 4:

[0759] The server sends the generated recommendation information back to the device in real time. This provides a list of recommended activities and attractions. The input is the recommendation information output by the generation AI model, and the output is the data packets sent to the device.

[0760] Step 5:

[0761] The device visualizes the received information using augmented reality technology and provides it to the user. This allows the user to intuitively understand where to go next or what events to attend. The input is recommendation information received from the server, and the output is AR content as visual data. AR.js is used to overlay digital information onto the real-world landscape in real time.

[0762] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0763] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0764] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0765] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0766] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0767] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0768] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0769] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0770] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0771] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0772] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0773] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0774] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0775] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0776] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0777] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0778] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0779] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0780] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0781] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0782] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0783] The following is further disclosed regarding the embodiments described above.

[0784] (Claim 1)

[0785] A means of acquiring data to collect visitor behavior information,

[0786] A generation model means that analyzes the aforementioned behavioral information and generates an attraction route and recommendation information optimized for visitors,

[0787] Information presentation means that provides the aforementioned recommended information to visitors in real time,

[0788] A visualization means that presents visual information using augmented reality technology,

[0789] A system that includes this.

[0790] (Claim 2)

[0791] The system according to claim 1, wherein the information presentation means includes a navigation function that provides an optimal travel route based on the visitor's current location.

[0792] (Claim 3)

[0793] The system according to claim 1, wherein the data acquisition means includes a setting function that accepts personalized settings based on the visitor's interests.

[0794] "Example 1"

[0795] (Claim 1)

[0796] A means for visitors to input information about their interests,

[0797] A profile creation means for forming a user profile based on the interests of the aforementioned visitor,

[0798] A data acquisition method for collecting visitor location information and behavioral data in real time,

[0799] A generation model means that analyzes the aforementioned data and generates an optimized attraction route and recommended information for visitors using a generation AI model,

[0800] Information presentation means that provides the aforementioned recommended information to visitors in real time,

[0801] A visualization means that presents visual information using augmented reality technology,

[0802] A feedback storage method that accumulates visitor feedback information to improve accuracy for future visits,

[0803] A system that includes this.

[0804] (Claim 2)

[0805] The system according to claim 1, wherein the information presentation means includes a navigation function that provides an optimal travel route based on the visitor's current location and congestion status.

[0806] (Claim 3)

[0807] The system according to claim 1, wherein the data acquisition means includes receiving personalized settings based on the visitor's interests and transmitting the setting information to a server.

[0808] "Application Example 1"

[0809] (Claim 1)

[0810] A means of acquiring data to collect visitor behavior information,

[0811] A generation means that analyzes the aforementioned behavioral information and generates activity routes and recommended information optimized for the visitor,

[0812] Information presentation means that provides the aforementioned recommended information to visitors in real time,

[0813] A visualization means that presents visual information using augmented reality technology,

[0814] A home navigation system that suggests optimized activity routes based on the interests and schedules of residents within the home,

[0815] A system that includes this.

[0816] (Claim 2)

[0817] The system according to claim 1, wherein the information presentation means includes a guidance function that provides an optimal travel route based on the visitor's current location.

[0818] (Claim 3)

[0819] The system according to claim 1, wherein the data acquisition means includes a setting means for accepting personalized settings based on the visitor's interests.

[0820] "Example 2 of combining an emotion engine"

[0821] (Claim 1)

[0822] A means for acquiring data to collect visitor behavioral and emotional information,

[0823] A generative model means analyzes the aforementioned behavioral and emotional information, generates an optimized attraction route and recommendation information for visitors, and dynamically adjusts it according to the user's emotional state.

[0824] An information presentation means that provides the aforementioned recommendation information to visitors in real time and further presents visual information using augmented reality technology,

[0825] A means of collecting feedback from visitors to help improve the accuracy of analysis results,

[0826] A system that includes this.

[0827] (Claim 2)

[0828] The system according to claim 1, wherein the information presentation means includes a navigation function that provides an optimal travel route based on the visitor's current location and emotional state.

[0829] (Claim 3)

[0830] The system according to claim 1, wherein the data acquisition means includes a setting function that accepts personalized settings based on the visitor's interests and emotional state.

[0831] "Application example 2 when combining with an emotional engine"

[0832] (Claim 1)

[0833] A means for acquiring data to collect visitor behavior information and emotional state,

[0834] A generative model means that analyzes the aforementioned behavioral information and emotional data to generate information and recommended content optimized for visitors,

[0835] An information presentation means that provides the aforementioned recommendation information to visitors in real time and makes adjustments based on emotions,

[0836] A visualization means that presents visual information using augmented reality technology,

[0837] A system that includes this.

[0838] (Claim 2)

[0839] The system according to claim 1, wherein the information presentation means includes a navigation function that provides an optimal travel route based on the visitor's current location and emotional state.

[0840] (Claim 3)

[0841] The system according to claim 1, wherein the data acquisition means includes a setting function that accepts personalized settings based on the visitor's interests and emotions. [Explanation of Symbols]

[0842] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of acquiring data to collect visitor behavior information, A generation means that analyzes the aforementioned behavioral information and generates activity routes and recommended information optimized for the visitor, Information presentation means that provides the aforementioned recommended information to visitors in real time, A visualization means that presents visual information using augmented reality technology, A home navigation system that suggests optimized activity routes based on the interests and schedules of residents within the home, A system that includes this.

2. The system according to claim 1, wherein the information presentation means includes a guidance function that provides an optimal travel route based on the visitor's current location.

3. The system according to claim 1, wherein the data acquisition means includes a setting means for accepting personalized settings based on the visitor's interests.

Images