system

Abstract

We provide the system. [Solution] The primary means of collecting user information, A second method involves using deep learning to analyze the user's interests and characteristics based on the aforementioned user information. Based on the analysis results of the second method described above, a third method is proposed to present a suitable place for the user, A fourth method of making a reservation and payment from the presented options for a place to stay, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In modern times, there is a problem that it is difficult for guardians to provide an optimal growth environment for children while being occupied with work and housework. In particular, finding appropriate activities and places according to the personality and interests of individual children is a time-consuming and laborious task. For this reason, many guardians are seeking a way to easily find places suitable for their children.

Means for Solving the Problems

[0005] This invention provides a system that automatically recommends suitable places for each child by collecting user information and analyzing that information using deep learning. Furthermore, it includes a means for centrally managing procedures including reservations and payments, enabling users to efficiently set up the optimal environment for their children. In addition, by utilizing real-time emotion recognition technology to improve the accuracy of optimal place suggestions according to the child's emotional state, a more personalized service is realized.

[0006] "Means of collecting user information" refers to functions that input or obtain relevant personal information and behavioral logs from users.

[0007] "Deep learning" is a type of artificial intelligence technology that uses algorithms to learn complex patterns and characteristics from large amounts of data, enabling prediction and classification.

[0008] "Means of analyzing interests and characteristics" refers to data processing functions that identify a user's preferences and personality based on their information, and then provide appropriate suggestions.

[0009] "Means of presenting a place to belong" refers to a function that displays the most suitable place or activity as an option for the user based on the analysis results.

[0010] "Methods for making reservations and payments" refers to the series of processes for confirming a reservation for a presented location option and completing the associated payment online.

[0011] "Means for acquiring facial expressions and voice" refers to a function that uses devices such as cameras and microphones to capture the user's facial expressions and voice data in real time.

[0012] "Means of recognizing emotions" refers to technology that analyzes acquired facial and voice data to interpret the user's emotional state.

[0013] "Means of displaying information" refers to an interface that visually presents analysis and suggestion results to users, making comparison and selection easier. [Brief explanation of the drawing]

[0014] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14]It is a sequence diagram showing the processing flow of a data processing system in Application Example 2 when a sentiment engine is combined.

Embodiments for Carrying Out the Invention

[0015] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0018] In the following embodiments, a numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0019] In the following embodiments, a numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0020] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0022] [First Embodiment]

[0023] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0024] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0027] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0030] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0034] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0035] This invention is a system that recommends the optimal location based on information entered by the user. This system effectively functions by proposing a personalized environment according to the user's characteristics and interests, and by supporting a series of procedures including reservation and payment. The operation of this system will be explained from the perspectives of the user, server, and terminal, with specific examples.

[0036] Step 1: Enter Information

[0037] First, the device provides an interface for collecting basic information about the child from the user. This information includes age, interests, and past behavioral history. For example, the user might input information about an 8-year-old child such as "interested in science" and "sociable."

[0038] Step 2: Data Analysis

[0039] Next, the server analyzes the received information using a deep learning model. The model extracts interests and characteristics to identify the optimal location for the child, comparing them with past data. In this process, facial and voice data transmitted in real time from the device are also used, and emotion recognition technology is employed to perform an even more accurate analysis. From the analysis results, the server generates multiple location options.

[0040] Step 3: Presentation of recommendations and selection

[0041] The device displays a list of potential locations sent from the server on the user interface. Each option is displayed along with specific activity details and location information. This allows the user to compare the displayed options and choose the most suitable one.

[0042] For example, based on the analysis, two options might be displayed: "Science Museum" and "Workshop at a Community Center." The options would include descriptions of their appeal and ratings to help the user make a choice.

[0043] Step 4: Booking and Payment

[0044] Finally, once the user selects a specific location, the terminal begins the process to proceed with the reservation. Furthermore, through the integrated payment system, users can make secure and fast payments. This ensures a smooth and seamless process from reservation to payment.

[0045] Through the embodiments described above, the present invention can provide users with an efficient and personalized activity environment for children, and support the realization of a manageable daily life.

[0046] The following describes the processing flow.

[0047] Step 1:

[0048] The device displays an interface for entering the user's basic information. This information includes the child's age, interests, personality, and behavioral history. The user enters the information according to the instructions.

[0049] Step 2:

[0050] The terminal temporarily stores the input information and sends it to the server using a secure communication protocol.

[0051] Step 3:

[0052] The server inputs the received information into a deep learning model. This model analyzes the user's preferences and characteristics to generate appropriate location options.

[0053] Step 4:

[0054] The server analyzes facial and voice data from the terminal as data for emotion recognition to determine the user's current emotional state.

[0055] Step 5:

[0056] Based on the analysis results, the server determines a list of optimal locations for the user and sends the detailed information to the terminal.

[0057] Step 6:

[0058] The device displays location options received from the server in its user interface. Each option includes location details and reasons for the recommendation.

[0059] Step 7:

[0060] The user selects their desired location from the displayed options and confirms their selection on their device.

[0061] Step 8:

[0062] The device initiates the reservation process for the selected location and notifies the server of the necessary information.

[0063] Step 9:

[0064] The terminal calls the payment module and prompts the user to enter payment information.

[0065] Step 10:

[0066] The user enters their payment information and completes the transaction. The terminal confirms this and displays a completion notification to the user.

[0067] (Example 1)

[0068] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0069] In today's diverse society, it is a major challenge to quickly and accurately propose activity environments optimized for the individual interests and characteristics of users. Furthermore, it is necessary to improve the user experience by considering behavioral history and real-time emotions during this process.

[0070] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0071] In this invention, the server includes a device means for collecting user attribute information, a device means for analyzing the user's interests and characteristics using an artificial intelligence model based on the user attribute information, and a device means for presenting a suitable activity environment based on the analysis results. This makes it possible to propose an optimal activity environment tailored to the individual needs of the user.

[0072] A "device for collecting user attribute information" is a technology that provides an interface for inputting or acquiring data about users, such as age, interests, and behavioral history.

[0073] A "device that analyzes interests and characteristics using artificial intelligence models" is a technology that utilizes deep learning and machine learning algorithms to analyze users' interests and characteristics from collected attribute information.

[0074] A "device that presents a suitable activity environment" is a technology that visually provides users with the most suitable activity or event options based on analysis.

[0075] A "device that continuously acquires the external state of a user and identifies their emotions" is a technology that recognizes and analyzes the emotional state of a user based on data such as facial expressions and voice acquired in real time.

[0076] A "device that displays information for evaluating the presented activity environment options" is a technology that provides detailed information, reviews, and evaluations for users to refer to when making a selection.

[0077] This invention is a system that proposes an optimal activity environment tailored to the individual needs of users. The system operates with a terminal and a server working in cooperation, and the invention is implemented using multiple means.

[0078] The device provides an interface for collecting user attribute information. This interface is a graphical user interface (GUI) with text fields and dropdown menus for entering data such as age, interests, and behavioral history.

[0079] The server utilizes a generative AI model using Python and TENSORFLOW® to analyze the received attribute information. This model uses deep learning algorithms to analyze the user's interests and characteristics and derive an activity environment suitable for the user. In doing so, it uses facial and voice data acquired in real time to perform emotion recognition and improve the accuracy of the analysis.

[0080] Furthermore, based on the analysis results, the server generates a selection of activity environments suitable for the user and sends them to the terminal. The terminal displays these options in its user interface, allowing the user to select one. The options include specific information and reviews, enabling the user to compare details and make a decision.

[0081] Ultimately, the user proceeds with booking their chosen activity environment, and the terminal integrates with the payment system to securely complete the payment. This entire process allows users to smoothly select and book the optimal activity environment.

[0082] For example, if a user enters a prompt such as, "Considering that my 8-year-old child is interested in science and is sociable, please suggest activities and events that would be perfect for this child," the system can provide optimal suggestions.

[0083] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0084] Step 1:

[0085] The terminal collects attribute information from the user. The user uses a graphical user interface (GUI) on the terminal to input their age, interests, and past behavioral history. The input information is prepared as formatted data and sent to the server. The output is the basic data that the server uses for analysis.

[0086] Step 2:

[0087] The server receives attribute information sent from the terminal and performs analysis using a generative AI model. Specifically, it uses libraries such as Python and TensorFlow to extract interests and characteristics using a deep learning model. The input is the user's attribute information, and the output is the analysis result to identify the most suitable activity candidates for the user.

[0088] Step 3:

[0089] The server performs emotion recognition using facial and voice data provided in real time from the terminal. Emotion recognition technology is applied to identify the user's emotional state. The input is real-time facial and voice data, and the output is emotion analysis results to improve analysis accuracy.

[0090] Step 4:

[0091] The server integrates the analysis results of interests and characteristics with the analysis results of emotions to generate a list of candidate activity environments suitable for the user. It extracts possible activities and events from the database and sends them to the terminal as ranked options. The input is the integrated analysis results, and the output is a list of recommended activities presented to the user.

[0092] Step 5:

[0093] The terminal displays a list of potential activity environments sent from the server on the user interface. Each option includes detailed activity information and reputation, and the user makes a selection based on this information. The input is the recommended activity list from the server, and the output is the user's optimal choice.

[0094] Step 6:

[0095] Once the user selects their activity environment, the terminal initiates the booking and payment process. It connects to the booking and payment systems and completes the procedure according to the user's selections. The input is the user's selections, and the output is a confirmation of the completed booking and payment.

[0096] (Application Example 1)

[0097] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0098] Modern consumers tend to strongly desire personalized experiences based on their interests and preferences, but the systems and services to achieve this are still not sufficiently developed. Therefore, there is a need to enable the provision of experiences optimized for each individual user.

[0099] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0100] In this invention, the server includes information acquisition means for collecting user information, analysis means for analyzing interests and characteristics using machine learning based on the user information, processing means for making reservations and electronic payments from suggested activity location options, and guidance means for providing a personalized experience to the user based on the in-store environment. This enables the provision of an optimized experience for each user and seamless reservation and purchase procedures.

[0101] "Information acquisition means" refers to a device or method for collecting data on users' interests and preferences.

[0102] "Analysis means" refers to a device or method that uses machine learning techniques to analyze collected user information and identify individual interests and characteristics.

[0103] "Proposed means" refers to a device or method that presents activity locations and experiences suitable for the user based on the analysis results.

[0104] "Processing means" refers to a device or method that provides the function of making a reservation and electronic payment for an activity location selected by the user.

[0105] "Recognition means" refers to a device or method that has the ability to acquire a user's facial expressions and voice data in real time and analyze their emotions.

[0106] "Display means" refers to a device or method for visually displaying information about the options presented to the user.

[0107] "Guidance means" refers to a device or method that provides users with a personalized experience and information based on the in-store environment in real time.

[0108] This invention is a system designed to provide a personalized experience based on the individual interests and preferences of each user. In the implementation of this invention, the server, terminal, and user each play their respective roles.

[0109] The device collects data on the user's interests and preferences, functioning as a means of information acquisition. This data includes age, past behavioral history, facial expressions, and voice data. This allows the device to capture the user's individual needs in detail. The hardware used includes smartphones and smart glasses, and real-time processing is employed to acquire facial expressions and voice data.

[0110] The server analyzes data received from the terminal using a deep learning model (e.g., TensorFlow) and proposes the optimal experience for the user. This analysis method utilizes machine learning algorithms to extract patterns from the data and reveal interests and characteristics. The processing results are used as a means of making suggestions to the user, and an optimized activity location is presented.

[0111] Users compare the options presented on their terminal, select the best one, and then make a reservation and electronic payment. The reservation system and digital payment platform work together as a processing mechanism, allowing users to enjoy a smooth experience.

[0112] Furthermore, this system can guide users to a personalized experience based on the real-time environment within the store. Specifically, when children's workshops or events are held in the store, the terminal is designed to immediately notify users and encourage their participation.

[0113] For example, if a user enters "I'm interested in science," the server will recommend the most suitable science-related events based on data from similar users in the past. An example of a prompt might be, "I'm looking for science events that my child can participate in. Please tell me about any experiences you recommend at this time."

[0114] Thus, the present invention is a system that combines deep learning and real-time data analysis to enable personalized suggestions and smooth execution for users.

[0115] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0116] Step 1:

[0117] The device collects data from the user regarding their interests and past behavioral history. This input includes age and interests, as well as facial expressions and voice data captured in real time. The device temporarily stores this data in local storage and then sends it to the server.

[0118] Step 2:

[0119] The server receives input data sent from the terminal and passes it on to a deep learning model for processing. The model extracts patterns of user interests and characteristics from the newly received data while comparing it with past data. In this process, machine learning algorithms are used to analyze the data and identify the most suitable activity locations and events for the user as output.

[0120] Step 3:

[0121] Based on the analysis results obtained from the server, the terminal displays suggested activity locations and experience options on the user interface. From this output, the terminal presents detailed information about a specific event to the user and organizes it for easy comparison. The displayed options include location information and available dates and times.

[0122] Step 4:

[0123] The user selects the experience best suited to themselves or their child from the options displayed on the device. Once the user makes a selection, the device immediately connects with the server and begins processing the reservation and electronic payment for the selected experience. After the process is complete, the device displays a confirmation notification to the user.

[0124] Step 5:

[0125] The terminal uses the store's environmental sensors to acquire real-time environmental information. Based on this, the terminal guides the user to new events and discount information tailored to their needs. The user can respond by choosing additional experiences. The guidance is delivered via smartphone or smart glasses.

[0126] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0127] This invention is a system that utilizes user input information and real-time sentiment data to recommend the optimal location personalized to the user, and by combining it with a sentiment engine, it improves the accuracy and user experience.

[0128] System Configuration

[0129] This system consists of a terminal, a server, and an emotion engine. The terminal is responsible for receiving information input from the user and collecting real-time emotion data, which is then transmitted to the server. The server utilizes deep learning and the emotion engine to analyze the user's attributes, characteristics, and emotional state, and based on this, selects potential locations.

[0130] Collection of user information

[0131] The device receives basic information from the user, such as the child's age, interests, and personality, through its user interface. In addition, it uses a camera and microphone to collect the user's facial expressions and voice data in real time. This emotional data is analyzed by an emotion engine to determine the user's current emotional state.

[0132] Data analysis and suggestions for places to belong

[0133] The server uses user information received from the terminal to input into a deep learning model. This model integrates the user's past data and current sentiment data to generate a list of suitable locations for each user.

[0134] The emotional state analysis provided in real time by the emotion engine is used to prioritize suggested locations. For example, if the system determines that the user is stressed, relaxing places and activities will be suggested at the top of the list.

[0135] Information presentation and selection

[0136] The device displays location options prioritized by the server to the user. By comparing this presented information, the user can make the most suitable choice. For example, the server might suggest "a drawing event at a nearby museum" and "a nature observation workshop at a park," and based on the user's current emotional state, display the drawing event higher.

[0137] Reservation and payment

[0138] The terminal proceeds with the reservation and payment process for the location selected by the user. Since all related procedures are integrated and executed at this stage, users can easily prepare to participate in the event.

[0139] According to embodiments of the present invention, users can easily select the optimal environment according to their emotional state, thereby providing a highly satisfying experience.

[0140] The following describes the processing flow.

[0141] Step 1:

[0142] The device displays a screen where the user can input basic information such as the child's age, interests, and personality. At this time, it prepares to collect the user's facial expressions and voice in real time using the camera and microphone.

[0143] Step 2:

[0144] The user follows the instructions on the device, enters the necessary information about the child into a form, and then authorizes the collection of facial expression and voice data.

[0145] Step 3:

[0146] The device transmits facial expression and voice data collected along with user input information to the server using a secure communication protocol.

[0147] Step 4:

[0148] The server analyzes the received data. Here, it uses a deep learning model to analyze the user's interests and characteristics, and an emotion engine to determine their emotional state in real time.

[0149] Step 5:

[0150] The server generates and prioritizes suitable location options for the user based on the analysis results. For example, if relaxation is deemed necessary, a quiet environment will be prioritized.

[0151] Step 6:

[0152] The server sends the generated options and their priority order to the terminal.

[0153] Step 7:

[0154] The terminal displays a list of locations received from the server on the user interface and organizes the information to make it easier for the user to select a location.

[0155] Step 8:

[0156] The user selects the appropriate option from the presented choices and confirms the reservation by following the instructions on the device.

[0157] Step 9:

[0158] The terminal initiates the reservation and payment process for the selected location. This allows the user to easily complete their participation reservation.

[0159] Step 10:

[0160] The terminal displays a reservation and payment confirmation notification to the user, informing them that the process is complete.

[0161] (Example 2)

[0162] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0163] In today's busy lifestyle, people lack the means to find the optimal place that suits their emotions and interests. This leads to users experiencing stress and being forced to make unsatisfactory choices. There is a need to provide personalized recommendations in real time, taking into account the user's emotional state.

[0164] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0165] In this invention, the server includes means for acquiring user information, means for analyzing user characteristics using deep learning, and means for analyzing emotional data in real time. This makes it possible to suggest the optimal location based on the user's emotional state and characteristics.

[0166] "Means for obtaining user information" refers to devices or functions for collecting attribute information from users, such as age, interests, and personality.

[0167] "Methods for analyzing user characteristics using deep learning" refers to a process of interpreting and evaluating user characteristics using AI technology based on acquired user information.

[0168] "Methods for analyzing emotional data in real time" refer to technologies that instantly process emotion-related data obtained from a user's facial expressions and voice to determine their current emotional state.

[0169] "Means of recommending optimal locations" refers to a system or function that, based on analysis results, presents the most suitable locations and activities for individual users.

[0170] "Means of making reservations and payments" refers to means of automating and executing reservation procedures and related payment processes for selected locations and activities.

[0171] This invention is implemented as a personalized recommendation system. The system consists of a terminal, a server, and an emotion analysis engine. The terminal is a device for user information input and real-time collection of emotion data, and is responsible for transmitting the data obtained from the user to the server.

[0172] The device provides the user with an input interface and receives personal and attribute information, such as age, interests, and personality. The device also includes a camera and microphone, which capture the user's real-time facial expressions and voice. This data is analyzed by an emotion analysis engine to determine the user's emotional state.

[0173] The server processes information transmitted from the terminal using deep learning technology and generative AI models. The acquired data is input into the model, and the user's characteristics and emotional state are integrated and analyzed. Based on these analysis results, the server suggests optimal locations and activities for the user. In this process, the results of the emotion analysis engine contribute to the priority of the suggestions.

[0174] Users can review and select location suggestions displayed on the device. After selection, the device automates the booking and payment process, providing users with a quick and smooth experience.

[0175] As a concrete example, the system suggests "relaxing places" based on the user's current emotional state. If the user is feeling stressed, the server will recommend things like "nearby nature parks" or "relaxation classes."

[0176] An example of a prompt used in a generative AI model might be: "The user is currently feeling stressed, so please suggest a place where they can relax. Specifically, please tell me what kind of nearby events or facilities would be suitable."

[0177] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0178] Step 1:

[0179] The terminal obtains basic information from the user through its user interface. This basic information includes age, interests, and personality. The entered information is converted into a digital format and prepared for subsequent data processing.

[0180] Step 2:

[0181] The device collects the user's facial expressions and voice in real time using its camera and microphone. The acquired emotional data is sent to an emotion analysis engine after signal processing. In this process, the data is pre-processed to quantify the user's emotional state.

[0182] Step 3:

[0183] The emotion analysis engine receives and analyzes emotion-related data transmitted from the device. Specifically, it extracts features from the data and identifies emotions such as joy and sadness. The results of this analysis are output as a numerical evaluation that identifies the user's emotional state.

[0184] Step 4:

[0185] The server receives basic user information from the terminal and the results of the sentiment analysis engine, and then integrates and processes them. The received data becomes input and is fed into a deep learning model. This model analyzes the user's attributes and emotional state in combination to generate a list of suitable locations for the user.

[0186] Step 5:

[0187] The server determines priorities based on the generated list of potential locations. Numerical evaluations of sentiment data are reflected in this prioritization. The prioritized list is sent to the terminal as output.

[0188] Step 6:

[0189] The device displays a list of potential locations received from the server to the user. The user can then choose the option that best suits their current mood and preferences. At this stage, the user experience is customized based on the available choices.

[0190] Step 7:

[0191] After the user makes a selection, the terminal begins the reservation and payment process based on that selection. The reservation is completed by accessing the reservation system and processing the generated information as input. Payment processing is carried out similarly, and a confirmation is output indicating that all necessary procedures have been completed.

[0192] (Application Example 2)

[0193] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0194] In recent years, consumers have been seeking personalized shopping experiences not only in physical stores but also in virtual environments. However, traditional online shopping systems and virtual stores have been insufficient in suggesting products and experiences that take into account the user's emotions and real-time state. As a result, there is a challenge in that users cannot smoothly select appropriate products and experiences that match their emotions at any given time.

[0195] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0196] In this invention, the server includes a device means for collecting user data, a device means for analyzing the user's interests and characteristics using machine learning based on the user data, and a device means for suggesting facilities suitable for the user based on the analyzed results. This makes it possible to suggest personalized products and experiences based on real-time emotional data.

[0197] A "device for collecting user data" is a device that acquires basic user information, facial expressions, voice, emotional data, etc., and transmits it to a server for analysis.

[0198] A "device that analyzes interests and characteristics using machine learning" is a system that uses acquired user data and data analysis techniques to understand users' areas of interest and individual characteristics.

[0199] A "device that suggests facilities suitable for users" is a system that suggests the most suitable products and experiences for users based on their analyzed interests, characteristics, and emotional state.

[0200] A "device that acquires emotional data in real time and recognizes emotional states" is a technology that acquires the user's facial expressions and voice through a camera and microphone, and quickly determines their emotional state at that moment.

[0201] A "device that displays products or experiences in a virtual environment according to the user's emotional state" is a system that visually presents the most suitable products or experiences based on the user's real-time emotional state within a virtual reality or augmented reality environment.

[0202] A "generative AI model" is an artificial intelligence algorithm used to generate and suggest personalized products and experiences based on user data and emotional states.

[0203] A "prompt sentence" is an instruction given to a generative AI model to obtain a specific output. Based on the content of the sentence, the AI ​​suggests appropriate products or experiences.

[0204] The system for realizing this invention enables the collection of information from users and the analysis of emotional data in real time. The system includes a terminal for collecting data, a server for analysis and making suggestions, and an AI model.

[0205] The devices used are such as smart glasses or smartphones. These devices are equipped with cameras and microphones to capture the user's facial expressions and voice, and collect emotional data in real time. The collected data is sent to a server.

[0206] The server uses machine learning algorithms and an emotion analysis engine to analyze the user's characteristics and current emotional state. This makes it possible to suggest facilities and experiences based on the user's interests and emotions. The emotion analysis engine uses software such as the Emotion Analysis SDK to quickly determine the user's emotional state.

[0207] The generative AI model runs on a server and generates personalized products and experiences tailored to the user's emotional state. For example, if a user is stressed, the AI ​​model can suggest relaxing candles or music. If they are excited, it can suggest new games or exciting events.

[0208] The terminal displays suggestions from the server to the user. The user can browse the presented options and select experiences or products that interest them. Depending on the user's selection, booking and payment procedures may also be carried out through the terminal.

[0209] As a concrete example, when a user visits a virtual store, the emotion analysis engine analyzes the user's facial expressions, and if it determines that the user's current emotional state is "excited," the generative AI model could suggest a "new adventure game."

[0210] To generate suggestions for a generative AI model, use the following prompt statements:

[0211] "Please suggest new or unique products that would be suitable for someone currently experiencing an excited emotional state."

[0212] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0213] Step 1:

[0214] The device collects the user's facial expressions and voice in real time. This involves capturing facial data using a camera and recording voice data using a microphone. This raw data is pre-processed for emotion analysis before being sent to the server. The input data consists of the user's facial expressions and voice data, while the output is pre-processed data.

[0215] Step 2:

[0216] The server analyzes the received data using the Emotion Analysis SDK. First, it takes in facial expression data and voice data as input, which the emotion analysis engine processes. As a result of the analysis, it outputs the user's current emotional state. This emotional state is used in the subsequent suggestion process.

[0217] Step 3:

[0218] The server inputs the analyzed emotional state and the user's past interests into a machine learning model. This model performs analysis to enumerate the products and experiences best suited to the user. The output is a personalized list of suggestions based on the user's emotional state.

[0219] Step 4:

[0220] The terminal displays a list of suggestions received from the server to the user. The user selects products or experiences that interest them from this list. A user interface is used here to facilitate visual selection. The displayed suggestions are optimized by a generative AI model, and the user makes their selection based on these suggestions.

[0221] Step 5:

[0222] Once the user makes a selection, the terminal sends that information back to the server to initiate the booking and payment process. The input here is the user's selection information, and the output is the booking confirmation and payment completion for the selected experience or product. The terminal processes these steps in the background and displays the completion status to the user.

[0223] This series of processing steps allows users to receive product and experience suggestions that match their current emotions, enabling them to smoothly select and purchase items.

[0224] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0225] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0226] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0227] [Second Embodiment]

[0228] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0229] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0230] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0231] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0232] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0233] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0234] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0235] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0236] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0237] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0238] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0239] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0240] This invention is a system that recommends the optimal location based on information entered by the user. This system effectively functions by proposing a personalized environment according to the user's characteristics and interests, and by supporting a series of procedures including reservation and payment. The operation of this system will be explained from the perspectives of the user, server, and terminal, with specific examples.

[0241] Step 1: Enter Information

[0242] First, the device provides an interface for collecting basic information about the child from the user. This information includes age, interests, and past behavioral history. For example, the user might input information about an 8-year-old child such as "interested in science" and "sociable."

[0243] Step 2: Data Analysis

[0244] Next, the server analyzes the received information using a deep learning model. The model extracts interests and characteristics to identify the optimal location for the child, comparing them with past data. In this process, facial and voice data transmitted in real time from the device are also used, and emotion recognition technology is employed to perform an even more accurate analysis. From the analysis results, the server generates multiple location options.

[0245] Step 3: Presentation of recommendations and selection

[0246] The device displays a list of potential locations sent from the server on the user interface. Each option is displayed along with specific activity details and location information. This allows the user to compare the displayed options and choose the most suitable one.

[0247] For example, based on the analysis, two options might be displayed: "Science Museum" and "Workshop at a Community Center." The options would include descriptions of their appeal and ratings to help the user make a choice.

[0248] Step 4: Booking and Payment

[0249] Finally, once the user selects a specific location, the terminal begins the process to proceed with the reservation. Furthermore, through the integrated payment system, users can make secure and fast payments. This ensures a smooth and seamless process from reservation to payment.

[0250] Through the embodiments described above, the present invention can provide users with an efficient and personalized activity environment for children, and support the realization of a manageable daily life.

[0251] The following describes the processing flow.

[0252] Step 1:

[0253] The device displays an interface for entering the user's basic information. This information includes the child's age, interests, personality, and behavioral history. The user enters the information according to the instructions.

[0254] Step 2:

[0255] The terminal temporarily stores the input information and sends it to the server using a secure communication protocol.

[0256] Step 3:

[0257] The server inputs the received information into a deep learning model. This model analyzes the user's preferences and characteristics to generate appropriate location options.

[0258] Step 4:

[0259] The server analyzes facial and voice data from the terminal as data for emotion recognition to determine the user's current emotional state.

[0260] Step 5:

[0261] Based on the analysis results, the server determines a list of optimal locations for the user and sends the detailed information to the terminal.

[0262] Step 6:

[0263] The device displays location options received from the server in its user interface. Each option includes location details and reasons for the recommendation.

[0264] Step 7:

[0265] The user selects their desired location from the displayed options and confirms their selection on their device.

[0266] Step 8:

[0267] The device initiates the reservation process for the selected location and notifies the server of the necessary information.

[0268] Step 9:

[0269] The terminal calls the payment module and prompts the user to enter payment information.

[0270] Step 10:

[0271] The user enters their payment information and completes the transaction. The terminal confirms this and displays a completion notification to the user.

[0272] (Example 1)

[0273] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0274] In today's diverse society, it is a major challenge to quickly and accurately propose activity environments optimized for the individual interests and characteristics of users. Furthermore, it is necessary to improve the user experience by considering behavioral history and real-time emotions during this process.

[0275] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0276] In this invention, the server includes a device means for collecting user attribute information, a device means for analyzing the user's interests and characteristics using an artificial intelligence model based on the user attribute information, and a device means for presenting a suitable activity environment based on the analysis results. This makes it possible to propose an optimal activity environment tailored to the individual needs of the user.

[0277] A "device for collecting user attribute information" is a technology that provides an interface for inputting or acquiring data about users, such as age, interests, and behavioral history.

[0278] A "device that analyzes interests and characteristics using artificial intelligence models" is a technology that utilizes deep learning and machine learning algorithms to analyze users' interests and characteristics from collected attribute information.

[0279] A "device that presents a suitable activity environment" is a technology that visually provides users with the most suitable activity or event options based on analysis.

[0280] A "device that continuously acquires the external state of a user and identifies their emotions" is a technology that recognizes and analyzes the emotional state of a user based on data such as facial expressions and voice acquired in real time.

[0281] A "device that displays information for evaluating the presented activity environment options" is a technology that provides detailed information, reviews, and evaluations for users to refer to when making a selection.

[0282] This invention is a system that proposes an optimal activity environment tailored to the individual needs of users. The system operates with a terminal and a server working in cooperation, and the invention is implemented using multiple means.

[0283] The device provides an interface for collecting user attribute information. This interface is a graphical user interface (GUI) with text fields and dropdown menus for entering data such as age, interests, and behavioral history.

[0284] The server utilizes a generative AI model using Python or TensorFlow to analyze the received attribute information. This model uses deep learning algorithms to analyze the user's interests and characteristics and derive an activity environment suitable for the user. At that time, emotion recognition is performed using the facial expressions and voice data acquired in real time to improve the accuracy of the analysis.

[0285] Also, based on the analysis results, the server generates options for the activity environment suitable for the user and sends them to the terminal. The terminal displays these options on the user interface so that the user can select them. Since specific information and reviews are displayed in the options, the user can compare the details and make a judgment.

[0286] Finally, the reservation for the activity environment selected by the user is advanced, and the terminal cooperates with the payment system to complete the payment safely. Through this series of procedures, the user can smoothly select and reserve the optimal activity environment.

[0287] For example, when the user inputs a prompt sentence such as "Considering that an 8-year-old child is interested in science and is social, please propose activities and events suitable for the child.", the system can make an optimal proposal.

[0288] The flow of the specific process in Example 1 will be described using FIG. 11.

[0289] Step 1:

[0290] The terminal collects attribute information from the user. The user uses the graphical user interface (GUI) on the terminal to input age, interests, and past behavior history. The input information is prepared as formatted data and sent to the server. The output is the basic data for the server to use in the analysis.

[0291] Step 2:

[0292] The server receives attribute information sent from the terminal and performs analysis using a generative AI model. Specifically, it uses libraries such as Python and TensorFlow to extract interests and characteristics using a deep learning model. The input is the user's attribute information, and the output is the analysis result to identify the most suitable activity candidates for the user.

[0293] Step 3:

[0294] The server performs emotion recognition using facial and voice data provided in real time from the terminal. Emotion recognition technology is applied to identify the user's emotional state. The input is real-time facial and voice data, and the output is emotion analysis results to improve analysis accuracy.

[0295] Step 4:

[0296] The server integrates the analysis results of interests and characteristics with the analysis results of emotions to generate a list of candidate activity environments suitable for the user. It extracts possible activities and events from the database and sends them to the terminal as ranked options. The input is the integrated analysis results, and the output is a list of recommended activities presented to the user.

[0297] Step 5:

[0298] The terminal displays a list of potential activity environments sent from the server on the user interface. Each option includes detailed activity information and reputation, and the user makes a selection based on this information. The input is the recommended activity list from the server, and the output is the user's optimal choice.

[0299] Step 6:

[0300] When the user selects an activity environment, the terminal starts the reservation and payment processes. It connects to the reservation system and the payment system and completes the procedures according to the user's selection. The input is the user's selection content, and the output is the confirmation of the completed reservation and payment.

[0301] (Application Example 1)

[0302] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0303] Modern consumers tend to seek personalized experiences based on their interests and preferences, but the systems and services to realize this have not been fully constructed yet. Therefore, it is required to enable the provision of experiences optimized for each user.

[0304] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0305] In this invention, the server includes an information acquisition means for collecting user information, an analysis means for analyzing interests and characteristics using machine learning based on the user information, a processing means for making reservations and electronic payments from the proposed activity location options, and a guidance means for providing a personalized experience to the user based on the in-store environment. Thereby, it becomes possible to provide an optimized experience for each user and seamless reservation and purchase procedures.

[0306] The "information acquisition means" is a device or method for collecting data related to the interests and preferences from the user.

[0307] The "analysis means" is a device or method that analyzes the collected user information and uses machine learning technology to identify the interests and characteristics of an individual.

[0308] "Proposed means" refers to a device or method that presents activity locations and experiences suitable for the user based on the analysis results.

[0309] "Processing means" refers to a device or method that provides the function of making a reservation and electronic payment for an activity location selected by the user.

[0310] "Recognition means" refers to a device or method that has the ability to acquire a user's facial expressions and voice data in real time and analyze their emotions.

[0311] "Display means" refers to a device or method for visually displaying information about the options presented to the user.

[0312] "Guidance means" refers to a device or method that provides users with a personalized experience and information based on the in-store environment in real time.

[0313] This invention is a system designed to provide a personalized experience based on the individual interests and preferences of each user. In the implementation of this invention, the server, terminal, and user each play their respective roles.

[0314] The device collects data on the user's interests and preferences, functioning as a means of information acquisition. This data includes age, past behavioral history, facial expressions, and voice data. This allows the device to capture the user's individual needs in detail. The hardware used includes smartphones and smart glasses, and real-time processing is employed to acquire facial expressions and voice data.

[0315] The server analyzes data received from the terminal using a deep learning model (e.g., TensorFlow) and proposes the optimal experience for the user. This analysis method utilizes machine learning algorithms to extract patterns from the data and reveal interests and characteristics. The processing results are used as a means of making suggestions to the user, and an optimized activity location is presented.

[0316] Users compare the options presented on their terminal, select the best one, and then make a reservation and electronic payment. The reservation system and digital payment platform work together as a processing mechanism, allowing users to enjoy a smooth experience.

[0317] Furthermore, this system can guide users to a personalized experience based on the real-time environment within the store. Specifically, when children's workshops or events are held in the store, the terminal is designed to immediately notify users and encourage their participation.

[0318] For example, if a user enters "I'm interested in science," the server will recommend the most suitable science-related events based on data from similar users in the past. An example of a prompt might be, "I'm looking for science events that my child can participate in. Please tell me about any experiences you recommend at this time."

[0319] Thus, the present invention is a system that combines deep learning and real-time data analysis to enable personalized suggestions and smooth execution for users.

[0320] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0321] Step 1:

[0322] The device collects data from the user regarding their interests and past behavioral history. This input includes age and interests, as well as facial expressions and voice data captured in real time. The device temporarily stores this data in local storage and then sends it to the server.

[0323] Step 2:

[0324] The server receives input data sent from the terminal and passes it on to a deep learning model for processing. The model extracts patterns of user interests and characteristics from the newly received data while comparing it with past data. In this process, machine learning algorithms are used to analyze the data and identify the most suitable activity locations and events for the user as output.

[0325] Step 3:

[0326] Based on the analysis results obtained from the server, the terminal displays suggested activity locations and experience options on the user interface. From this output, the terminal presents detailed information about a specific event to the user and organizes it for easy comparison. The displayed options include location information and available dates and times.

[0327] Step 4:

[0328] The user selects the experience best suited to themselves or their child from the options displayed on the device. Once the user makes a selection, the device immediately connects with the server and begins processing the reservation and electronic payment for the selected experience. After the process is complete, the device displays a confirmation notification to the user.

[0329] Step 5:

[0330] The terminal uses the store's environmental sensors to acquire real-time environmental information. Based on this, the terminal guides the user to new events and discount information tailored to their needs. The user can respond by choosing additional experiences. The guidance is delivered via smartphone or smart glasses.

[0331] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0332] This invention is a system that utilizes user input information and real-time sentiment data to recommend the optimal location personalized to the user, and by combining it with a sentiment engine, it improves the accuracy and user experience.

[0333] System Configuration

[0334] This system consists of a terminal, a server, and an emotion engine. The terminal is responsible for receiving information input from the user and collecting real-time emotion data, which is then transmitted to the server. The server utilizes deep learning and the emotion engine to analyze the user's attributes, characteristics, and emotional state, and based on this, selects potential locations.

[0335] Collection of user information

[0336] The device receives basic information from the user, such as the child's age, interests, and personality, through its user interface. In addition, it uses a camera and microphone to collect the user's facial expressions and voice data in real time. This emotional data is analyzed by an emotion engine to determine the user's current emotional state.

[0337] Data analysis and suggestions for places to belong

[0338] The server uses user information received from the terminal to input into a deep learning model. This model integrates the user's past data and current sentiment data to generate a list of suitable locations for each user.

[0339] The emotional state analysis provided in real time by the emotion engine is used to prioritize suggested locations. For example, if the system determines that the user is stressed, relaxing places and activities will be suggested at the top of the list.

[0340] Information presentation and selection

[0341] The device displays location options prioritized by the server to the user. By comparing this presented information, the user can make the most suitable choice. For example, the server might suggest "a drawing event at a nearby museum" and "a nature observation workshop at a park," and based on the user's current emotional state, display the drawing event higher.

[0342] Reservation and payment

[0343] The terminal proceeds with the reservation and payment process for the location selected by the user. Since all related procedures are integrated and executed at this stage, users can easily prepare to participate in the event.

[0344] According to embodiments of the present invention, users can easily select the optimal environment according to their emotional state, thereby providing a highly satisfying experience.

[0345] The following describes the processing flow.

[0346] Step 1:

[0347] The device displays a screen where the user can input basic information such as the child's age, interests, and personality. At this time, it prepares to collect the user's facial expressions and voice in real time using the camera and microphone.

[0348] Step 2:

[0349] The user follows the instructions on the device, enters the necessary information about the child into a form, and then authorizes the collection of facial expression and voice data.

[0350] Step 3:

[0351] The device transmits facial expression and voice data collected along with user input information to the server using a secure communication protocol.

[0352] Step 4:

[0353] The server analyzes the received data. Here, it uses a deep learning model to analyze the user's interests and characteristics, and an emotion engine to determine their emotional state in real time.

[0354] Step 5:

[0355] The server generates and prioritizes suitable location options for the user based on the analysis results. For example, if relaxation is deemed necessary, a quiet environment will be prioritized.

[0356] Step 6:

[0357] The server sends the generated options and their priority order to the terminal.

[0358] Step 7:

[0359] The terminal displays a list of locations received from the server on the user interface and organizes the information to make it easier for the user to select a location.

[0360] Step 8:

[0361] The user selects the appropriate option from the presented choices and confirms the reservation by following the instructions on the device.

[0362] Step 9:

[0363] The terminal initiates the reservation and payment process for the selected location. This allows the user to easily complete their participation reservation.

[0364] Step 10:

[0365] The terminal displays a reservation and payment confirmation notification to the user, informing them that the process is complete.

[0366] (Example 2)

[0367] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0368] In today's busy lifestyle, people lack the means to find the optimal place that suits their emotions and interests. This leads to users experiencing stress and being forced to make unsatisfactory choices. There is a need to provide personalized recommendations in real time, taking into account the user's emotional state.

[0369] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0370] In this invention, the server includes means for acquiring user information, means for analyzing user characteristics using deep learning, and means for analyzing emotional data in real time. This makes it possible to suggest the optimal location based on the user's emotional state and characteristics.

[0371] "Means for obtaining user information" refers to devices or functions for collecting attribute information from users, such as age, interests, and personality.

[0372] "Methods for analyzing user characteristics using deep learning" refers to a process of interpreting and evaluating user characteristics using AI technology based on acquired user information.

[0373] "Methods for analyzing emotional data in real time" refer to technologies that instantly process emotion-related data obtained from a user's facial expressions and voice to determine their current emotional state.

[0374] "Means of recommending optimal locations" refers to a system or function that, based on analysis results, presents the most suitable locations and activities for individual users.

[0375] "Means of making reservations and payments" refers to means of automating and executing reservation procedures and related payment processes for selected locations and activities.

[0376] This invention is implemented as a personalized recommendation system. The system consists of a terminal, a server, and an emotion analysis engine. The terminal is a device for user information input and real-time collection of emotion data, and is responsible for transmitting the data obtained from the user to the server.

[0377] The device provides the user with an input interface and receives personal and attribute information, such as age, interests, and personality. The device also includes a camera and microphone, which capture the user's real-time facial expressions and voice. This data is analyzed by an emotion analysis engine to determine the user's emotional state.

[0378] The server processes information transmitted from the terminal using deep learning technology and generative AI models. The acquired data is input into the model, and the user's characteristics and emotional state are integrated and analyzed. Based on these analysis results, the server suggests optimal locations and activities for the user. In this process, the results of the emotion analysis engine contribute to the priority of the suggestions.

[0379] Users can review and select location suggestions displayed on the device. After selection, the device automates the booking and payment process, providing users with a quick and smooth experience.

[0380] As a concrete example, the system suggests "relaxing places" based on the user's current emotional state. If the user is feeling stressed, the server will recommend things like "nearby nature parks" or "relaxation classes."

[0381] An example of a prompt used in a generative AI model might be: "The user is currently feeling stressed, so please suggest a place where they can relax. Specifically, please tell me what kind of nearby events or facilities would be suitable."

[0382] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0383] Step 1:

[0384] The terminal obtains basic information from the user through its user interface. This basic information includes age, interests, and personality. The entered information is converted into a digital format and prepared for subsequent data processing.

[0385] Step 2:

[0386] The device collects the user's facial expressions and voice in real time using its camera and microphone. The acquired emotional data is sent to an emotion analysis engine after signal processing. In this process, the data is pre-processed to quantify the user's emotional state.

[0387] Step 3:

[0388] The emotion analysis engine receives and analyzes emotion-related data transmitted from the device. Specifically, it extracts features from the data and identifies emotions such as joy and sadness. The results of this analysis are output as a numerical evaluation that identifies the user's emotional state.

[0389] Step 4:

[0390] The server receives basic user information from the terminal and the results of the sentiment analysis engine, and then integrates and processes them. The received data becomes input and is fed into a deep learning model. This model analyzes the user's attributes and emotional state in combination to generate a list of suitable locations for the user.

[0391] Step 5:

[0392] The server determines priorities based on the generated list of potential locations. Numerical evaluations of sentiment data are reflected in this prioritization. The prioritized list is sent to the terminal as output.

[0393] Step 6:

[0394] The device displays a list of potential locations received from the server to the user. The user can then choose the option that best suits their current mood and preferences. At this stage, the user experience is customized based on the available choices.

[0395] Step 7:

[0396] After the user makes a selection, the terminal begins the reservation and payment process based on that selection. The reservation is completed by accessing the reservation system and processing the generated information as input. Payment processing is carried out similarly, and a confirmation is output indicating that all necessary procedures have been completed.

[0397] (Application Example 2)

[0398] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0399] In recent years, consumers have been seeking personalized shopping experiences not only in physical stores but also in virtual environments. However, traditional online shopping systems and virtual stores have been insufficient in suggesting products and experiences that take into account the user's emotions and real-time state. As a result, there is a challenge in that users cannot smoothly select appropriate products and experiences that match their emotions at any given time.

[0400] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0401] In this invention, the server includes a device means for collecting user data, a device means for analyzing the user's interests and characteristics using machine learning based on the user data, and a device means for suggesting facilities suitable for the user based on the analyzed results. This makes it possible to suggest personalized products and experiences based on real-time emotional data.

[0402] A "device for collecting user data" is a device that acquires basic user information, facial expressions, voice, emotional data, etc., and transmits it to a server for analysis.

[0403] A "device that analyzes interests and characteristics using machine learning" is a system that uses acquired user data and data analysis techniques to understand users' areas of interest and individual characteristics.

[0404] A "device that suggests facilities suitable for users" is a system that suggests the most suitable products and experiences for users based on their analyzed interests, characteristics, and emotional state.

[0405] A "device that acquires emotional data in real time and recognizes emotional states" is a technology that acquires the user's facial expressions and voice through a camera and microphone, and quickly determines their emotional state at that moment.

[0406] A "device that displays products or experiences in a virtual environment according to the user's emotional state" is a system that visually presents the most suitable products or experiences based on the user's real-time emotional state within a virtual reality or augmented reality environment.

[0407] A "generative AI model" is an artificial intelligence algorithm used to generate and suggest personalized products and experiences based on user data and emotional states.

[0408] A "prompt sentence" is an instruction given to a generative AI model to obtain a specific output. Based on the content of the sentence, the AI ​​suggests appropriate products or experiences.

[0409] The system for realizing this invention enables the collection of information from users and the analysis of emotional data in real time. The system includes a terminal for collecting data, a server for analysis and making suggestions, and an AI model.

[0410] The devices used are such as smart glasses or smartphones. These devices are equipped with cameras and microphones to capture the user's facial expressions and voice, and collect emotional data in real time. The collected data is sent to a server.

[0411] The server uses machine learning algorithms and an emotion analysis engine to analyze the user's characteristics and current emotional state. This makes it possible to suggest facilities and experiences based on the user's interests and emotions. The emotion analysis engine uses software such as the Emotion Analysis SDK to quickly determine the user's emotional state.

[0412] The generative AI model runs on a server and generates personalized products and experiences tailored to the user's emotional state. For example, if a user is stressed, the AI ​​model can suggest relaxing candles or music. If they are excited, it can suggest new games or exciting events.

[0413] The terminal displays suggestions from the server to the user. The user can browse the presented options and select experiences or products that interest them. Depending on the user's selection, booking and payment procedures may also be carried out through the terminal.

[0414] As a concrete example, when a user visits a virtual store, the emotion analysis engine analyzes the user's facial expressions, and if it determines that the user's current emotional state is "excited," the generative AI model could suggest a "new adventure game."

[0415] To generate suggestions for a generative AI model, use the following prompt statements:

[0416] "Please suggest new or unique products that would be suitable for someone currently experiencing an excited emotional state."

[0417] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0418] Step 1:

[0419] The device collects the user's facial expressions and voice in real time. This involves capturing facial data using a camera and recording voice data using a microphone. This raw data is pre-processed for emotion analysis before being sent to the server. The input data consists of the user's facial expressions and voice data, while the output is pre-processed data.

[0420] Step 2:

[0421] The server analyzes the received data using the Emotion Analysis SDK. First, it takes in facial expression data and voice data as input, which the emotion analysis engine processes. As a result of the analysis, it outputs the user's current emotional state. This emotional state is used in the subsequent suggestion process.

[0422] Step 3:

[0423] The server inputs the analyzed emotional state and the user's past interests into a machine learning model. This model performs analysis to enumerate the products and experiences best suited to the user. The output is a personalized list of suggestions based on the user's emotional state.

[0424] Step 4:

[0425] The terminal displays a list of suggestions received from the server to the user. The user selects products or experiences that interest them from this list. A user interface is used here to facilitate visual selection. The displayed suggestions are optimized by a generative AI model, and the user makes their selection based on these suggestions.

[0426] Step 5:

[0427] Once the user makes a selection, the terminal sends that information back to the server to initiate the booking and payment process. The input here is the user's selection information, and the output is the booking confirmation and payment completion for the selected experience or product. The terminal processes these steps in the background and displays the completion status to the user.

[0428] This series of processing steps allows users to receive product and experience suggestions that match their current emotions, enabling them to smoothly select and purchase items.

[0429] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0430] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0431] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0432] [Third Embodiment]

[0433] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0434] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0435] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0436] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0437] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0438] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0439] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0440] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0441] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0442] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0443] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0444] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0445] This invention is a system that recommends the optimal location based on information entered by the user. This system effectively functions by proposing a personalized environment according to the user's characteristics and interests, and by supporting a series of procedures including reservation and payment. The operation of this system will be explained from the perspectives of the user, server, and terminal, with specific examples.

[0446] Step 1: Enter Information

[0447] First, the device provides an interface for collecting basic information about the child from the user. This information includes age, interests, and past behavioral history. For example, the user might input information about an 8-year-old child such as "interested in science" and "sociable."

[0448] Step 2: Data Analysis

[0449] Next, the server analyzes the received information using a deep learning model. The model extracts interests and characteristics to identify the optimal location for the child, comparing them with past data. In this process, facial and voice data transmitted in real time from the device are also used, and emotion recognition technology is employed to perform an even more accurate analysis. From the analysis results, the server generates multiple location options.

[0450] Step 3: Presentation of recommendations and selection

[0451] The device displays a list of potential locations sent from the server on the user interface. Each option is displayed along with specific activity details and location information. This allows the user to compare the displayed options and choose the most suitable one.

[0452] For example, based on the analysis, two options might be displayed: "Science Museum" and "Workshop at a Community Center." The options would include descriptions of their appeal and ratings to help the user make a choice.

[0453] Step 4: Booking and Payment

[0454] Finally, once the user selects a specific location, the terminal begins the process to proceed with the reservation. Furthermore, through the integrated payment system, users can make secure and fast payments. This ensures a smooth and seamless process from reservation to payment.

[0455] Through the embodiments described above, the present invention can provide users with an efficient and personalized activity environment for children, and support the realization of a manageable daily life.

[0456] The following describes the processing flow.

[0457] Step 1:

[0458] The device displays an interface for entering the user's basic information. This information includes the child's age, interests, personality, and behavioral history. The user enters the information according to the instructions.

[0459] Step 2:

[0460] The terminal temporarily stores the input information and sends it to the server using a secure communication protocol.

[0461] Step 3:

[0462] The server inputs the received information into a deep learning model. This model analyzes the user's preferences and characteristics to generate appropriate location options.

[0463] Step 4:

[0464] The server analyzes facial and voice data from the terminal as data for emotion recognition to determine the user's current emotional state.

[0465] Step 5:

[0466] Based on the analysis results, the server determines a list of optimal locations for the user and sends the detailed information to the terminal.

[0467] Step 6:

[0468] The device displays location options received from the server in its user interface. Each option includes location details and reasons for the recommendation.

[0469] Step 7:

[0470] The user selects their desired location from the displayed options and confirms their selection on their device.

[0471] Step 8:

[0472] The device initiates the reservation process for the selected location and notifies the server of the necessary information.

[0473] Step 9:

[0474] The terminal calls the payment module and prompts the user to enter payment information.

[0475] Step 10:

[0476] The user enters their payment information and completes the transaction. The terminal confirms this and displays a completion notification to the user.

[0477] (Example 1)

[0478] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0479] In today's diverse society, it is a major challenge to quickly and accurately propose activity environments optimized for the individual interests and characteristics of users. Furthermore, it is necessary to improve the user experience by considering behavioral history and real-time emotions during this process.

[0480] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0481] In this invention, the server includes a device means for collecting user attribute information, a device means for analyzing the user's interests and characteristics using an artificial intelligence model based on the user attribute information, and a device means for presenting a suitable activity environment based on the analysis results. This makes it possible to propose an optimal activity environment tailored to the individual needs of the user.

[0482] A "device for collecting user attribute information" is a technology that provides an interface for inputting or acquiring data about users, such as age, interests, and behavioral history.

[0483] A "device that analyzes interests and characteristics using artificial intelligence models" is a technology that utilizes deep learning and machine learning algorithms to analyze users' interests and characteristics from collected attribute information.

[0484] A "device that presents a suitable activity environment" is a technology that visually provides users with the most suitable activity or event options based on analysis.

[0485] A "device that continuously acquires the external state of a user and identifies their emotions" is a technology that recognizes and analyzes the emotional state of a user based on data such as facial expressions and voice acquired in real time.

[0486] A "device that displays information for evaluating the presented activity environment options" is a technology that provides detailed information, reviews, and evaluations for users to refer to when making a selection.

[0487] This invention is a system that proposes an optimal activity environment tailored to the individual needs of users. The system operates with a terminal and a server working in cooperation, and the invention is implemented using multiple means.

[0488] The device provides an interface for collecting user attribute information. This interface is a graphical user interface (GUI) with text fields and dropdown menus for entering data such as age, interests, and behavioral history.

[0489] The server utilizes generative AI models based on Python and TensorFlow to analyze the received attribute information. These models employ deep learning algorithms to analyze the user's interests and characteristics, and then determine an activity environment suitable for the user. In doing so, real-time facial and voice data are used for emotion recognition, improving the accuracy of the analysis.

[0490] Furthermore, based on the analysis results, the server generates a selection of activity environments suitable for the user and sends them to the terminal. The terminal displays these options in its user interface, allowing the user to select one. The options include specific information and reviews, enabling the user to compare details and make a decision.

[0491] Ultimately, the user proceeds with booking their chosen activity environment, and the terminal integrates with the payment system to securely complete the payment. This entire process allows users to smoothly select and book the optimal activity environment.

[0492] For example, if a user enters a prompt such as, "Considering that my 8-year-old child is interested in science and is sociable, please suggest activities and events that would be perfect for this child," the system can provide optimal suggestions.

[0493] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0494] Step 1:

[0495] The terminal collects attribute information from the user. The user uses a graphical user interface (GUI) on the terminal to input their age, interests, and past behavioral history. The input information is prepared as formatted data and sent to the server. The output is the basic data that the server uses for analysis.

[0496] Step 2:

[0497] The server receives attribute information sent from the terminal and performs analysis using a generative AI model. Specifically, it uses libraries such as Python and TensorFlow to extract interests and characteristics using a deep learning model. The input is the user's attribute information, and the output is the analysis result to identify the most suitable activity candidates for the user.

[0498] Step 3:

[0499] The server performs emotion recognition using facial and voice data provided in real time from the terminal. Emotion recognition technology is applied to identify the user's emotional state. The input is real-time facial and voice data, and the output is emotion analysis results to improve analysis accuracy.

[0500] Step 4:

[0501] The server integrates the analysis results of interests and characteristics with the analysis results of emotions to generate a list of candidate activity environments suitable for the user. It extracts possible activities and events from the database and sends them to the terminal as ranked options. The input is the integrated analysis results, and the output is a list of recommended activities presented to the user.

[0502] Step 5:

[0503] The terminal displays a list of potential activity environments sent from the server on the user interface. Each option includes detailed activity information and reputation, and the user makes a selection based on this information. The input is the recommended activity list from the server, and the output is the user's optimal choice.

[0504] Step 6:

[0505] Once the user selects their activity environment, the terminal initiates the booking and payment process. It connects to the booking and payment systems and completes the procedure according to the user's selections. The input is the user's selections, and the output is a confirmation of the completed booking and payment.

[0506] (Application Example 1)

[0507] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0508] Modern consumers tend to strongly desire personalized experiences based on their interests and preferences, but the systems and services to achieve this are still not sufficiently developed. Therefore, there is a need to enable the provision of experiences optimized for each individual user.

[0509] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0510] In this invention, the server includes information acquisition means for collecting user information, analysis means for analyzing interests and characteristics using machine learning based on the user information, processing means for making reservations and electronic payments from suggested activity location options, and guidance means for providing a personalized experience to the user based on the in-store environment. This enables the provision of an optimized experience for each user and seamless reservation and purchase procedures.

[0511] "Information acquisition means" refers to a device or method for collecting data on users' interests and preferences.

[0512] "Analysis means" refers to a device or method that uses machine learning techniques to analyze collected user information and identify individual interests and characteristics.

[0513] "Proposed means" refers to a device or method that presents activity locations and experiences suitable for the user based on the analysis results.

[0514] "Processing means" refers to a device or method that provides the function of making a reservation and electronic payment for an activity location selected by the user.

[0515] "Recognition means" refers to a device or method that has the ability to acquire a user's facial expressions and voice data in real time and analyze their emotions.

[0516] "Display means" refers to a device or method for visually displaying information about the options presented to the user.

[0517] "Guidance means" refers to a device or method that provides users with a personalized experience and information based on the in-store environment in real time.

[0518] This invention is a system designed to provide a personalized experience based on the individual interests and preferences of each user. In the implementation of this invention, the server, terminal, and user each play their respective roles.

[0519] The device collects data on the user's interests and preferences, functioning as a means of information acquisition. This data includes age, past behavioral history, facial expressions, and voice data. This allows the device to capture the user's individual needs in detail. The hardware used includes smartphones and smart glasses, and real-time processing is employed to acquire facial expressions and voice data.

[0520] The server analyzes data received from the terminal using a deep learning model (e.g., TensorFlow) and proposes the optimal experience for the user. This analysis method utilizes machine learning algorithms to extract patterns from the data and reveal interests and characteristics. The processing results are used as a means of making suggestions to the user, and an optimized activity location is presented.

[0521] Users compare the options presented on their terminal, select the best one, and then make a reservation and electronic payment. The reservation system and digital payment platform work together as a processing mechanism, allowing users to enjoy a smooth experience.

[0522] Furthermore, this system can guide users to a personalized experience based on the real-time environment within the store. Specifically, when children's workshops or events are held in the store, the terminal is designed to immediately notify users and encourage their participation.

[0523] For example, if a user enters "I'm interested in science," the server will recommend the most suitable science-related events based on data from similar users in the past. An example of a prompt might be, "I'm looking for science events that my child can participate in. Please tell me about any experiences you recommend at this time."

[0524] Thus, the present invention is a system that combines deep learning and real-time data analysis to enable personalized suggestions and smooth execution for users.

[0525] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0526] Step 1:

[0527] The device collects data from the user regarding their interests and past behavioral history. This input includes age and interests, as well as facial expressions and voice data captured in real time. The device temporarily stores this data in local storage and then sends it to the server.

[0528] Step 2:

[0529] The server receives input data sent from the terminal and passes it on to a deep learning model for processing. The model extracts patterns of user interests and characteristics from the newly received data while comparing it with past data. In this process, machine learning algorithms are used to analyze the data and identify the most suitable activity locations and events for the user as output.

[0530] Step 3:

[0531] Based on the analysis results obtained from the server, the terminal displays suggested activity locations and experience options on the user interface. From this output, the terminal presents detailed information about a specific event to the user and organizes it for easy comparison. The displayed options include location information and available dates and times.

[0532] Step 4:

[0533] The user selects the experience best suited to themselves or their child from the options displayed on the device. Once the user makes a selection, the device immediately connects with the server and begins processing the reservation and electronic payment for the selected experience. After the process is complete, the device displays a confirmation notification to the user.

[0534] Step 5:

[0535] The terminal uses the store's environmental sensors to acquire real-time environmental information. Based on this, the terminal guides the user to new events and discount information tailored to their needs. The user can respond by choosing additional experiences. The guidance is delivered via smartphone or smart glasses.

[0536] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0537] This invention is a system that utilizes user input information and real-time sentiment data to recommend the optimal location personalized to the user, and by combining it with a sentiment engine, it improves the accuracy and user experience.

[0538] System Configuration

[0539] This system consists of a terminal, a server, and an emotion engine. The terminal is responsible for receiving information input from the user and collecting real-time emotion data, which is then transmitted to the server. The server utilizes deep learning and the emotion engine to analyze the user's attributes, characteristics, and emotional state, and based on this, selects potential locations.

[0540] Collection of user information

[0541] The device receives basic information from the user, such as the child's age, interests, and personality, through its user interface. In addition, it uses a camera and microphone to collect the user's facial expressions and voice data in real time. This emotional data is analyzed by an emotion engine to determine the user's current emotional state.

[0542] Data analysis and suggestions for places to belong

[0543] The server uses user information received from the terminal to input into a deep learning model. This model integrates the user's past data and current sentiment data to generate a list of suitable locations for each user.

[0544] The emotional state analysis provided in real time by the emotion engine is used to prioritize suggested locations. For example, if the system determines that the user is stressed, relaxing places and activities will be suggested at the top of the list.

[0545] Information presentation and selection

[0546] The device displays location options prioritized by the server to the user. By comparing this presented information, the user can make the most suitable choice. For example, the server might suggest "a drawing event at a nearby museum" and "a nature observation workshop at a park," and based on the user's current emotional state, display the drawing event higher.

[0547] Reservation and payment

[0548] The terminal proceeds with the reservation and payment process for the location selected by the user. Since all related procedures are integrated and executed at this stage, users can easily prepare to participate in the event.

[0549] According to embodiments of the present invention, users can easily select the optimal environment according to their emotional state, thereby providing a highly satisfying experience.

[0550] The following describes the processing flow.

[0551] Step 1:

[0552] The device displays a screen where the user can input basic information such as the child's age, interests, and personality. At this time, it prepares to collect the user's facial expressions and voice in real time using the camera and microphone.

[0553] Step 2:

[0554] The user follows the instructions on the device, enters the necessary information about the child into a form, and then authorizes the collection of facial expression and voice data.

[0555] Step 3:

[0556] The device transmits facial expression and voice data collected along with user input information to the server using a secure communication protocol.

[0557] Step 4:

[0558] The server analyzes the received data. Here, it uses a deep learning model to analyze the user's interests and characteristics, and an emotion engine to determine their emotional state in real time.

[0559] Step 5:

[0560] The server generates and prioritizes suitable location options for the user based on the analysis results. For example, if relaxation is deemed necessary, a quiet environment will be prioritized.

[0561] Step 6:

[0562] The server sends the generated options and their priority order to the terminal.

[0563] Step 7:

[0564] The terminal displays a list of locations received from the server on the user interface and organizes the information to make it easier for the user to select a location.

[0565] Step 8:

[0566] The user selects the appropriate option from the presented choices and confirms the reservation by following the instructions on the device.

[0567] Step 9:

[0568] The terminal initiates the reservation and payment process for the selected location. This allows the user to easily complete their participation reservation.

[0569] Step 10:

[0570] The terminal displays a reservation and payment confirmation notification to the user, informing them that the process is complete.

[0571] (Example 2)

[0572] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0573] In today's busy lifestyle, people lack the means to find the optimal place that suits their emotions and interests. This leads to users experiencing stress and being forced to make unsatisfactory choices. There is a need to provide personalized recommendations in real time, taking into account the user's emotional state.

[0574] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0575] In this invention, the server includes means for acquiring user information, means for analyzing user characteristics using deep learning, and means for analyzing emotional data in real time. This makes it possible to suggest the optimal location based on the user's emotional state and characteristics.

[0576] "Means for obtaining user information" refers to devices or functions for collecting attribute information from users, such as age, interests, and personality.

[0577] "Methods for analyzing user characteristics using deep learning" refers to a process of interpreting and evaluating user characteristics using AI technology based on acquired user information.

[0578] "Methods for analyzing emotional data in real time" refer to technologies that instantly process emotion-related data obtained from a user's facial expressions and voice to determine their current emotional state.

[0579] "Means of recommending optimal locations" refers to a system or function that, based on analysis results, presents the most suitable locations and activities for individual users.

[0580] "Means of making reservations and payments" refers to means of automating and executing reservation procedures and related payment processes for selected locations and activities.

[0581] This invention is implemented as a personalized recommendation system. The system consists of a terminal, a server, and an emotion analysis engine. The terminal is a device for user information input and real-time collection of emotion data, and is responsible for transmitting the data obtained from the user to the server.

[0582] The device provides the user with an input interface and receives personal and attribute information, such as age, interests, and personality. The device also includes a camera and microphone, which capture the user's real-time facial expressions and voice. This data is analyzed by an emotion analysis engine to determine the user's emotional state.

[0583] The server processes information transmitted from the terminal using deep learning technology and generative AI models. The acquired data is input into the model, and the user's characteristics and emotional state are integrated and analyzed. Based on these analysis results, the server suggests optimal locations and activities for the user. In this process, the results of the emotion analysis engine contribute to the priority of the suggestions.

[0584] Users can review and select location suggestions displayed on the device. After selection, the device automates the booking and payment process, providing users with a quick and smooth experience.

[0585] As a concrete example, the system suggests "relaxing places" based on the user's current emotional state. If the user is feeling stressed, the server will recommend things like "nearby nature parks" or "relaxation classes."

[0586] An example of a prompt used in a generative AI model might be: "The user is currently feeling stressed, so please suggest a place where they can relax. Specifically, please tell me what kind of nearby events or facilities would be suitable."

[0587] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0588] Step 1:

[0589] The terminal obtains basic information from the user through its user interface. This basic information includes age, interests, and personality. The entered information is converted into a digital format and prepared for subsequent data processing.

[0590] Step 2:

[0591] The device collects the user's facial expressions and voice in real time using its camera and microphone. The acquired emotional data is sent to an emotion analysis engine after signal processing. In this process, the data is pre-processed to quantify the user's emotional state.

[0592] Step 3:

[0593] The emotion analysis engine receives and analyzes emotion-related data transmitted from the device. Specifically, it extracts features from the data and identifies emotions such as joy and sadness. The results of this analysis are output as a numerical evaluation that identifies the user's emotional state.

[0594] Step 4:

[0595] The server receives basic user information from the terminal and the results of the sentiment analysis engine, and then integrates and processes them. The received data becomes input and is fed into a deep learning model. This model analyzes the user's attributes and emotional state in combination to generate a list of suitable locations for the user.

[0596] Step 5:

[0597] The server determines priorities based on the generated list of potential locations. Numerical evaluations of sentiment data are reflected in this prioritization. The prioritized list is sent to the terminal as output.

[0598] Step 6:

[0599] The device displays a list of potential locations received from the server to the user. The user can then choose the option that best suits their current mood and preferences. At this stage, the user experience is customized based on the available choices.

[0600] Step 7:

[0601] After the user makes a selection, the terminal begins the reservation and payment process based on that selection. The reservation is completed by accessing the reservation system and processing the generated information as input. Payment processing is carried out similarly, and a confirmation is output indicating that all necessary procedures have been completed.

[0602] (Application Example 2)

[0603] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0604] In recent years, consumers have been seeking personalized shopping experiences not only in physical stores but also in virtual environments. However, traditional online shopping systems and virtual stores have been insufficient in suggesting products and experiences that take into account the user's emotions and real-time state. As a result, there is a challenge in that users cannot smoothly select appropriate products and experiences that match their emotions at any given time.

[0605] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0606] In this invention, the server includes a device means for collecting user data, a device means for analyzing the user's interests and characteristics using machine learning based on the user data, and a device means for suggesting facilities suitable for the user based on the analyzed results. This makes it possible to suggest personalized products and experiences based on real-time emotional data.

[0607] A "device for collecting user data" is a device that acquires basic user information, facial expressions, voice, emotional data, etc., and transmits it to a server for analysis.

[0608] A "device that analyzes interests and characteristics using machine learning" is a system that uses acquired user data and data analysis techniques to understand users' areas of interest and individual characteristics.

[0609] A "device that suggests facilities suitable for users" is a system that suggests the most suitable products and experiences for users based on their analyzed interests, characteristics, and emotional state.

[0610] A "device that acquires emotional data in real time and recognizes emotional states" is a technology that acquires the user's facial expressions and voice through a camera and microphone, and quickly determines their emotional state at that moment.

[0611] A "device that displays products or experiences in a virtual environment according to the user's emotional state" is a system that visually presents the most suitable products or experiences based on the user's real-time emotional state within a virtual reality or augmented reality environment.

[0612] A "generative AI model" is an artificial intelligence algorithm used to generate and suggest personalized products and experiences based on user data and emotional states.

[0613] A "prompt sentence" is an instruction given to a generative AI model to obtain a specific output. Based on the content of the sentence, the AI ​​suggests appropriate products or experiences.

[0614] The system for realizing this invention enables the collection of information from users and the analysis of emotional data in real time. The system includes a terminal for collecting data, a server for analysis and making suggestions, and an AI model.

[0615] The devices used are such as smart glasses or smartphones. These devices are equipped with cameras and microphones to capture the user's facial expressions and voice, and collect emotional data in real time. The collected data is sent to a server.

[0616] The server uses machine learning algorithms and an emotion analysis engine to analyze the user's characteristics and current emotional state. This makes it possible to suggest facilities and experiences based on the user's interests and emotions. The emotion analysis engine uses software such as the Emotion Analysis SDK to quickly determine the user's emotional state.

[0617] The generative AI model runs on a server and generates personalized products and experiences tailored to the user's emotional state. For example, if a user is stressed, the AI ​​model can suggest relaxing candles or music. If they are excited, it can suggest new games or exciting events.

[0618] The terminal displays suggestions from the server to the user. The user can browse the presented options and select experiences or products that interest them. Depending on the user's selection, booking and payment procedures may also be carried out through the terminal.

[0619] As a concrete example, when a user visits a virtual store, the emotion analysis engine analyzes the user's facial expressions, and if it determines that the user's current emotional state is "excited," the generative AI model could suggest a "new adventure game."

[0620] To generate suggestions for a generative AI model, use the following prompt statements:

[0621] "Please suggest new or unique products that would be suitable for someone currently experiencing an excited emotional state."

[0622] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0623] Step 1:

[0624] The device collects the user's facial expressions and voice in real time. This involves capturing facial data using a camera and recording voice data using a microphone. This raw data is pre-processed for emotion analysis before being sent to the server. The input data consists of the user's facial expressions and voice data, while the output is pre-processed data.

[0625] Step 2:

[0626] The server analyzes the received data using the Emotion Analysis SDK. First, it takes in facial expression data and voice data as input, which the emotion analysis engine processes. As a result of the analysis, it outputs the user's current emotional state. This emotional state is used in the subsequent suggestion process.

[0627] Step 3:

[0628] The server inputs the analyzed emotional state and the user's past interests into a machine learning model. This model performs analysis to enumerate the products and experiences best suited to the user. The output is a personalized list of suggestions based on the user's emotional state.

[0629] Step 4:

[0630] The terminal displays a list of suggestions received from the server to the user. The user selects products or experiences that interest them from this list. A user interface is used here to facilitate visual selection. The displayed suggestions are optimized by a generative AI model, and the user makes their selection based on these suggestions.

[0631] Step 5:

[0632] Once the user makes a selection, the terminal sends that information back to the server to initiate the booking and payment process. The input here is the user's selection information, and the output is the booking confirmation and payment completion for the selected experience or product. The terminal processes these steps in the background and displays the completion status to the user.

[0633] This series of processing steps allows users to receive product and experience suggestions that match their current emotions, enabling them to smoothly select and purchase items.

[0634] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0635] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0636] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0637] [Fourth Embodiment]

[0638] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0639] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0640] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0641] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0642] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0643] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0644] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0645] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0646] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0647] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0648] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0649] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0650] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0651] This invention is a system that recommends the optimal location based on information entered by the user. This system effectively functions by proposing a personalized environment according to the user's characteristics and interests, and by supporting a series of procedures including reservation and payment. The operation of this system will be explained from the perspectives of the user, server, and terminal, with specific examples.

[0652] Step 1: Enter Information

[0653] First, the device provides an interface for collecting basic information about the child from the user. This information includes age, interests, and past behavioral history. For example, the user might input information about an 8-year-old child such as "interested in science" and "sociable."

[0654] Step 2: Data Analysis

[0655] Next, the server analyzes the received information using a deep learning model. The model extracts interests and characteristics to identify the optimal location for the child, comparing them with past data. In this process, facial and voice data transmitted in real time from the device are also used, and emotion recognition technology is employed to perform an even more accurate analysis. From the analysis results, the server generates multiple location options.

[0656] Step 3: Presentation of recommendations and selection

[0657] The device displays a list of potential locations sent from the server on the user interface. Each option is displayed along with specific activity details and location information. This allows the user to compare the displayed options and choose the most suitable one.

[0658] For example, based on the analysis, two options might be displayed: "Science Museum" and "Workshop at a Community Center." The options would include descriptions of their appeal and ratings to help the user make a choice.

[0659] Step 4: Booking and Payment

[0660] Finally, once the user selects a specific location, the terminal begins the process to proceed with the reservation. Furthermore, through the integrated payment system, users can make secure and fast payments. This ensures a smooth and seamless process from reservation to payment.

[0661] Through the embodiments described above, the present invention can provide users with an efficient and personalized activity environment for children, and support the realization of a manageable daily life.

[0662] The following describes the processing flow.

[0663] Step 1:

[0664] The device displays an interface for entering the user's basic information. This information includes the child's age, interests, personality, and behavioral history. The user enters the information according to the instructions.

[0665] Step 2:

[0666] The terminal temporarily stores the input information and sends it to the server using a secure communication protocol.

[0667] Step 3:

[0668] The server inputs the received information into a deep learning model. This model analyzes the user's preferences and characteristics to generate appropriate location options.

[0669] Step 4:

[0670] The server analyzes facial and voice data from the terminal as data for emotion recognition to determine the user's current emotional state.

[0671] Step 5:

[0672] Based on the analysis results, the server determines a list of optimal locations for the user and sends the detailed information to the terminal.

[0673] Step 6:

[0674] The device displays location options received from the server in its user interface. Each option includes location details and reasons for the recommendation.

[0675] Step 7:

[0676] The user selects their desired location from the displayed options and confirms their selection on their device.

[0677] Step 8:

[0678] The device initiates the reservation process for the selected location and notifies the server of the necessary information.

[0679] Step 9:

[0680] The terminal calls the payment module and prompts the user to enter payment information.

[0681] Step 10:

[0682] The user enters their payment information and completes the transaction. The terminal confirms this and displays a completion notification to the user.

[0683] (Example 1)

[0684] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0685] In today's diverse society, it is a major challenge to quickly and accurately propose activity environments optimized for the individual interests and characteristics of users. Furthermore, it is necessary to improve the user experience by considering behavioral history and real-time emotions during this process.

[0686] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0687] In this invention, the server includes a device means for collecting user attribute information, a device means for analyzing the user's interests and characteristics using an artificial intelligence model based on the user attribute information, and a device means for presenting a suitable activity environment based on the analysis results. This makes it possible to propose an optimal activity environment tailored to the individual needs of the user.

[0688] A "device for collecting user attribute information" is a technology that provides an interface for inputting or acquiring data about users, such as age, interests, and behavioral history.

[0689] A "device that analyzes interests and characteristics using artificial intelligence models" is a technology that utilizes deep learning and machine learning algorithms to analyze users' interests and characteristics from collected attribute information.

[0690] A "device that presents a suitable activity environment" is a technology that visually provides users with the most suitable activity or event options based on analysis.

[0691] A "device that continuously acquires the external state of a user and identifies their emotions" is a technology that recognizes and analyzes the emotional state of a user based on data such as facial expressions and voice acquired in real time.

[0692] A "device that displays information for evaluating the presented activity environment options" is a technology that provides detailed information, reviews, and evaluations for users to refer to when making a selection.

[0693] This invention is a system that proposes an optimal activity environment tailored to the individual needs of users. The system operates with a terminal and a server working in cooperation, and the invention is implemented using multiple means.

[0694] The device provides an interface for collecting user attribute information. This interface is a graphical user interface (GUI) with text fields and dropdown menus for entering data such as age, interests, and behavioral history.

[0695] The server utilizes generative AI models based on Python and TensorFlow to analyze the received attribute information. These models employ deep learning algorithms to analyze the user's interests and characteristics, and then determine an activity environment suitable for the user. In doing so, real-time facial and voice data are used for emotion recognition, improving the accuracy of the analysis.

[0696] Furthermore, based on the analysis results, the server generates a selection of activity environments suitable for the user and sends them to the terminal. The terminal displays these options in its user interface, allowing the user to select one. The options include specific information and reviews, enabling the user to compare details and make a decision.

[0697] Ultimately, the user proceeds with booking their chosen activity environment, and the terminal integrates with the payment system to securely complete the payment. This entire process allows users to smoothly select and book the optimal activity environment.

[0698] For example, if a user enters a prompt such as, "Considering that my 8-year-old child is interested in science and is sociable, please suggest activities and events that would be perfect for this child," the system can provide optimal suggestions.

[0699] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0700] Step 1:

[0701] The terminal collects attribute information from the user. The user uses a graphical user interface (GUI) on the terminal to input their age, interests, and past behavioral history. The input information is prepared as formatted data and sent to the server. The output is the basic data that the server uses for analysis.

[0702] Step 2:

[0703] The server receives attribute information sent from the terminal and performs analysis using a generative AI model. Specifically, it uses libraries such as Python and TensorFlow to extract interests and characteristics using a deep learning model. The input is the user's attribute information, and the output is the analysis result to identify the most suitable activity candidates for the user.

[0704] Step 3:

[0705] The server performs emotion recognition using facial and voice data provided in real time from the terminal. Emotion recognition technology is applied to identify the user's emotional state. The input is real-time facial and voice data, and the output is emotion analysis results to improve analysis accuracy.

[0706] Step 4:

[0707] The server integrates the analysis results of interests and characteristics with the analysis results of emotions to generate a list of candidate activity environments suitable for the user. It extracts possible activities and events from the database and sends them to the terminal as ranked options. The input is the integrated analysis results, and the output is a list of recommended activities presented to the user.

[0708] Step 5:

[0709] The terminal displays a list of potential activity environments sent from the server on the user interface. Each option includes detailed activity information and reputation, and the user makes a selection based on this information. The input is the recommended activity list from the server, and the output is the user's optimal choice.

[0710] Step 6:

[0711] Once the user selects their activity environment, the terminal initiates the booking and payment process. It connects to the booking and payment systems and completes the procedure according to the user's selections. The input is the user's selections, and the output is a confirmation of the completed booking and payment.

[0712] (Application Example 1)

[0713] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0714] Modern consumers tend to strongly desire personalized experiences based on their interests and preferences, but the systems and services to achieve this are still not sufficiently developed. Therefore, there is a need to enable the provision of experiences optimized for each individual user.

[0715] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0716] In this invention, the server includes information acquisition means for collecting user information, analysis means for analyzing interests and characteristics using machine learning based on the user information, processing means for making reservations and electronic payments from suggested activity location options, and guidance means for providing a personalized experience to the user based on the in-store environment. This enables the provision of an optimized experience for each user and seamless reservation and purchase procedures.

[0717] "Information acquisition means" refers to a device or method for collecting data on users' interests and preferences.

[0718] "Analysis means" refers to a device or method that uses machine learning techniques to analyze collected user information and identify individual interests and characteristics.

[0719] "Proposed means" refers to a device or method that presents activity locations and experiences suitable for the user based on the analysis results.

[0720] "Processing means" refers to a device or method that provides the function of making a reservation and electronic payment for an activity location selected by the user.

[0721] "Recognition means" refers to a device or method that has the ability to acquire a user's facial expressions and voice data in real time and analyze their emotions.

[0722] "Display means" refers to a device or method for visually displaying information about the options presented to the user.

[0723] "Guidance means" refers to a device or method that provides users with a personalized experience and information based on the in-store environment in real time.

[0724] This invention is a system designed to provide a personalized experience based on the individual interests and preferences of each user. In the implementation of this invention, the server, terminal, and user each play their respective roles.

[0725] The device collects data on the user's interests and preferences, functioning as a means of information acquisition. This data includes age, past behavioral history, facial expressions, and voice data. This allows the device to capture the user's individual needs in detail. The hardware used includes smartphones and smart glasses, and real-time processing is employed to acquire facial expressions and voice data.

[0726] The server analyzes data received from the terminal using a deep learning model (e.g., TensorFlow) and proposes the optimal experience for the user. This analysis method utilizes machine learning algorithms to extract patterns from the data and reveal interests and characteristics. The processing results are used as a means of making suggestions to the user, and an optimized activity location is presented.

[0727] Users compare the options presented on their terminal, select the best one, and then make a reservation and electronic payment. The reservation system and digital payment platform work together as a processing mechanism, allowing users to enjoy a smooth experience.

[0728] Furthermore, this system can guide users to a personalized experience based on the real-time environment within the store. Specifically, when children's workshops or events are held in the store, the terminal is designed to immediately notify users and encourage their participation.

[0729] For example, if a user enters "I'm interested in science," the server will recommend the most suitable science-related events based on data from similar users in the past. An example of a prompt might be, "I'm looking for science events that my child can participate in. Please tell me about any experiences you recommend at this time."

[0730] Thus, the present invention is a system that combines deep learning and real-time data analysis to enable personalized suggestions and smooth execution for users.

[0731] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0732] Step 1:

[0733] The device collects data from the user regarding their interests and past behavioral history. This input includes age and interests, as well as facial expressions and voice data captured in real time. The device temporarily stores this data in local storage and then sends it to the server.

[0734] Step 2:

[0735] The server receives input data sent from the terminal and passes it on to a deep learning model for processing. The model extracts patterns of user interests and characteristics from the newly received data while comparing it with past data. In this process, machine learning algorithms are used to analyze the data and identify the most suitable activity locations and events for the user as output.

[0736] Step 3:

[0737] Based on the analysis results obtained from the server, the terminal displays suggested activity locations and experience options on the user interface. From this output, the terminal presents detailed information about a specific event to the user and organizes it for easy comparison. The displayed options include location information and available dates and times.

[0738] Step 4:

[0739] The user selects the experience best suited to themselves or their child from the options displayed on the device. Once the user makes a selection, the device immediately connects with the server and begins processing the reservation and electronic payment for the selected experience. After the process is complete, the device displays a confirmation notification to the user.

[0740] Step 5:

[0741] The terminal uses the store's environmental sensors to acquire real-time environmental information. Based on this, the terminal guides the user to new events and discount information tailored to their needs. The user can respond by choosing additional experiences. The guidance is delivered via smartphone or smart glasses.

[0742] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0743] This invention is a system that utilizes user input information and real-time sentiment data to recommend the optimal location personalized to the user, and by combining it with a sentiment engine, it improves the accuracy and user experience.

[0744] System Configuration

[0745] This system consists of a terminal, a server, and an emotion engine. The terminal is responsible for receiving information input from the user and collecting real-time emotion data, which is then transmitted to the server. The server utilizes deep learning and the emotion engine to analyze the user's attributes, characteristics, and emotional state, and based on this, selects potential locations.

[0746] Collection of user information

[0747] The device receives basic information from the user, such as the child's age, interests, and personality, through its user interface. In addition, it uses a camera and microphone to collect the user's facial expressions and voice data in real time. This emotional data is analyzed by an emotion engine to determine the user's current emotional state.

[0748] Data analysis and suggestions for places to belong

[0749] The server uses user information received from the terminal to input into a deep learning model. This model integrates the user's past data and current sentiment data to generate a list of suitable locations for each user.

[0750] The emotional state analysis provided in real time by the emotion engine is used to prioritize suggested locations. For example, if the system determines that the user is stressed, relaxing places and activities will be suggested at the top of the list.

[0751] Information presentation and selection

[0752] The device displays location options prioritized by the server to the user. By comparing this presented information, the user can make the most suitable choice. For example, the server might suggest "a drawing event at a nearby museum" and "a nature observation workshop at a park," and based on the user's current emotional state, display the drawing event higher.

[0753] Reservation and payment

[0754] The terminal proceeds with the reservation and payment process for the location selected by the user. Since all related procedures are integrated and executed at this stage, users can easily prepare to participate in the event.

[0755] According to embodiments of the present invention, users can easily select the optimal environment according to their emotional state, thereby providing a highly satisfying experience.

[0756] The following describes the processing flow.

[0757] Step 1:

[0758] The device displays a screen where the user can input basic information such as the child's age, interests, and personality. At this time, it prepares to collect the user's facial expressions and voice in real time using the camera and microphone.

[0759] Step 2:

[0760] The user follows the instructions on the device, enters the necessary information about the child into a form, and then authorizes the collection of facial expression and voice data.

[0761] Step 3:

[0762] The device transmits facial expression and voice data collected along with user input information to the server using a secure communication protocol.

[0763] Step 4:

[0764] The server analyzes the received data. Here, it uses a deep learning model to analyze the user's interests and characteristics, and an emotion engine to determine their emotional state in real time.

[0765] Step 5:

[0766] The server generates and prioritizes suitable location options for the user based on the analysis results. For example, if relaxation is deemed necessary, a quiet environment will be prioritized.

[0767] Step 6:

[0768] The server sends the generated options and their priority order to the terminal.

[0769] Step 7:

[0770] The terminal displays a list of locations received from the server on the user interface and organizes the information to make it easier for the user to select a location.

[0771] Step 8:

[0772] The user selects the appropriate option from the presented choices and confirms the reservation by following the instructions on the device.

[0773] Step 9:

[0774] The terminal initiates the reservation and payment process for the selected location. This allows the user to easily complete their participation reservation.

[0775] Step 10:

[0776] The terminal displays a reservation and payment confirmation notification to the user, informing them that the process is complete.

[0777] (Example 2)

[0778] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0779] In today's busy lifestyle, people lack the means to find the optimal place that suits their emotions and interests. This leads to users experiencing stress and being forced to make unsatisfactory choices. There is a need to provide personalized recommendations in real time, taking into account the user's emotional state.

[0780] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0781] In this invention, the server includes means for acquiring user information, means for analyzing user characteristics using deep learning, and means for analyzing emotional data in real time. This makes it possible to suggest the optimal location based on the user's emotional state and characteristics.

[0782] "Means for obtaining user information" refers to devices or functions for collecting attribute information from users, such as age, interests, and personality.

[0783] "Methods for analyzing user characteristics using deep learning" refers to a process of interpreting and evaluating user characteristics using AI technology based on acquired user information.

[0784] "Methods for analyzing emotional data in real time" refer to technologies that instantly process emotion-related data obtained from a user's facial expressions and voice to determine their current emotional state.

[0785] "Means of recommending optimal locations" refers to a system or function that, based on analysis results, presents the most suitable locations and activities for individual users.

[0786] "Means of making reservations and payments" refers to means of automating and executing reservation procedures and related payment processes for selected locations and activities.

[0787] This invention is implemented as a personalized recommendation system. The system consists of a terminal, a server, and an emotion analysis engine. The terminal is a device for user information input and real-time collection of emotion data, and is responsible for transmitting the data obtained from the user to the server.

[0788] The device provides the user with an input interface and receives personal and attribute information, such as age, interests, and personality. The device also includes a camera and microphone, which capture the user's real-time facial expressions and voice. This data is analyzed by an emotion analysis engine to determine the user's emotional state.

[0789] The server processes information transmitted from the terminal using deep learning technology and generative AI models. The acquired data is input into the model, and the user's characteristics and emotional state are integrated and analyzed. Based on these analysis results, the server suggests optimal locations and activities for the user. In this process, the results of the emotion analysis engine contribute to the priority of the suggestions.

[0790] Users can review and select location suggestions displayed on the device. After selection, the device automates the booking and payment process, providing users with a quick and smooth experience.

[0791] As a concrete example, the system suggests "relaxing places" based on the user's current emotional state. If the user is feeling stressed, the server will recommend things like "nearby nature parks" or "relaxation classes."

[0792] An example of a prompt used in a generative AI model might be: "The user is currently feeling stressed, so please suggest a place where they can relax. Specifically, please tell me what kind of nearby events or facilities would be suitable."

[0793] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0794] Step 1:

[0795] The terminal obtains basic information from the user through its user interface. This basic information includes age, interests, and personality. The entered information is converted into a digital format and prepared for subsequent data processing.

[0796] Step 2:

[0797] The device collects the user's facial expressions and voice in real time using its camera and microphone. The acquired emotional data is sent to an emotion analysis engine after signal processing. In this process, the data is pre-processed to quantify the user's emotional state.

[0798] Step 3:

[0799] The emotion analysis engine receives and analyzes emotion-related data transmitted from the device. Specifically, it extracts features from the data and identifies emotions such as joy and sadness. The results of this analysis are output as a numerical evaluation that identifies the user's emotional state.

[0800] Step 4:

[0801] The server receives basic user information from the terminal and the results of the sentiment analysis engine, and then integrates and processes them. The received data becomes input and is fed into a deep learning model. This model analyzes the user's attributes and emotional state in combination to generate a list of suitable locations for the user.

[0802] Step 5:

[0803] The server determines priorities based on the generated list of potential locations. Numerical evaluations of sentiment data are reflected in this prioritization. The prioritized list is sent to the terminal as output.

[0804] Step 6:

[0805] The device displays a list of potential locations received from the server to the user. The user can then choose the option that best suits their current mood and preferences. At this stage, the user experience is customized based on the available choices.

[0806] Step 7:

[0807] After the user makes a selection, the terminal begins the reservation and payment process based on that selection. The reservation is completed by accessing the reservation system and processing the generated information as input. Payment processing is carried out similarly, and a confirmation is output indicating that all necessary procedures have been completed.

[0808] (Application Example 2)

[0809] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0810] In recent years, consumers have been seeking personalized shopping experiences not only in physical stores but also in virtual environments. However, traditional online shopping systems and virtual stores have been insufficient in suggesting products and experiences that take into account the user's emotions and real-time state. As a result, there is a challenge in that users cannot smoothly select appropriate products and experiences that match their emotions at any given time.

[0811] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0812] In this invention, the server includes a device means for collecting user data, a device means for analyzing the user's interests and characteristics using machine learning based on the user data, and a device means for suggesting facilities suitable for the user based on the analyzed results. This makes it possible to suggest personalized products and experiences based on real-time emotional data.

[0813] A "device for collecting user data" is a device that acquires basic user information, facial expressions, voice, emotional data, etc., and transmits it to a server for analysis.

[0814] A "device that analyzes interests and characteristics using machine learning" is a system that uses acquired user data and data analysis techniques to understand users' areas of interest and individual characteristics.

[0815] A "device that suggests facilities suitable for users" is a system that suggests the most suitable products and experiences for users based on their analyzed interests, characteristics, and emotional state.

[0816] A "device that acquires emotional data in real time and recognizes emotional states" is a technology that acquires the user's facial expressions and voice through a camera and microphone, and quickly determines their emotional state at that moment.

[0817] A "device that displays products or experiences in a virtual environment according to the user's emotional state" is a system that visually presents the most suitable products or experiences based on the user's real-time emotional state within a virtual reality or augmented reality environment.

[0818] A "generative AI model" is an artificial intelligence algorithm used to generate and suggest personalized products and experiences based on user data and emotional states.

[0819] A "prompt sentence" is an instruction given to a generative AI model to obtain a specific output. Based on the content of the sentence, the AI ​​suggests appropriate products or experiences.

[0820] The system for realizing this invention enables the collection of information from users and the analysis of emotional data in real time. The system includes a terminal for collecting data, a server for analysis and making suggestions, and an AI model.

[0821] The devices used are such as smart glasses or smartphones. These devices are equipped with cameras and microphones to capture the user's facial expressions and voice, and collect emotional data in real time. The collected data is sent to a server.

[0822] The server uses machine learning algorithms and an emotion analysis engine to analyze the user's characteristics and current emotional state. This makes it possible to suggest facilities and experiences based on the user's interests and emotions. The emotion analysis engine uses software such as the Emotion Analysis SDK to quickly determine the user's emotional state.

[0823] The generative AI model runs on a server and generates personalized products and experiences tailored to the user's emotional state. For example, if a user is stressed, the AI ​​model can suggest relaxing candles or music. If they are excited, it can suggest new games or exciting events.

[0824] The terminal displays suggestions from the server to the user. The user can browse the presented options and select experiences or products that interest them. Depending on the user's selection, booking and payment procedures may also be carried out through the terminal.

[0825] As a concrete example, when a user visits a virtual store, the emotion analysis engine analyzes the user's facial expressions, and if it determines that the user's current emotional state is "excited," the generative AI model could suggest a "new adventure game."

[0826] To generate suggestions for a generative AI model, use the following prompt statements:

[0827] "Please suggest new or unique products that would be suitable for someone currently experiencing an excited emotional state."

[0828] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0829] Step 1:

[0830] The device collects the user's facial expressions and voice in real time. This involves capturing facial data using a camera and recording voice data using a microphone. This raw data is pre-processed for emotion analysis before being sent to the server. The input data consists of the user's facial expressions and voice data, while the output is pre-processed data.

[0831] Step 2:

[0832] The server analyzes the received data using the Emotion Analysis SDK. First, it takes in facial expression data and voice data as input, which the emotion analysis engine processes. As a result of the analysis, it outputs the user's current emotional state. This emotional state is used in the subsequent suggestion process.

[0833] Step 3:

[0834] The server inputs the analyzed emotional state and the user's past interests into a machine learning model. This model performs analysis to enumerate the products and experiences best suited to the user. The output is a personalized list of suggestions based on the user's emotional state.

[0835] Step 4:

[0836] The terminal displays a list of suggestions received from the server to the user. The user selects products or experiences that interest them from this list. A user interface is used here to facilitate visual selection. The displayed suggestions are optimized by a generative AI model, and the user makes their selection based on these suggestions.

[0837] Step 5:

[0838] Once the user makes a selection, the terminal sends that information back to the server to initiate the booking and payment process. The input here is the user's selection information, and the output is the booking confirmation and payment completion for the selected experience or product. The terminal processes these steps in the background and displays the completion status to the user.

[0839] This series of processing steps allows users to receive product and experience suggestions that match their current emotions, enabling them to smoothly select and purchase items.

[0840] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0841] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0842] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0843] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0844] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0845] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0846] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0847] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0848] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0849] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0850] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0851] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0852] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0853] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0854] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0855] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0856] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0857] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0858] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0859] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0860] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0861] The following is further disclosed regarding the embodiments described above.

[0862] (Claim 1)

[0863] The primary means of collecting user information,

[0864] A second method involves using deep learning to analyze the user's interests and characteristics based on the aforementioned user information.

[0865] Based on the analysis results of the second method described above, a third method is proposed to present a suitable place for the user,

[0866] A fourth method of booking and paying for a location from the presented options.

[0867] A system that includes this.

[0868] (Claim 2)

[0869] The system according to claim 1, further comprising means for acquiring the user's facial expressions and voice in real time and recognizing their emotions.

[0870] (Claim 3)

[0871] The system according to claim 1, further comprising means for displaying information for comparing the presented options for a place to live.

[0872] "Example 1"

[0873] (Claim 1)

[0874] A device for collecting user attribute information,

[0875] Based on the aforementioned user attribute information, the apparatus and means analyze the user's interests and characteristics using an artificial intelligence model.

[0876] Based on the aforementioned analysis results, a device and means for presenting a suitable activity environment,

[0877] A system including a device for making reservations and payments from a selection of activity environments.

[0878] (Claim 2)

[0879] A system according to claim 1, which continuously acquires the external state of a user and identifies their emotions.

[0880] (Claim 3)

[0881] A device for displaying information for evaluating the presented options for the activity environment, according to claim 1.

[0882] "Application Example 1"

[0883] (Claim 1)

[0884] Information acquisition means for collecting user information,

[0885] Based on the user information, an analytical means is provided to analyze interests and characteristics using machine learning.

[0886] Based on the analysis results of the aforementioned analysis means, a suggestion means presents an activity location suitable for the user,

[0887] A processing method for making reservations and electronic payments from the proposed activity location options,

[0888] A system that includes this.

[0889] (Claim 2)

[0890] The system according to claim 1, further comprising recognition means for acquiring the user's facial expressions and voice in real time and recognizing their emotions.

[0891] (Claim 3)

[0892] The system according to claim 1, further comprising a display means for displaying information for comparing the presented options for activity locations.

[0893] (Claim 4)

[0894] The system according to claim 1, further comprising a means of providing guidance to users based on the in-store environment to provide a personalized experience.

[0895] "Example 2 of combining an emotion engine"

[0896] (Claim 1)

[0897] Means of obtaining user information,

[0898] Based on the acquired information, a method for analyzing user characteristics using deep learning,

[0899] A means of analyzing emotional data acquired in real time,

[0900] Based on the analysis results, a means to recommend the optimal location for the user,

[0901] A method for making a reservation and payment from the presented options.

[0902] A system that includes this.

[0903] (Claim 2)

[0904] The system according to claim 1, further comprising means for acquiring the user's facial expressions and voice in real time and determining their emotions.

[0905] (Claim 3)

[0906] The system according to claim 1, further comprising means for displaying information for evaluating the options of the displayed location.

[0907] "Application example 2 when combining with an emotional engine"

[0908] (Claim 1)

[0909] A device for collecting user data,

[0910] Based on the aforementioned user data, a device means for analyzing interests and characteristics using machine learning,

[0911] Based on the results of the analysis, the apparatus and means propose a facility suitable for the user,

[0912] A device for making reservations and payments from the proposed facility options,

[0913] A device means for acquiring user emotional data in real time and recognizing their emotional state,

[0914] A device means for displaying products or experiences in a virtual environment according to the user's emotional state,

[0915] A system that includes this.

[0916] (Claim 2)

[0917] A device for displaying information for comparing proposed experience options, according to claim 1.

[0918] (Claim 3)

[0919] A system according to claim 1, comprising a generative AI model for generating recommendations based on the emotional state of a user, and a device using a prompt statement associated therewith. [Explanation of symbols]

[0920] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. The primary means of collecting user information, A second method involves using deep learning to analyze the user's interests and characteristics based on the aforementioned user information. Based on the analysis results of the second method described above, a third method is proposed to present a suitable place for the user, A fourth method of making a reservation and payment from the presented options for a place to stay, A system that includes this.

2. The system according to claim 1, further comprising means for acquiring the user's facial expressions and voice in real time and recognizing their emotions.

3. The system according to claim 1, further comprising means for displaying information for comparing the presented options for a place to live.

Images