system

Abstract

We provide the system. [Solution] A means of receiving visitor identification information, A means for accessing a visitor's behavior history database based on the received identification information and extracting the relevant history data, A means of analyzing extracted historical data to generate personalized information suggestions for visitors, A means for outputting the generated proposal to a display device and presenting the information to visitors visually and audibly, A means of receiving visitor input and recording the feedback received, A system that includes means for analyzing recorded feedback and using a generative AI model to learn how to optimize future suggestions.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In conventional guidance systems, it was only possible to provide uniform information to visitors, and it was difficult to provide experiences according to individual interests and needs. For this reason, customer satisfaction did not improve, and the convenience of the facility could not be maximally exerted. Also, there was a problem that it was difficult to effectively utilize the customer's action history to make real-time proposals.

Means for Solving the Problems

[0005] This invention provides a means for receiving visitor identification information and accessing a database that manages behavioral history data based on that information. By analyzing the history data, it generates personalized information suggestions for visitors and outputs them to a display device, enabling guidance tailored to individual needs. Furthermore, it provides a system that includes means for recording and analyzing feedback from visitors and learning to optimize suggestions for future visits. In addition, by providing suggestions in real time while considering the visitor's location and schedule information, it realizes the provision of a more effective customer experience.

[0006] "Visitor identification information" refers to a unique number or ID for each visitor, and is data used to identify individuals.

[0007] A "behavioral history database" is a collection of data used to store and manage information such as visitors' past behavior, purchase history, and event participation.

[0008] "Personalized information suggestions" refer to the provision of customized event and product information based on a visitor's past behavior history and individual preferences.

[0009] A "display device" refers to a screen or digital signage equipment used to present information to visitors visually.

[0010] "Feedback" refers to the reactions and opinions that visitors give to the information and suggestions provided, and is collected as data to improve future suggestions.

[0011] "Real-time suggestions" refer to informational guidance that is generated instantly based on the visitor's current situation and upcoming event schedule.

[0012] "Analysis" is a method of examining collected data to understand visitors' preferences and behavioral patterns.

[0013] "Learning" is the process of continuously improving the system by incorporating visitor feedback. [Brief explanation of the drawing]

[0014] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14]It is a sequence diagram showing the processing flow of a data processing system in Application Example 2 when a sentiment engine is combined.

Embodiments for Carrying Out the Invention

[0015] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0018] In the following embodiments, a numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0019] In the following embodiments, a numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0020] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0022] [First Embodiment]

[0023] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0024] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0027] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0030] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0034] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0035] This invention is a system for providing personalized information to visitors. This system acquires visitor identification information, extracts relevant information from a behavioral history database based on that information, and analyzes it. Based on the analysis results, it generates optimized suggestions for the visitor and provides the information visually through a display device. The following describes each of its operations in detail.

[0036] Server operation

[0037] The server receives visitor identification information transmitted from the terminal. Based on the received identification information, the server queries a behavioral history database and extracts relevant historical data. This includes past purchase history and event participation records. Next, an AI algorithm is applied to analyze the visitor's preferences. Based on the analysis results, the server generates personalized suggestions for each visitor. These suggestions include real-time information and product recommendations, and are sent to the terminal.

[0038] Terminal operation

[0039] The terminal receives suggestion data sent from the server and outputs it to the display device. The display device shows an avatar, visually presenting the suggestion to the visitor. The visitor can interact with the terminal through voice or touch controls. The terminal sends the input information back to the server as feedback.

[0040] User actions

[0041] Users review the information displayed on the display screen. This information may include details about nearby events, special offers, and recommended products. Furthermore, users can interact with avatars to obtain more detailed information and learn about products that interest them. This allows users to quickly obtain the information they need and enhance their experience within the facility.

[0042] Specific example

[0043] For example, suppose a user visits a department store. A terminal detects the user and sends the data to a server. The server analyzes the user's past purchase history and recognizes that the user is interested in cosmetics. As a result, it suggests information such as "A special sale is currently underway on the cosmetics floor." The user accepts this suggestion and can hear the sale details from an avatar. Through this entire process, the user can obtain more personally tailored information and improve their shopping experience.

[0044] Thus, the system of the present invention realizes the provision of individualized experiences tailored to visitors, thereby enhancing the convenience of the facility and customer satisfaction.

[0045] The following describes the processing flow.

[0046] Step 1:

[0047] The server receives visitor identification information transmitted from the terminal.

[0048] Step 2:

[0049] The server accesses the behavioral history database based on the received identification information and extracts the relevant historical data.

[0050] Step 3:

[0051] The server analyzes the extracted historical data using an AI algorithm. This analysis identifies the visitor's preferences and interests.

[0052] Step 4:

[0053] Based on the analysis results, the server generates information suggestions optimized for the visitor. These suggestions include currently available event information and product recommendations.

[0054] Step 5:

[0055] The server sends the generated suggestions to the terminal.

[0056] Step 6:

[0057] The terminal outputs the suggestions received from the server to the display device. An avatar is displayed on the screen, conveying visitor information.

[0058] Step 7:

[0059] The user reviews the displayed information and interacts with the avatar as needed. Interactions are conducted via voice or touch.

[0060] Step 8:

[0061] The terminal receives input from the user and sends that feedback to the server.

[0062] Step 9:

[0063] The server records the feedback it receives and stores it in a database. This information is then used to learn how to improve future suggestions.

[0064] Step 10:

[0065] The server and terminal confirm that the user has left the area and terminate the information provision session.

[0066] (Example 1)

[0067] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0068] The challenge lies in providing personalized information in real time to individual users visiting commercial facilities and other similar establishments, based on their purchase history and interests. Solving this challenge requires technology that can present information tailored to each user's needs in a timely manner.

[0069] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0070] In this invention, the server includes means for receiving visitor identification data and identifying the visitor using a recording medium, means for accessing a recording medium that stores the visitor's behavior history based on the received identification data and obtaining the corresponding history information, and means for analyzing the obtained history information and applying a generation AI algorithm to generate personalized suggestion information for the visitor. This makes it possible to quickly provide information optimized for each user and enrich the visitor experience.

[0071] "Visitor identification data" refers to information used to identify individual users who visit a facility, and includes facial recognition data, QR codes (registered trademark), RFID tags, etc.

[0072] A "recording medium" is a physical or electronic device or system used to record, store, and retrieve digital information.

[0073] "Behavioral history" refers to the accumulation of information about specific actions and activities, such as a user's past visits, purchase history, and event participation history.

[0074] A "generative AI algorithm" is a computational method and procedure for automatically generating new information suggestions by analyzing user behavior and preferences based on data.

[0075] "Personalized suggestion information" refers to content that suggests the most relevant and suitable information and products to a specific user based on their visitor identification data and behavioral history.

[0076] A "display device" refers to a digital screen or monitor used to present visual information.

[0077] "Feedback" refers to information such as opinions, preferences, and usage history obtained from visitors, and is used to improve the quality of proposals.

[0078] "Learning for optimization" is a data analysis and improvement process that uses feedback and AI algorithms to improve the accuracy of suggestions.

[0079] This invention is a system that provides personalized information to visitors. The system aims to improve the user experience by providing information tailored to the individual needs and interests of visitors in real time.

[0080] The server receives visitor identification data transmitted from terminals within the facility. Identification can utilize technologies such as facial recognition, QR codes, and RFID tags. Based on the received identification data, the server searches for visitor history from storage media and retrieves relevant historical information. These storage media include cloud storage and database management systems.

[0081] The server analyzes historical information using a generation AI algorithm and generates personalized suggestion information based on the visitor's preferences. The generated suggestions are immediately sent to the terminal, which visually displays this information through a display device. Touch panels or monitors are used as the display device.

[0082] Users can review the information displayed on the device and, if necessary, request additional information or respond using voice or touch controls. The device sends this user feedback to the server, which is then used to optimize future recommendations. A concrete example of this might be a prompt message such as, "Please suggest products based on the user's past purchase history."

[0083] In this way, the system can provide information tailored to each visitor, enabling efficient and effective service delivery.

[0084] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0085] Step 1:

[0086] The terminal acquires visitor identification data. This data is obtained through facial recognition via sensors and cameras, QR code scanning, or RFID reading. This data serves as input. The terminal then sends this data to the server to initiate the identification process.

[0087] Step 2:

[0088] The server queries the activity history stored on the recording medium based on the received identification data. The server uses a database management system to search for and retrieve past history information that matches the identification data. This becomes the output. This history information includes the visitor's past purchase history and event participation history.

[0089] Step 3:

[0090] The server uses the acquired historical information to perform analysis with a generating AI model. This analysis identifies the visitor's preferences and interests, and generates personalized information suggestions. The input is historical information, and the output is suggestion information. The suggestion information includes recommended products and event announcements.

[0091] Step 4:

[0092] The server sends the generated suggestion information to the terminal. The terminal visually displays the received suggestions on a display device. To present the suggestions in a way that is easy for the user to see and understand, the display device uses a touch panel or monitor.

[0093] Step 5:

[0094] Users can review suggestions displayed on the display device and provide feedback via voice or touch. For example, they can request further details or react to the suggestions. This feedback constitutes input.

[0095] Step 6:

[0096] The device sends user feedback to the server. The server records this feedback and uses it in an optimization process to improve the accuracy of future suggestions using a generative AI model. The output is training data that will be used for future suggestions.

[0097] (Application Example 1)

[0098] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0099] To improve the visitor's purchasing experience, it is crucial to provide information in real time based on individual preferences and behavioral history. However, conventional systems have struggled to effectively utilize visitor feedback and optimize recommendations. Furthermore, there has been a lack of methods to present information to visitors using both visual and auditory means, providing an interactive user experience.

[0100] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0101] In this invention, the server includes means for receiving visitor identification information, means for referring to a behavioral history database based on the received identification information and extracting relevant history data, means for analyzing the extracted history data and generating personalized information suggestions, means for outputting the generated suggestions visually and audibly to present the information, and means for recording visitor feedback and using a generating AI model to learn and optimize future suggestions. This enables the provision of dynamic and interactive information to visitors, thereby improving the purchasing experience.

[0102] "Visitor identification information" refers to data used by the system to identify each individual, and includes information such as smartphone IDs and facial recognition data.

[0103] A "behavioral history database" is a database that stores data related to visitors' behavior, such as past purchase history and event participation records.

[0104] "Personalized information suggestions" are suggestions generated to recommend specific information or products individually based on the visitor's preferences and needs.

[0105] "Visual and audible output" refers to the use of displays and speakers to communicate information visually and audibly.

[0106] A "generative AI model" refers to an artificial intelligence technology used to analyze data and generate personalized information.

[0107] "Dynamic and interactive information delivery" refers to a method of providing information that is instantly updated and adjusted in response to user feedback and actions, and delivered through dialogue.

[0108] The system implementing this invention is based on a network system including a server and terminals to receive visitor identification information and generate personalized information suggestions. The server receives identification information transmitted from a smartphone or in-store terminal when a visitor enters the facility and uses this information to access a behavioral history database. This database stores the visitor's past purchase history and event participation records, and relevant data can be extracted.

[0109] The server uses AI algorithms to analyze extracted historical data and generate personalized information suggestions based on the visitor's preferences and interests. These suggestions include visual and auditory presentations, delivered through displays and speakers. The generated suggestions are dynamic and interactive, forming the basis for optimizing subsequent information delivery based on visitor feedback.

[0110] The terminal serves to facilitate interaction with visitors, allowing them to provide feedback through voice or touch operations. This feedback is then sent back to the server and analyzed using a generative AI model to continuously optimize the suggestions.

[0111] As a concrete example, in one bookstore, if analysis indicates that a customer is interested in mystery novels, information about ongoing "mystery novel fairs" and new releases is visually displayed on the screen, along with audio recommendations. Based on this information, the user can also interact with an avatar to learn more details.

[0112] An example of a prompt message would be: "Use the user's past purchase history data to generate personalized information suggestions. In particular, create suggestions that are appealing to the user, including promotional information for new products."

[0113] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0114] Step 1:

[0115] The server receives visitor identification information from a smartphone or in-store terminal. This identification information is used to identify the visitor. The data entered includes an identification ID and the visitor's location information. Based on this, the server verifies the visitor's uniqueness and references it to the database.

[0116] Step 2:

[0117] The server references the behavioral history database based on the received identification information and extracts the relevant historical data. The input here is the visitor's identification information, and the output includes data such as the visitor's past purchase history and event participation records. This data is used as foundational data in subsequent analyses.

[0118] Step 3:

[0119] The server analyzes the extracted historical data using an AI algorithm. This algorithm uses a generative AI model to analyze the visitor's preferences and interests. The input is behavioral history data, and the output generates suggested information related to the visitor's areas of interest. This analysis is performed based on prompt messages to determine the content of the suggestions.

[0120] Step 4:

[0121] The server sends the generated personalized information suggestions to the terminal. The terminal provides the suggested information to the visitor visually and audibly. The input here is the suggested information, and the output is the display on the screen and audio guidance via the speaker. The visitor reviews the presented information and chooses an action based on its content.

[0122] Step 5:

[0123] Users provide feedback to the device using voice or touch controls. This feedback is sent to the server as data to help the generative AI model improve its suggestions. The input is the user's actions and selections, and the output is improvement suggestions based on that information.

[0124] Step 6:

[0125] The server receives feedback and continuously optimizes its suggestions. This learning process using the data improves the accuracy of subsequent suggestions. The input here is feedback data, and the output is optimized suggestion information. This enables the continuous provision of high-quality information to visitors.

[0126] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0127] This invention is a system for providing personalized information to visitors, and by combining it with an emotion engine, it recognizes the visitor's emotional state and provides information and experiences appropriate to that emotional state. This system acquires the visitor's identification information, extracts relevant information from a behavioral history database based on that information, and analyzes it. The system comprehensively judges the analysis results and the emotions recognized by the emotion engine to generate the most appropriate information suggestions for the visitor. These suggestions are then visually presented by an avatar through a display device.

[0128] Server operation

[0129] The server accesses a behavioral history database based on visitor identification information received from the terminal and extracts relevant historical data. An AI algorithm is applied to analyze the historical data to identify the visitor's preferences and interests. Furthermore, the visitor's emotional state, as determined by the emotion engine, is also incorporated into the analysis results. For example, the suggestions are adjusted to differ depending on whether the visitor is relaxed or stressed.

[0130] Terminal operation

[0131] The terminal outputs the suggested data received from the server to the display device. The avatar visually provides information, adjusting its tone and facial expressions according to the visitor's emotional state. In addition, the terminal accepts input from the visitor, enabling interaction through voice and touch controls. The received input is sent directly to the server as feedback.

[0132] User actions

[0133] Users can view the information displayed on the screen and obtain more detailed information through interaction with the avatar. For example, if the user is calm, the avatar will suggest relaxing restaurant information in a gentle tone. On the other hand, if the user is stressed, the avatar will provide information on refreshing activities in an encouraging tone.

[0134] Specific example

[0135] As a concrete example of use, suppose a user visits an information center at a travel destination. The device detects the user's face, and the emotion engine recognizes the emotion of "surprise." Based on past behavior history and this emotion information, the server decides to recommend "the spectacular view from the observation deck" to the user. The avatar speaks in an excited tone to keep the user interested. Through this entire process, the user can have an emotionally satisfying experience.

[0136] Thus, the present invention aims to realize a personalized experience that takes into account the emotional state of visitors, thereby improving facility services and dramatically increasing customer satisfaction.

[0137] The following describes the processing flow.

[0138] Step 1:

[0139] The terminal receives visitor identification information using sensors and camera technology.

[0140] Step 2:

[0141] The device transmits the visitor's facial expressions and voice to an emotion engine, recognizing emotions in real time.

[0142] Step 3:

[0143] The server accesses the behavioral history database based on the identification information sent from the terminal and extracts the corresponding history data.

[0144] Step 4:

[0145] The server analyzes the extracted historical data using an AI algorithm to identify the visitor's preferences.

[0146] Step 5:

[0147] The server integrates the analysis results with emotion data from the emotion engine to generate personalized information suggestions for visitors. These suggestions include information tailored to their emotional state.

[0148] Step 6:

[0149] The server sends the generated suggestions to the terminal.

[0150] Step 7:

[0151] The terminal outputs suggestions received from the server to a display device, providing visitors with information visually through an avatar. The avatar's tone and facial expressions are adjusted to match the recognized emotions.

[0152] Step 8:

[0153] The user reviews the displayed information and interacts with the avatar as needed. Interactions are conducted via voice or touch, and the input is recorded on the device.

[0154] Step 9:

[0155] The device sends user input and feedback to the server.

[0156] Step 10:

[0157] The server records the feedback it receives and stores it in a database. The server uses this information to learn from future suggestions and make improvements.

[0158] Step 11:

[0159] The server and terminal detect when the user leaves the signage area and terminate the information provision process.

[0160] (Example 2)

[0161] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0162] Providing visitors with personalized information and experiences immediately upon arrival at a facility has been difficult with conventional technology. In particular, there was a lack of systems to understand visitors' emotional states and provide the most appropriate services based on that information. Therefore, it is necessary to increase the satisfaction of each individual visitor and improve the overall service quality of the facility.

[0163] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0164] In this invention, the server includes means for accessing a behavioral history database based on visitor identification information and extracting relevant history information, means for recognizing the visitor's emotional state using an emotion analysis engine, and means for visually presenting personalized information suggestions to the visitor. This enables the provision of information and optimization of the experience according to the individual emotional state of each visitor.

[0165] A "visitor" refers to an individual or group visiting the facility, whose behavior and emotional state are observed by the system.

[0166] "Identification information" refers to data used to identify visitors, and may take the form of facial recognition or device ID.

[0167] A "behavioral history database" is an accessible information resource that stores information about visitors' past visit history and preferences.

[0168] An "emotion analysis engine" is a technology used to analyze the emotional state of visitors, recognizing emotions by analyzing facial expressions, tone of voice, and other factors.

[0169] "Information suggestions" are personalized suggestions of information and experiences generated based on a visitor's identification information and emotional state.

[0170] A "display means" is a device used to visually present information suggestions to visitors, and may consist of a screen or an avatar.

[0171] "Feedback information" refers to input or response information obtained from visitors, and is data used to optimize future proposal generation.

[0172] This invention is a system that provides personalized information to visitors and consists mainly of three elements: a server, a terminal, and a user. The core of this system is information suggestion based on the visitor's identification information and emotions.

[0173] The server receives visitor identification information transmitted from terminals via the facility's network. Based on this identification information, the server accesses a behavioral history database and extracts relevant historical information. This historical information is analyzed using AI algorithms to identify the visitor's preferences and interests. The server also uses an emotion analysis engine to recognize the visitor's emotional state from their facial expressions and voice. The resulting emotional state becomes an important element in generating information suggestions.

[0174] The terminal visually displays information suggestions received from the server. This display uses an avatar, which exhibits a tone and facial expression corresponding to the visitor's emotional state. The terminal can also accept voice input and touch input from the visitor. This allows the user to obtain detailed information through interaction with the avatar.

[0175] As a concrete example, consider a scenario where a user visits a tourist attraction. Suppose the device detects the user's face, and the emotion analysis engine recognizes the emotion of "surprise." Based on the user's behavioral history and emotional information, the server decides to suggest "a spectacular view from the observation deck." The avatar then conveys this information in an excited tone, piquing the user's interest.

[0176] An example of a prompt might be: "Please create a plan for a travel information system to provide personalized suggestions based on the visitor's emotions. Provide appropriate experiences to suggest when the visitor's emotional state is 'surprise'." This helps the generative AI model suggest services tailored to the individual emotions of each visitor.

[0177] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0178] Step 1:

[0179] The terminal acquires visitor identification information using sensors and cameras installed within the facility. The input includes the visitor's facial image and device ID. This information is encrypted and transmitted to the server via a secure protocol. The output is encrypted identification data.

[0180] Step 2:

[0181] The server accesses the behavioral history database based on the identification information received from the terminal. The visitor's identification information is provided as input. A database query is performed to extract the relevant history information. This outputs data about the visitor's past behavior and preferences.

[0182] Step 3:

[0183] The server uses an emotion analysis engine to analyze the visitor's emotional state. The input includes the visitor's facial expressions and voice tone. Natural language processing and speech recognition technologies are used to recognize emotions such as "surprise" and "stress." The output is the recognized emotion information.

[0184] Step 4:

[0185] The server integrates behavioral history and emotional information to generate personalized information suggestions. The input consists of extracted historical data and emotional information. A generative AI model is applied to calculate the most suitable information and experience for the visitor. The output is personalized information suggestion data.

[0186] Step 5:

[0187] The terminal provides visitors with information suggestions received from the server via a display mechanism. The input is generated information suggestion data. An avatar is introduced, presenting information with changes in tone and facial expression that correspond to the visitor's emotional state. The output is the visually presented information.

[0188] Step 6:

[0189] The user reviews the information presented by the avatar and requests further details as needed. Input is the information displayed on the device. The user interacts using voice input and touch gestures. Output is the detailed information obtained by the user.

[0190] Step 7:

[0191] The terminal sends user input to the server as feedback. This input consists of user operation data. The server analyzes and learns from this feedback to improve the accuracy of information suggestions. The output is the improved suggestion generation algorithm.

[0192] (Application Example 2)

[0193] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0194] In modern commercial facilities, providing more personalized services is required to increase visitor satisfaction. However, it is difficult to provide information that is appropriate to the different emotional states of each visitor. Furthermore, conventional technology is insufficient in suggesting products and services that match the visitor's emotions at that moment, making it difficult to maximize visitors' purchasing intent.

[0195] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0196] In this invention, the server includes means for receiving visitor identification information, means for accessing a set of visitor behavior history data based on the received identification information and extracting the relevant history information, and means for recognizing the visitor's emotional state and adjusting information suggestions based on that emotion. This makes it possible to suggest personalized products and services that correspond to the visitor's emotional state.

[0197] "Visitor identification information" refers to information used to identify individual people who visit a facility, and is data obtained through facial recognition, ID cards, etc.

[0198] A "behavioral history data set" is a collection of data that records visitors' past activities, preferences, and purchase history.

[0199] "Personalized information suggestions" refer to suggestions that provide information optimized according to the visitor's characteristics, preferences, and emotional state.

[0200] A "display device" is a device used to present information visually, and includes displays and smart glasses.

[0201] "Feedback" refers to data recorded from visitor responses and reactions, which is used to improve the system and optimize information suggestions.

[0202] "Emotional state" refers to the mental state a visitor is experiencing at a given time, and includes a variety of emotions such as relaxation, tension, and excitement.

[0203] A "visual display device" is a device used to provide visual information to users, and is particularly used to display emotion-based information and advertisements to visitors.

[0204] To implement this invention, a system using a server, a terminal, and a visual display device is required. The server receives visitor identification information, accesses a set of behavioral history data, and extracts relevant historical information. Furthermore, it analyzes the historical information using an AI algorithm and generates personalized information suggestions for the visitor. It also uses sentiment analysis software to recognize the visitor's emotional state and adjust the information suggestions accordingly. This makes it possible to suggest products and services that are appropriate to the visitor's emotions.

[0205] The terminal outputs information suggestions received from the server to a visual display device and presents them to the visitor. Smart glasses are used as the visual display device, transparently displaying the information. This allows visitors to obtain the necessary information without using their hands.

[0206] Users review the presented information and provide feedback as needed. This feedback is sent directly to the server and recorded in the database. The recorded feedback is used for learning to optimize future information suggestions.

[0207] As a concrete example, when a user enters a store, their emotional state is sensed through smart glasses. If the system determines the user is relaxed, it will gently guide them through information about new products. If the system determines the user is tense, it can provide promotional information about products with relaxing effects.

[0208] Example prompt: "Identify the product categories that customers are most interested in when they enter the store, and suggest appropriate product information for that situation."

[0209] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0210] Step 1:

[0211] The server receives the visitor's identification information. This identification information, such as the visitor's facial recognition data or ID card information, is passed to the server as input data. Based on the received information, the server accesses the corresponding visitor's activity history data set.

[0212] Step 2:

[0213] The server searches the behavioral history data set based on the received identification information and extracts the relevant historical information. Database queries are used for the search, and in this process, the historical data is used as input, and data calculations such as weighting and filtering are performed. Finally, historical information suitable for analysis is output.

[0214] Step 3:

[0215] The server analyzes the extracted historical information using an AI algorithm to generate personalized information suggestions for visitors. The AI ​​algorithm narrows down the candidate suggestions based on behavioral patterns and past preferences. The resulting information suggestions become the server's output.

[0216] Step 4:

[0217] The server uses sentiment analysis software to recognize the visitor's emotional state. Current facial expression data of the visitor is used as input, and the sentiment engine performs real-time analysis. The results, based on the emotional state, are used to adjust the information suggestions.

[0218] Step 5:

[0219] The terminal outputs information suggestions received from the server to smart glasses, which are a visual display device. The input here is information suggestion data, and the output is information that is visually presented to the visitor.

[0220] Step 6:

[0221] The user reviews the presented information on a visual display device and provides feedback. User feedback is input via voice or gestures, and the device sends this back to the server as input data.

[0222] Step 7:

[0223] The server records the received feedback in a historical database, accumulating data to optimize future information suggestions. The input is feedback data, and the output is an updated database.

[0224] These processing steps enable the provision of optimal information to visitors, tailored to their emotional state.

[0225] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0226] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0227] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0228] [Second Embodiment]

[0229] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0230] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0231] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0232] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0233] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0234] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0235] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0236] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0237] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0238] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0239] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0240] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0241] This invention is a system for providing personalized information to visitors. This system acquires visitor identification information, extracts relevant information from a behavioral history database based on that information, and analyzes it. Based on the analysis results, it generates optimized suggestions for the visitor and provides the information visually through a display device. The following describes each of its operations in detail.

[0242] Server operation

[0243] The server receives visitor identification information transmitted from the terminal. Based on the received identification information, the server queries a behavioral history database and extracts relevant historical data. This includes past purchase history and event participation records. Next, an AI algorithm is applied to analyze the visitor's preferences. Based on the analysis results, the server generates personalized suggestions for each visitor. These suggestions include real-time information and product recommendations, and are sent to the terminal.

[0244] Terminal operation

[0245] The terminal receives suggestion data sent from the server and outputs it to the display device. The display device shows an avatar, visually presenting the suggestion to the visitor. The visitor can interact with the terminal through voice or touch controls. The terminal sends the input information back to the server as feedback.

[0246] User actions

[0247] Users review the information displayed on the display screen. This information may include details about nearby events, special offers, and recommended products. Furthermore, users can interact with avatars to obtain more detailed information and learn about products that interest them. This allows users to quickly obtain the information they need and enhance their experience within the facility.

[0248] Specific example

[0249] For example, suppose a user visits a department store. A terminal detects the user and sends the data to a server. The server analyzes the user's past purchase history and recognizes that the user is interested in cosmetics. As a result, it suggests information such as "A special sale is currently underway on the cosmetics floor." The user accepts this suggestion and can hear the sale details from an avatar. Through this entire process, the user can obtain more personally tailored information and improve their shopping experience.

[0250] Thus, the system of the present invention realizes the provision of individualized experiences tailored to visitors, thereby enhancing the convenience of the facility and customer satisfaction.

[0251] The following describes the processing flow.

[0252] Step 1:

[0253] The server receives visitor identification information transmitted from the terminal.

[0254] Step 2:

[0255] The server accesses the behavioral history database based on the received identification information and extracts the relevant historical data.

[0256] Step 3:

[0257] The server analyzes the extracted historical data using an AI algorithm. This analysis identifies the visitor's preferences and interests.

[0258] Step 4:

[0259] Based on the analysis results, the server generates information suggestions optimized for the visitor. These suggestions include currently available event information and product recommendations.

[0260] Step 5:

[0261] The server sends the generated suggestions to the terminal.

[0262] Step 6:

[0263] The terminal outputs the suggestions received from the server to the display device. An avatar is displayed on the screen, conveying visitor information.

[0264] Step 7:

[0265] The user reviews the displayed information and interacts with the avatar as needed. Interactions are conducted via voice or touch.

[0266] Step 8:

[0267] The terminal receives input from the user and sends that feedback to the server.

[0268] Step 9:

[0269] The server records the feedback it receives and stores it in a database. This information is then used to learn how to improve future suggestions.

[0270] Step 10:

[0271] The server and terminal confirm that the user has left the area and terminate the information provision session.

[0272] (Example 1)

[0273] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0274] The challenge lies in providing personalized information in real time to individual users visiting commercial facilities and other similar establishments, based on their purchase history and interests. Solving this challenge requires technology that can present information tailored to each user's needs in a timely manner.

[0275] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0276] In this invention, the server includes means for receiving visitor identification data and identifying the visitor using a recording medium, means for accessing a recording medium that stores the visitor's behavior history based on the received identification data and obtaining the corresponding history information, and means for analyzing the obtained history information and applying a generation AI algorithm to generate personalized suggestion information for the visitor. This makes it possible to quickly provide information optimized for each user and enrich the visitor experience.

[0277] "Visitor identification data" refers to information used to identify individual users who visit a facility, and includes facial recognition data, QR codes, RFID tags, etc.

[0278] A "recording medium" is a physical or electronic device or system used to record, store, and retrieve digital information.

[0279] "Behavioral history" refers to the accumulation of information about specific actions and activities, such as a user's past visits, purchase history, and event participation history.

[0280] A "generative AI algorithm" is a computational method and procedure for automatically generating new information suggestions by analyzing user behavior and preferences based on data.

[0281] "Personalized suggestion information" refers to content that suggests the most relevant and suitable information and products to a specific user based on their visitor identification data and behavioral history.

[0282] A "display device" refers to a digital screen or monitor used to present visual information.

[0283] "Feedback" refers to information such as opinions, preferences, and usage history obtained from visitors, and is used to improve the quality of proposals.

[0284] "Learning for Optimization" is a data analysis and improvement process for improving the accuracy of proposals using feedback and AI algorithms.

[0285] The present invention is a system that provides personalized information to visitors. This system aims to provide information in real time according to the individual needs and interests of visitors and improve the user experience.

[0286] The server receives the identification data of visitors transmitted from terminals within the facility. For identification, face recognition technology, QR codes, RFID tags, etc. can be used. Based on the received identification data, the server searches the recording medium for the behavior history and obtains the relevant history information. The recording medium corresponds to cloud storage or a database management system.

[0287] The server applies a generative AI algorithm to analyze the history information and generates personalized proposal information based on the preferences of the visitors. The generated proposals are immediately transmitted to the terminal, and the terminal visually shows this information through a display device. A touch panel or a monitor is used as the display device.

[0288] The user can check the information displayed on the display device and, if necessary, request additional information or respond by voice or touch operations. The terminal transmits this feedback from the users to the server and utilizes it for the optimization of future proposals. As a specific example, for instance, a prompt sentence such as "Please propose recommended products based on the user's past purchase history" can be considered.

[0289] In this way, this system can make information proposals optimized for each individual visitor and provide services efficiently and effectively.

[0290] The flow of the specific process in Example 1 will be described using FIG. 11.

[0291] Step 1:

[0292] The terminal acquires visitor identification data. This data is obtained through facial recognition via sensors and cameras, QR code scanning, or RFID reading. This data serves as input. The terminal then sends this data to the server to initiate the identification process.

[0293] Step 2:

[0294] The server queries the activity history stored on the recording medium based on the received identification data. The server uses a database management system to search for and retrieve past history information that matches the identification data. This becomes the output. This history information includes the visitor's past purchase history and event participation history.

[0295] Step 3:

[0296] The server uses the acquired historical information to perform analysis with a generating AI model. This analysis identifies the visitor's preferences and interests, and generates personalized information suggestions. The input is historical information, and the output is suggestion information. The suggestion information includes recommended products and event announcements.

[0297] Step 4:

[0298] The server sends the generated suggestion information to the terminal. The terminal visually displays the received suggestions on a display device. To present the suggestions in a way that is easy for the user to see and understand, the display device uses a touch panel or monitor.

[0299] Step 5:

[0300] Users can review suggestions displayed on the display device and provide feedback via voice or touch. For example, they can request further details or react to the suggestions. This feedback constitutes input.

[0301] Step 6:

[0302] The terminal sends the feedback from the user to the server. The server records this feedback and uses it in an optimization process to improve the future proposal accuracy using the generative AI model. The output is the learning data to be utilized in the next proposal.

[0303] (Application Example 1)

[0304] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0305] In order to improve the purchasing experience of visitors, it is important to provide information in real time based on individual preferences and behavioral histories. However, in conventional systems, it has been difficult to effectively utilize the feedback from visitors and optimize the proposed content. Also, there has been a lack of methods to present information to visitors both visually and audibly and provide an interactive user experience.

[0306] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following respective means.

[0307] In this invention, the server includes means for receiving the identification information of the visitor, means for referring to the behavioral history database based on the received identification information and extracting the corresponding history data, means for analyzing the extracted history data and generating a personalized information proposal, means for visually and audibly outputting the generated proposal and presenting the information, and means for recording the feedback from the visitor and performing learning to optimize future proposals using the generative AI model. Thereby, it becomes possible to provide dynamic and interactive information to the visitor and improve the purchasing experience.

[0308] The "identification information of the visitor" is data used by the system to identify each individual, and includes information such as the ID of a smartphone and face authentication data.

[0309] A "behavioral history database" is a database that stores data related to visitors' behavior, such as past purchase history and event participation records.

[0310] "Personalized information suggestions" are suggestions generated to recommend specific information or products individually based on the visitor's preferences and needs.

[0311] "Visual and audible output" refers to the use of displays and speakers to communicate information visually and audibly.

[0312] A "generative AI model" refers to an artificial intelligence technology used to analyze data and generate personalized information.

[0313] "Dynamic and interactive information delivery" refers to a method of providing information that is instantly updated and adjusted in response to user feedback and actions, and delivered through dialogue.

[0314] The system implementing this invention is based on a network system including a server and terminals to receive visitor identification information and generate personalized information suggestions. The server receives identification information transmitted from a smartphone or in-store terminal when a visitor enters the facility and uses this information to access a behavioral history database. This database stores the visitor's past purchase history and event participation records, and relevant data can be extracted.

[0315] The server uses AI algorithms to analyze extracted historical data and generate personalized information suggestions based on the visitor's preferences and interests. These suggestions include visual and auditory presentations, delivered through displays and speakers. The generated suggestions are dynamic and interactive, forming the basis for optimizing subsequent information delivery based on visitor feedback.

[0316] The terminal serves to facilitate interaction with visitors, allowing them to provide feedback through voice or touch operations. This feedback is then sent back to the server and analyzed using a generative AI model to continuously optimize the suggestions.

[0317] As a concrete example, in one bookstore, if analysis indicates that a customer is interested in mystery novels, information about ongoing "mystery novel fairs" and new releases is visually displayed on the screen, along with audio recommendations. Based on this information, the user can also interact with an avatar to learn more details.

[0318] An example of a prompt message would be: "Use the user's past purchase history data to generate personalized information suggestions. In particular, create suggestions that are appealing to the user, including promotional information for new products."

[0319] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0320] Step 1:

[0321] The server receives visitor identification information from a smartphone or in-store terminal. This identification information is used to identify the visitor. The data entered includes an identification ID and the visitor's location information. Based on this, the server verifies the visitor's uniqueness and references it to the database.

[0322] Step 2:

[0323] The server references the behavioral history database based on the received identification information and extracts the relevant historical data. The input here is the visitor's identification information, and the output includes data such as the visitor's past purchase history and event participation records. This data is used as foundational data in subsequent analyses.

[0324] Step 3:

[0325] The server analyzes the extracted historical data using an AI algorithm. This algorithm uses a generative AI model to analyze the visitor's preferences and interests. The input is behavioral history data, and the output generates suggested information related to the visitor's areas of interest. This analysis is performed based on prompt messages to determine the content of the suggestions.

[0326] Step 4:

[0327] The server sends the generated personalized information suggestions to the terminal. The terminal provides the suggested information to the visitor visually and audibly. The input here is the suggested information, and the output is the display on the screen and audio guidance via the speaker. The visitor reviews the presented information and chooses an action based on its content.

[0328] Step 5:

[0329] Users provide feedback to the device using voice or touch controls. This feedback is sent to the server as data to help the generative AI model improve its suggestions. The input is the user's actions and selections, and the output is improvement suggestions based on that information.

[0330] Step 6:

[0331] The server receives feedback and continuously optimizes its suggestions. This learning process using the data improves the accuracy of subsequent suggestions. The input here is feedback data, and the output is optimized suggestion information. This enables the continuous provision of high-quality information to visitors.

[0332] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0333] This invention is a system for providing personalized information to visitors, and by combining it with an emotion engine, it recognizes the visitor's emotional state and provides information and experiences appropriate to that emotional state. This system acquires the visitor's identification information, extracts relevant information from a behavioral history database based on that information, and analyzes it. The system comprehensively judges the analysis results and the emotions recognized by the emotion engine to generate the most appropriate information suggestions for the visitor. These suggestions are then visually presented by an avatar through a display device.

[0334] Server operation

[0335] The server accesses a behavioral history database based on visitor identification information received from the terminal and extracts relevant historical data. An AI algorithm is applied to analyze the historical data to identify the visitor's preferences and interests. Furthermore, the visitor's emotional state, as determined by the emotion engine, is also incorporated into the analysis results. For example, the suggestions are adjusted to differ depending on whether the visitor is relaxed or stressed.

[0336] Terminal operation

[0337] The terminal outputs the suggested data received from the server to the display device. The avatar visually provides information, adjusting its tone and facial expressions according to the visitor's emotional state. In addition, the terminal accepts input from the visitor, enabling interaction through voice and touch controls. The received input is sent directly to the server as feedback.

[0338] User actions

[0339] Users can view the information displayed on the screen and obtain more detailed information through interaction with the avatar. For example, if the user is calm, the avatar will suggest relaxing restaurant information in a gentle tone. On the other hand, if the user is stressed, the avatar will provide information on refreshing activities in an encouraging tone.

[0340] Specific example

[0341] As a concrete example of use, suppose a user visits an information center at a travel destination. The device detects the user's face, and the emotion engine recognizes the emotion of "surprise." Based on past behavior history and this emotion information, the server decides to recommend "the spectacular view from the observation deck" to the user. The avatar speaks in an excited tone to keep the user interested. Through this entire process, the user can have an emotionally satisfying experience.

[0342] Thus, the present invention aims to realize a personalized experience that takes into account the emotional state of visitors, thereby improving facility services and dramatically increasing customer satisfaction.

[0343] The following describes the processing flow.

[0344] Step 1:

[0345] The terminal receives visitor identification information using sensors and camera technology.

[0346] Step 2:

[0347] The device transmits the visitor's facial expressions and voice to an emotion engine, recognizing emotions in real time.

[0348] Step 3:

[0349] The server accesses the behavioral history database based on the identification information sent from the terminal and extracts the corresponding history data.

[0350] Step 4:

[0351] The server analyzes the extracted historical data using an AI algorithm to identify the visitor's preferences.

[0352] Step 5:

[0353] The server integrates the analysis results with emotion data from the emotion engine to generate personalized information suggestions for visitors. These suggestions include information tailored to their emotional state.

[0354] Step 6:

[0355] The server sends the generated suggestions to the terminal.

[0356] Step 7:

[0357] The terminal outputs suggestions received from the server to a display device, providing visitors with information visually through an avatar. The avatar's tone and facial expressions are adjusted to match the recognized emotions.

[0358] Step 8:

[0359] The user reviews the displayed information and interacts with the avatar as needed. Interactions are conducted via voice or touch, and the input is recorded on the device.

[0360] Step 9:

[0361] The device sends user input and feedback to the server.

[0362] Step 10:

[0363] The server records the feedback it receives and stores it in a database. The server uses this information to learn from future suggestions and make improvements.

[0364] Step 11:

[0365] The server and terminal detect when the user leaves the signage area and terminate the information provision process.

[0366] (Example 2)

[0367] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0368] Providing visitors with personalized information and experiences immediately upon arrival at a facility has been difficult with conventional technology. In particular, there was a lack of systems to understand visitors' emotional states and provide the most appropriate services based on that information. Therefore, it is necessary to increase the satisfaction of each individual visitor and improve the overall service quality of the facility.

[0369] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0370] In this invention, the server includes means for accessing a behavioral history database based on visitor identification information and extracting relevant history information, means for recognizing the visitor's emotional state using an emotion analysis engine, and means for visually presenting personalized information suggestions to the visitor. This enables the provision of information and optimization of the experience according to the individual emotional state of each visitor.

[0371] A "visitor" refers to an individual or group visiting the facility, whose behavior and emotional state are observed by the system.

[0372] "Identification information" refers to data used to identify visitors, and may take the form of facial recognition or device ID.

[0373] A "behavioral history database" is an accessible information resource that stores information about visitors' past visit history and preferences.

[0374] An "emotion analysis engine" is a technology used to analyze the emotional state of visitors, recognizing emotions by analyzing facial expressions, tone of voice, and other factors.

[0375] "Information suggestions" are personalized suggestions of information and experiences generated based on a visitor's identification information and emotional state.

[0376] A "display means" is a device used to visually present information suggestions to visitors, and may consist of a screen or an avatar.

[0377] "Feedback information" refers to input or response information obtained from visitors, and is data used to optimize future proposal generation.

[0378] This invention is a system that provides personalized information to visitors and consists mainly of three elements: a server, a terminal, and a user. The core of this system is information suggestion based on the visitor's identification information and emotions.

[0379] The server receives visitor identification information transmitted from terminals via the facility's network. Based on this identification information, the server accesses a behavioral history database and extracts relevant historical information. This historical information is analyzed using AI algorithms to identify the visitor's preferences and interests. The server also uses an emotion analysis engine to recognize the visitor's emotional state from their facial expressions and voice. The resulting emotional state becomes an important element in generating information suggestions.

[0380] The terminal visually displays information suggestions received from the server. This display uses an avatar, which exhibits a tone and facial expression corresponding to the visitor's emotional state. The terminal can also accept voice input and touch input from the visitor. This allows the user to obtain detailed information through interaction with the avatar.

[0381] As a concrete example, consider a scenario where a user visits a tourist attraction. Suppose the device detects the user's face, and the emotion analysis engine recognizes the emotion of "surprise." Based on the user's behavioral history and emotional information, the server decides to suggest "a spectacular view from the observation deck." The avatar then conveys this information in an excited tone, piquing the user's interest.

[0382] An example of a prompt might be: "Please create a plan for a travel information system to provide personalized suggestions based on the visitor's emotions. Provide appropriate experiences to suggest when the visitor's emotional state is 'surprise'." This helps the generative AI model suggest services tailored to the individual emotions of each visitor.

[0383] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0384] Step 1:

[0385] The terminal acquires visitor identification information using sensors and cameras installed within the facility. The input includes the visitor's facial image and device ID. This information is encrypted and transmitted to the server via a secure protocol. The output is encrypted identification data.

[0386] Step 2:

[0387] The server accesses the behavioral history database based on the identification information received from the terminal. The visitor's identification information is provided as input. A database query is performed to extract the relevant history information. This outputs data about the visitor's past behavior and preferences.

[0388] Step 3:

[0389] The server uses an emotion analysis engine to analyze the visitor's emotional state. The input includes the visitor's facial expressions and voice tone. Natural language processing and speech recognition technologies are used to recognize emotions such as "surprise" and "stress." The output is the recognized emotion information.

[0390] Step 4:

[0391] The server integrates behavioral history and emotional information to generate personalized information suggestions. The input consists of extracted historical data and emotional information. A generative AI model is applied to calculate the most suitable information and experience for the visitor. The output is personalized information suggestion data.

[0392] Step 5:

[0393] The terminal provides visitors with information suggestions received from the server via a display mechanism. The input is generated information suggestion data. An avatar is introduced, presenting information with changes in tone and facial expression that correspond to the visitor's emotional state. The output is the visually presented information.

[0394] Step 6:

[0395] The user reviews the information presented by the avatar and requests further details as needed. Input is the information displayed on the device. The user interacts using voice input and touch gestures. Output is the detailed information obtained by the user.

[0396] Step 7:

[0397] The terminal sends user input to the server as feedback. This input consists of user operation data. The server analyzes and learns from this feedback to improve the accuracy of information suggestions. The output is the improved suggestion generation algorithm.

[0398] (Application Example 2)

[0399] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0400] In modern commercial facilities, providing more personalized services is required to increase visitor satisfaction. However, it is difficult to provide information that is appropriate to the different emotional states of each visitor. Furthermore, conventional technology is insufficient in suggesting products and services that match the visitor's emotions at that moment, making it difficult to maximize visitors' purchasing intent.

[0401] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0402] In this invention, the server includes means for receiving visitor identification information, means for accessing a set of visitor behavior history data based on the received identification information and extracting the relevant history information, and means for recognizing the visitor's emotional state and adjusting information suggestions based on that emotion. This makes it possible to suggest personalized products and services that correspond to the visitor's emotional state.

[0403] "Visitor identification information" refers to information used to identify individual people who visit a facility, and is data obtained through facial recognition, ID cards, etc.

[0404] A "behavioral history data set" is a collection of data that records visitors' past activities, preferences, and purchase history.

[0405] "Personalized information suggestions" refer to suggestions that provide information optimized according to the visitor's characteristics, preferences, and emotional state.

[0406] A "display device" is a device used to present information visually, and includes displays and smart glasses.

[0407] "Feedback" refers to data recorded from visitor responses and reactions, which is used to improve the system and optimize information suggestions.

[0408] "Emotional state" refers to the mental state a visitor is experiencing at a given time, and includes a variety of emotions such as relaxation, tension, and excitement.

[0409] A "visual display device" is a device used to provide visual information to users, and is particularly used to display emotion-based information and advertisements to visitors.

[0410] To implement this invention, a system using a server, a terminal, and a visual display device is required. The server receives visitor identification information, accesses a set of behavioral history data, and extracts relevant historical information. Furthermore, it analyzes the historical information using an AI algorithm and generates personalized information suggestions for the visitor. It also uses sentiment analysis software to recognize the visitor's emotional state and adjust the information suggestions accordingly. This makes it possible to suggest products and services that are appropriate to the visitor's emotions.

[0411] The terminal outputs information suggestions received from the server to a visual display device and presents them to the visitor. Smart glasses are used as the visual display device, transparently displaying the information. This allows visitors to obtain the necessary information without using their hands.

[0412] Users review the presented information and provide feedback as needed. This feedback is sent directly to the server and recorded in the database. The recorded feedback is used for learning to optimize future information suggestions.

[0413] As a concrete example, when a user enters a store, their emotional state is sensed through smart glasses. If the system determines the user is relaxed, it will gently guide them through information about new products. If the system determines the user is tense, it can provide promotional information about products with relaxing effects.

[0414] Example prompt: "Identify the product categories that customers are most interested in when they enter the store, and suggest appropriate product information for that situation."

[0415] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0416] Step 1:

[0417] The server receives the visitor's identification information. This identification information, such as the visitor's facial recognition data or ID card information, is passed to the server as input data. Based on the received information, the server accesses the corresponding visitor's activity history data set.

[0418] Step 2:

[0419] The server searches the behavioral history data set based on the received identification information and extracts the relevant historical information. Database queries are used for the search, and in this process, the historical data is used as input, and data calculations such as weighting and filtering are performed. Finally, historical information suitable for analysis is output.

[0420] Step 3:

[0421] The server analyzes the extracted historical information using an AI algorithm to generate personalized information suggestions for visitors. The AI ​​algorithm narrows down the candidate suggestions based on behavioral patterns and past preferences. The resulting information suggestions become the server's output.

[0422] Step 4:

[0423] The server uses sentiment analysis software to recognize the visitor's emotional state. Current facial expression data of the visitor is used as input, and the sentiment engine performs real-time analysis. The results, based on the emotional state, are used to adjust the information suggestions.

[0424] Step 5:

[0425] The terminal outputs information suggestions received from the server to smart glasses, which are a visual display device. The input here is information suggestion data, and the output is information that is visually presented to the visitor.

[0426] Step 6:

[0427] The user reviews the presented information on a visual display device and provides feedback. User feedback is input via voice or gestures, and the device sends this back to the server as input data.

[0428] Step 7:

[0429] The server records the received feedback in a historical database, accumulating data to optimize future information suggestions. The input is feedback data, and the output is an updated database.

[0430] These processing steps enable the provision of optimal information to visitors, tailored to their emotional state.

[0431] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0432] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0433] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0434] [Third Embodiment]

[0435] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0436] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0437] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0438] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0439] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0440] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0441] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0442] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0443] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0444] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0445] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0446] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0447] This invention is a system for providing personalized information to visitors. This system acquires visitor identification information, extracts relevant information from a behavioral history database based on that information, and analyzes it. Based on the analysis results, it generates optimized suggestions for the visitor and provides the information visually through a display device. The following describes each of its operations in detail.

[0448] Server operation

[0449] The server receives visitor identification information transmitted from the terminal. Based on the received identification information, the server queries a behavioral history database and extracts relevant historical data. This includes past purchase history and event participation records. Next, an AI algorithm is applied to analyze the visitor's preferences. Based on the analysis results, the server generates personalized suggestions for each visitor. These suggestions include real-time information and product recommendations, and are sent to the terminal.

[0450] Terminal operation

[0451] The terminal receives suggestion data sent from the server and outputs it to the display device. The display device shows an avatar, visually presenting the suggestion to the visitor. The visitor can interact with the terminal through voice or touch controls. The terminal sends the input information back to the server as feedback.

[0452] User actions

[0453] Users review the information displayed on the display screen. This information may include details about nearby events, special offers, and recommended products. Furthermore, users can interact with avatars to obtain more detailed information and learn about products that interest them. This allows users to quickly obtain the information they need and enhance their experience within the facility.

[0454] Specific example

[0455] For example, suppose a user visits a department store. A terminal detects the user and sends the data to a server. The server analyzes the user's past purchase history and recognizes that the user is interested in cosmetics. As a result, it suggests information such as "A special sale is currently underway on the cosmetics floor." The user accepts this suggestion and can hear the sale details from an avatar. Through this entire process, the user can obtain more personally tailored information and improve their shopping experience.

[0456] Thus, the system of the present invention realizes the provision of individualized experiences tailored to visitors, thereby enhancing the convenience of the facility and customer satisfaction.

[0457] The following describes the processing flow.

[0458] Step 1:

[0459] The server receives visitor identification information transmitted from the terminal.

[0460] Step 2:

[0461] The server accesses the behavioral history database based on the received identification information and extracts the relevant historical data.

[0462] Step 3:

[0463] The server analyzes the extracted historical data using an AI algorithm. This analysis identifies the visitor's preferences and interests.

[0464] Step 4:

[0465] Based on the analysis results, the server generates information suggestions optimized for the visitor. These suggestions include currently available event information and product recommendations.

[0466] Step 5:

[0467] The server sends the generated suggestions to the terminal.

[0468] Step 6:

[0469] The terminal outputs the suggestions received from the server to the display device. An avatar is displayed on the screen, conveying visitor information.

[0470] Step 7:

[0471] The user reviews the displayed information and interacts with the avatar as needed. Interactions are conducted via voice or touch.

[0472] Step 8:

[0473] The terminal receives input from the user and sends that feedback to the server.

[0474] Step 9:

[0475] The server records the feedback it receives and stores it in a database. This information is then used to learn how to improve future suggestions.

[0476] Step 10:

[0477] The server and terminal confirm that the user has left the area and terminate the information provision session.

[0478] (Example 1)

[0479] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0480] The challenge lies in providing personalized information in real time to individual users visiting commercial facilities and other similar establishments, based on their purchase history and interests. Solving this challenge requires technology that can present information tailored to each user's needs in a timely manner.

[0481] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0482] In this invention, the server includes means for receiving visitor identification data and identifying the visitor using a recording medium, means for accessing a recording medium that stores the visitor's behavior history based on the received identification data and obtaining the corresponding history information, and means for analyzing the obtained history information and applying a generation AI algorithm to generate personalized suggestion information for the visitor. This makes it possible to quickly provide information optimized for each user and enrich the visitor experience.

[0483] "Visitor identification data" refers to information used to identify individual users who visit a facility, and includes facial recognition data, QR codes, RFID tags, etc.

[0484] A "recording medium" is a physical or electronic device or system used to record, store, and retrieve digital information.

[0485] "Behavioral history" refers to the accumulation of information about specific actions and activities, such as a user's past visits, purchase history, and event participation history.

[0486] A "generative AI algorithm" is a computational method and procedure for automatically generating new information suggestions by analyzing user behavior and preferences based on data.

[0487] "Personalized suggestion information" refers to content that suggests the most relevant and suitable information and products to a specific user based on their visitor identification data and behavioral history.

[0488] A "display device" refers to a digital screen or monitor used to present visual information.

[0489] "Feedback" refers to information such as opinions, preferences, and usage history obtained from visitors, and is used to improve the quality of proposals.

[0490] "Learning for optimization" is a data analysis and improvement process that uses feedback and AI algorithms to improve the accuracy of suggestions.

[0491] This invention is a system that provides personalized information to visitors. The system aims to improve the user experience by providing information tailored to the individual needs and interests of visitors in real time.

[0492] The server receives visitor identification data transmitted from terminals within the facility. Identification can utilize technologies such as facial recognition, QR codes, and RFID tags. Based on the received identification data, the server searches for visitor history from storage media and retrieves relevant historical information. These storage media include cloud storage and database management systems.

[0493] The server analyzes historical information using a generation AI algorithm and generates personalized suggestion information based on the visitor's preferences. The generated suggestions are immediately sent to the terminal, which visually displays this information through a display device. Touch panels or monitors are used as the display device.

[0494] Users can review the information displayed on the device and, if necessary, request additional information or respond using voice or touch controls. The device sends this user feedback to the server, which is then used to optimize future recommendations. A concrete example of this might be a prompt message such as, "Please suggest products based on the user's past purchase history."

[0495] In this way, the system can provide information tailored to each visitor, enabling efficient and effective service delivery.

[0496] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0497] Step 1:

[0498] The terminal acquires visitor identification data. This data is obtained through facial recognition via sensors and cameras, QR code scanning, or RFID reading. This data serves as input. The terminal then sends this data to the server to initiate the identification process.

[0499] Step 2:

[0500] The server queries the activity history stored on the recording medium based on the received identification data. The server uses a database management system to search for and retrieve past history information that matches the identification data. This becomes the output. This history information includes the visitor's past purchase history and event participation history.

[0501] Step 3:

[0502] The server uses the acquired historical information to perform analysis with a generating AI model. This analysis identifies the visitor's preferences and interests, and generates personalized information suggestions. The input is historical information, and the output is suggestion information. The suggestion information includes recommended products and event announcements.

[0503] Step 4:

[0504] The server sends the generated suggestion information to the terminal. The terminal visually displays the received suggestions on a display device. To present the suggestions in a way that is easy for the user to see and understand, the display device uses a touch panel or monitor.

[0505] Step 5:

[0506] Users can review suggestions displayed on the display device and provide feedback via voice or touch. For example, they can request further details or react to the suggestions. This feedback constitutes input.

[0507] Step 6:

[0508] The device sends user feedback to the server. The server records this feedback and uses it in an optimization process to improve the accuracy of future suggestions using a generative AI model. The output is training data that will be used for future suggestions.

[0509] (Application Example 1)

[0510] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0511] To improve the visitor's purchasing experience, it is crucial to provide information in real time based on individual preferences and behavioral history. However, conventional systems have struggled to effectively utilize visitor feedback and optimize recommendations. Furthermore, there has been a lack of methods to present information to visitors using both visual and auditory means, providing an interactive user experience.

[0512] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0513] In this invention, the server includes means for receiving visitor identification information, means for referring to a behavioral history database based on the received identification information and extracting relevant history data, means for analyzing the extracted history data and generating personalized information suggestions, means for outputting the generated suggestions visually and audibly to present the information, and means for recording visitor feedback and using a generating AI model to learn and optimize future suggestions. This enables the provision of dynamic and interactive information to visitors, thereby improving the purchasing experience.

[0514] "Visitor identification information" refers to data used by the system to identify each individual, and includes information such as smartphone IDs and facial recognition data.

[0515] A "behavioral history database" is a database that stores data related to visitors' behavior, such as past purchase history and event participation records.

[0516] "Personalized information suggestions" are suggestions generated to recommend specific information or products individually based on the visitor's preferences and needs.

[0517] "Visual and audible output" refers to the use of displays and speakers to communicate information visually and audibly.

[0518] A "generative AI model" refers to an artificial intelligence technology used to analyze data and generate personalized information.

[0519] "Dynamic and interactive information delivery" refers to a method of providing information that is instantly updated and adjusted in response to user feedback and actions, and delivered through dialogue.

[0520] The system implementing this invention is based on a network system including a server and terminals to receive visitor identification information and generate personalized information suggestions. The server receives identification information transmitted from a smartphone or in-store terminal when a visitor enters the facility and uses this information to access a behavioral history database. This database stores the visitor's past purchase history and event participation records, and relevant data can be extracted.

[0521] The server uses AI algorithms to analyze extracted historical data and generate personalized information suggestions based on the visitor's preferences and interests. These suggestions include visual and auditory presentations, delivered through displays and speakers. The generated suggestions are dynamic and interactive, forming the basis for optimizing subsequent information delivery based on visitor feedback.

[0522] The terminal serves to facilitate interaction with visitors, allowing them to provide feedback through voice or touch operations. This feedback is then sent back to the server and analyzed using a generative AI model to continuously optimize the suggestions.

[0523] As a concrete example, in one bookstore, if analysis indicates that a customer is interested in mystery novels, information about ongoing "mystery novel fairs" and new releases is visually displayed on the screen, along with audio recommendations. Based on this information, the user can also interact with an avatar to learn more details.

[0524] An example of a prompt message would be: "Use the user's past purchase history data to generate personalized information suggestions. In particular, create suggestions that are appealing to the user, including promotional information for new products."

[0525] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0526] Step 1:

[0527] The server receives visitor identification information from a smartphone or in-store terminal. This identification information is used to identify the visitor. The data entered includes an identification ID and the visitor's location information. Based on this, the server verifies the visitor's uniqueness and references it to the database.

[0528] Step 2:

[0529] The server references the behavioral history database based on the received identification information and extracts the relevant historical data. The input here is the visitor's identification information, and the output includes data such as the visitor's past purchase history and event participation records. This data is used as foundational data in subsequent analyses.

[0530] Step 3:

[0531] The server analyzes the extracted historical data using an AI algorithm. This algorithm uses a generative AI model to analyze the visitor's preferences and interests. The input is behavioral history data, and the output generates suggested information related to the visitor's areas of interest. This analysis is performed based on prompt messages to determine the content of the suggestions.

[0532] Step 4:

[0533] The server sends the generated personalized information suggestions to the terminal. The terminal provides the suggested information to the visitor visually and audibly. The input here is the suggested information, and the output is the display on the screen and audio guidance via the speaker. The visitor reviews the presented information and chooses an action based on its content.

[0534] Step 5:

[0535] Users provide feedback to the device using voice or touch controls. This feedback is sent to the server as data to help the generative AI model improve its suggestions. The input is the user's actions and selections, and the output is improvement suggestions based on that information.

[0536] Step 6:

[0537] The server receives feedback and continuously optimizes its suggestions. This learning process using the data improves the accuracy of subsequent suggestions. The input here is feedback data, and the output is optimized suggestion information. This enables the continuous provision of high-quality information to visitors.

[0538] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0539] This invention is a system for providing personalized information to visitors, and by combining it with an emotion engine, it recognizes the visitor's emotional state and provides information and experiences appropriate to that emotional state. This system acquires the visitor's identification information, extracts relevant information from a behavioral history database based on that information, and analyzes it. The system comprehensively judges the analysis results and the emotions recognized by the emotion engine to generate the most appropriate information suggestions for the visitor. These suggestions are then visually presented by an avatar through a display device.

[0540] Server operation

[0541] The server accesses a behavioral history database based on visitor identification information received from the terminal and extracts relevant historical data. An AI algorithm is applied to analyze the historical data to identify the visitor's preferences and interests. Furthermore, the visitor's emotional state, as determined by the emotion engine, is also incorporated into the analysis results. For example, the suggestions are adjusted to differ depending on whether the visitor is relaxed or stressed.

[0542] Terminal operation

[0543] The terminal outputs the suggested data received from the server to the display device. The avatar visually provides information, adjusting its tone and facial expressions according to the visitor's emotional state. In addition, the terminal accepts input from the visitor, enabling interaction through voice and touch controls. The received input is sent directly to the server as feedback.

[0544] User actions

[0545] Users can view the information displayed on the screen and obtain more detailed information through interaction with the avatar. For example, if the user is calm, the avatar will suggest relaxing restaurant information in a gentle tone. On the other hand, if the user is stressed, the avatar will provide information on refreshing activities in an encouraging tone.

[0546] Specific example

[0547] As a concrete example of use, suppose a user visits an information center at a travel destination. The device detects the user's face, and the emotion engine recognizes the emotion of "surprise." Based on past behavior history and this emotion information, the server decides to recommend "the spectacular view from the observation deck" to the user. The avatar speaks in an excited tone to keep the user interested. Through this entire process, the user can have an emotionally satisfying experience.

[0548] Thus, the present invention aims to realize a personalized experience that takes into account the emotional state of visitors, thereby improving facility services and dramatically increasing customer satisfaction.

[0549] The following describes the processing flow.

[0550] Step 1:

[0551] The terminal receives visitor identification information using sensors and camera technology.

[0552] Step 2:

[0553] The device transmits the visitor's facial expressions and voice to an emotion engine, recognizing emotions in real time.

[0554] Step 3:

[0555] The server accesses the behavioral history database based on the identification information sent from the terminal and extracts the corresponding history data.

[0556] Step 4:

[0557] The server analyzes the extracted historical data using an AI algorithm to identify the visitor's preferences.

[0558] Step 5:

[0559] The server integrates the analysis results with emotion data from the emotion engine to generate personalized information suggestions for visitors. These suggestions include information tailored to their emotional state.

[0560] Step 6:

[0561] The server sends the generated suggestions to the terminal.

[0562] Step 7:

[0563] The terminal outputs suggestions received from the server to a display device, providing visitors with information visually through an avatar. The avatar's tone and facial expressions are adjusted to match the recognized emotions.

[0564] Step 8:

[0565] The user reviews the displayed information and interacts with the avatar as needed. Interactions are conducted via voice or touch, and the input is recorded on the device.

[0566] Step 9:

[0567] The device sends user input and feedback to the server.

[0568] Step 10:

[0569] The server records the feedback it receives and stores it in a database. The server uses this information to learn from future suggestions and make improvements.

[0570] Step 11:

[0571] The server and terminal detect when the user leaves the signage area and terminate the information provision process.

[0572] (Example 2)

[0573] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0574] Providing visitors with personalized information and experiences immediately upon arrival at a facility has been difficult with conventional technology. In particular, there was a lack of systems to understand visitors' emotional states and provide the most appropriate services based on that information. Therefore, it is necessary to increase the satisfaction of each individual visitor and improve the overall service quality of the facility.

[0575] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0576] In this invention, the server includes means for accessing a behavioral history database based on visitor identification information and extracting relevant history information, means for recognizing the visitor's emotional state using an emotion analysis engine, and means for visually presenting personalized information suggestions to the visitor. This enables the provision of information and optimization of the experience according to the individual emotional state of each visitor.

[0577] A "visitor" refers to an individual or group visiting the facility, whose behavior and emotional state are observed by the system.

[0578] "Identification information" refers to data used to identify visitors, and may take the form of facial recognition or device ID.

[0579] A "behavioral history database" is an accessible information resource that stores information about visitors' past visit history and preferences.

[0580] An "emotion analysis engine" is a technology used to analyze the emotional state of visitors, recognizing emotions by analyzing facial expressions, tone of voice, and other factors.

[0581] "Information suggestions" are personalized suggestions of information and experiences generated based on a visitor's identification information and emotional state.

[0582] A "display means" is a device used to visually present information suggestions to visitors, and may consist of a screen or an avatar.

[0583] "Feedback information" refers to input or response information obtained from visitors, and is data used to optimize future proposal generation.

[0584] This invention is a system that provides personalized information to visitors and consists mainly of three elements: a server, a terminal, and a user. The core of this system is information suggestion based on the visitor's identification information and emotions.

[0585] The server receives visitor identification information transmitted from terminals via the facility's network. Based on this identification information, the server accesses a behavioral history database and extracts relevant historical information. This historical information is analyzed using AI algorithms to identify the visitor's preferences and interests. The server also uses an emotion analysis engine to recognize the visitor's emotional state from their facial expressions and voice. The resulting emotional state becomes an important element in generating information suggestions.

[0586] The terminal visually displays information suggestions received from the server. This display uses an avatar, which exhibits a tone and facial expression corresponding to the visitor's emotional state. The terminal can also accept voice input and touch input from the visitor. This allows the user to obtain detailed information through interaction with the avatar.

[0587] As a concrete example, consider a scenario where a user visits a tourist attraction. Suppose the device detects the user's face, and the emotion analysis engine recognizes the emotion of "surprise." Based on the user's behavioral history and emotional information, the server decides to suggest "a spectacular view from the observation deck." The avatar then conveys this information in an excited tone, piquing the user's interest.

[0588] An example of a prompt might be: "Please create a plan for a travel information system to provide personalized suggestions based on the visitor's emotions. Provide appropriate experiences to suggest when the visitor's emotional state is 'surprise'." This helps the generative AI model suggest services tailored to the individual emotions of each visitor.

[0589] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0590] Step 1:

[0591] The terminal acquires visitor identification information using sensors and cameras installed within the facility. The input includes the visitor's facial image and device ID. This information is encrypted and transmitted to the server via a secure protocol. The output is encrypted identification data.

[0592] Step 2:

[0593] The server accesses the behavioral history database based on the identification information received from the terminal. The visitor's identification information is provided as input. A database query is performed to extract the relevant history information. This outputs data about the visitor's past behavior and preferences.

[0594] Step 3:

[0595] The server uses an emotion analysis engine to analyze the visitor's emotional state. The input includes the visitor's facial expressions and voice tone. Natural language processing and speech recognition technologies are used to recognize emotions such as "surprise" and "stress." The output is the recognized emotion information.

[0596] Step 4:

[0597] The server integrates behavioral history and emotional information to generate personalized information suggestions. The input consists of extracted historical data and emotional information. A generative AI model is applied to calculate the most suitable information and experience for the visitor. The output is personalized information suggestion data.

[0598] Step 5:

[0599] The terminal provides visitors with information suggestions received from the server via a display mechanism. The input is generated information suggestion data. An avatar is introduced, presenting information with changes in tone and facial expression that correspond to the visitor's emotional state. The output is the visually presented information.

[0600] Step 6:

[0601] The user reviews the information presented by the avatar and requests further details as needed. Input is the information displayed on the device. The user interacts using voice input and touch gestures. Output is the detailed information obtained by the user.

[0602] Step 7:

[0603] The terminal sends user input to the server as feedback. This input consists of user operation data. The server analyzes and learns from this feedback to improve the accuracy of information suggestions. The output is the improved suggestion generation algorithm.

[0604] (Application Example 2)

[0605] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0606] In modern commercial facilities, providing more personalized services is required to increase visitor satisfaction. However, it is difficult to provide information that is appropriate to the different emotional states of each visitor. Furthermore, conventional technology is insufficient in suggesting products and services that match the visitor's emotions at that moment, making it difficult to maximize visitors' purchasing intent.

[0607] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0608] In this invention, the server includes means for receiving visitor identification information, means for accessing a set of visitor behavior history data based on the received identification information and extracting the relevant history information, and means for recognizing the visitor's emotional state and adjusting information suggestions based on that emotion. This makes it possible to suggest personalized products and services that correspond to the visitor's emotional state.

[0609] "Visitor identification information" refers to information used to identify individual people who visit a facility, and is data obtained through facial recognition, ID cards, etc.

[0610] A "behavioral history data set" is a collection of data that records visitors' past activities, preferences, and purchase history.

[0611] "Personalized information suggestions" refer to suggestions that provide information optimized according to the visitor's characteristics, preferences, and emotional state.

[0612] A "display device" is a device used to present information visually, and includes displays and smart glasses.

[0613] "Feedback" refers to data recorded from visitor responses and reactions, which is used to improve the system and optimize information suggestions.

[0614] "Emotional state" refers to the mental state a visitor is experiencing at a given time, and includes a variety of emotions such as relaxation, tension, and excitement.

[0615] A "visual display device" is a device used to provide visual information to users, and is particularly used to display emotion-based information and advertisements to visitors.

[0616] To implement this invention, a system using a server, a terminal, and a visual display device is required. The server receives visitor identification information, accesses a set of behavioral history data, and extracts relevant historical information. Furthermore, it analyzes the historical information using an AI algorithm and generates personalized information suggestions for the visitor. It also uses sentiment analysis software to recognize the visitor's emotional state and adjust the information suggestions accordingly. This makes it possible to suggest products and services that are appropriate to the visitor's emotions.

[0617] The terminal outputs information suggestions received from the server to a visual display device and presents them to the visitor. Smart glasses are used as the visual display device, transparently displaying the information. This allows visitors to obtain the necessary information without using their hands.

[0618] Users review the presented information and provide feedback as needed. This feedback is sent directly to the server and recorded in the database. The recorded feedback is used for learning to optimize future information suggestions.

[0619] As a concrete example, when a user enters a store, their emotional state is sensed through smart glasses. If the system determines the user is relaxed, it will gently guide them through information about new products. If the system determines the user is tense, it can provide promotional information about products with relaxing effects.

[0620] Example prompt: "Identify the product categories that customers are most interested in when they enter the store, and suggest appropriate product information for that situation."

[0621] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0622] Step 1:

[0623] The server receives the visitor's identification information. This identification information, such as the visitor's facial recognition data or ID card information, is passed to the server as input data. Based on the received information, the server accesses the corresponding visitor's activity history data set.

[0624] Step 2:

[0625] The server searches the behavioral history data set based on the received identification information and extracts the relevant historical information. Database queries are used for the search, and in this process, the historical data is used as input, and data calculations such as weighting and filtering are performed. Finally, historical information suitable for analysis is output.

[0626] Step 3:

[0627] The server analyzes the extracted historical information using an AI algorithm to generate personalized information suggestions for visitors. The AI ​​algorithm narrows down the candidate suggestions based on behavioral patterns and past preferences. The resulting information suggestions become the server's output.

[0628] Step 4:

[0629] The server uses sentiment analysis software to recognize the visitor's emotional state. Current facial expression data of the visitor is used as input, and the sentiment engine performs real-time analysis. The results, based on the emotional state, are used to adjust the information suggestions.

[0630] Step 5:

[0631] The terminal outputs information suggestions received from the server to smart glasses, which are a visual display device. The input here is information suggestion data, and the output is information that is visually presented to the visitor.

[0632] Step 6:

[0633] The user reviews the presented information on a visual display device and provides feedback. User feedback is input via voice or gestures, and the device sends this back to the server as input data.

[0634] Step 7:

[0635] The server records the received feedback in a historical database, accumulating data to optimize future information suggestions. The input is feedback data, and the output is an updated database.

[0636] These processing steps enable the provision of optimal information to visitors, tailored to their emotional state.

[0637] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0638] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0639] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0640] [Fourth Embodiment]

[0641] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0642] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0643] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0644] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0645] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0646] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0647] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0648] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0649] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0650] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0651] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0652] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0653] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0654] This invention is a system for providing personalized information to visitors. This system acquires visitor identification information, extracts relevant information from a behavioral history database based on that information, and analyzes it. Based on the analysis results, it generates optimized suggestions for the visitor and provides the information visually through a display device. The following describes each of its operations in detail.

[0655] Server operation

[0656] The server receives visitor identification information transmitted from the terminal. Based on the received identification information, the server queries a behavioral history database and extracts relevant historical data. This includes past purchase history and event participation records. Next, an AI algorithm is applied to analyze the visitor's preferences. Based on the analysis results, the server generates personalized suggestions for each visitor. These suggestions include real-time information and product recommendations, and are sent to the terminal.

[0657] Terminal operation

[0658] The terminal receives suggestion data sent from the server and outputs it to the display device. The display device shows an avatar, visually presenting the suggestion to the visitor. The visitor can interact with the terminal through voice or touch controls. The terminal sends the input information back to the server as feedback.

[0659] User actions

[0660] Users review the information displayed on the display screen. This information may include details about nearby events, special offers, and recommended products. Furthermore, users can interact with avatars to obtain more detailed information and learn about products that interest them. This allows users to quickly obtain the information they need and enhance their experience within the facility.

[0661] Specific example

[0662] For example, suppose a user visits a department store. A terminal detects the user and sends the data to a server. The server analyzes the user's past purchase history and recognizes that the user is interested in cosmetics. As a result, it suggests information such as "A special sale is currently underway on the cosmetics floor." The user accepts this suggestion and can hear the sale details from an avatar. Through this entire process, the user can obtain more personally tailored information and improve their shopping experience.

[0663] Thus, the system of the present invention realizes the provision of individualized experiences tailored to visitors, thereby enhancing the convenience of the facility and customer satisfaction.

[0664] The following describes the processing flow.

[0665] Step 1:

[0666] The server receives visitor identification information transmitted from the terminal.

[0667] Step 2:

[0668] The server accesses the behavioral history database based on the received identification information and extracts the relevant historical data.

[0669] Step 3:

[0670] The server analyzes the extracted historical data using an AI algorithm. This analysis identifies the visitor's preferences and interests.

[0671] Step 4:

[0672] Based on the analysis results, the server generates information suggestions optimized for the visitor. These suggestions include currently available event information and product recommendations.

[0673] Step 5:

[0674] The server sends the generated suggestions to the terminal.

[0675] Step 6:

[0676] The terminal outputs the suggestions received from the server to the display device. An avatar is displayed on the screen, conveying visitor information.

[0677] Step 7:

[0678] The user reviews the displayed information and interacts with the avatar as needed. Interactions are conducted via voice or touch.

[0679] Step 8:

[0680] The terminal receives input from the user and sends that feedback to the server.

[0681] Step 9:

[0682] The server records the feedback it receives and stores it in a database. This information is then used to learn how to improve future suggestions.

[0683] Step 10:

[0684] The server and terminal confirm that the user has left the area and terminate the information provision session.

[0685] (Example 1)

[0686] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0687] The challenge lies in providing personalized information in real time to individual users visiting commercial facilities and other similar establishments, based on their purchase history and interests. Solving this challenge requires technology that can present information tailored to each user's needs in a timely manner.

[0688] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0689] In this invention, the server includes means for receiving visitor identification data and identifying the visitor using a recording medium, means for accessing a recording medium that stores the visitor's behavior history based on the received identification data and obtaining the corresponding history information, and means for analyzing the obtained history information and applying a generation AI algorithm to generate personalized suggestion information for the visitor. This makes it possible to quickly provide information optimized for each user and enrich the visitor experience.

[0690] "Visitor identification data" refers to information used to identify individual users who visit a facility, and includes facial recognition data, QR codes, RFID tags, etc.

[0691] A "recording medium" is a physical or electronic device or system used to record, store, and retrieve digital information.

[0692] "Behavioral history" refers to the accumulation of information about specific actions and activities, such as a user's past visits, purchase history, and event participation history.

[0693] A "generative AI algorithm" is a computational method and procedure for automatically generating new information suggestions by analyzing user behavior and preferences based on data.

[0694] "Personalized suggestion information" refers to content that suggests the most relevant and suitable information and products to a specific user based on their visitor identification data and behavioral history.

[0695] A "display device" refers to a digital screen or monitor used to present visual information.

[0696] "Feedback" refers to information such as opinions, preferences, and usage history obtained from visitors, and is used to improve the quality of proposals.

[0697] "Learning for optimization" is a data analysis and improvement process that uses feedback and AI algorithms to improve the accuracy of suggestions.

[0698] This invention is a system that provides personalized information to visitors. The system aims to improve the user experience by providing information tailored to the individual needs and interests of visitors in real time.

[0699] The server receives visitor identification data transmitted from terminals within the facility. Identification can utilize technologies such as facial recognition, QR codes, and RFID tags. Based on the received identification data, the server searches for visitor history from storage media and retrieves relevant historical information. These storage media include cloud storage and database management systems.

[0700] The server analyzes historical information using a generation AI algorithm and generates personalized suggestion information based on the visitor's preferences. The generated suggestions are immediately sent to the terminal, which visually displays this information through a display device. Touch panels or monitors are used as the display device.

[0701] Users can review the information displayed on the device and, if necessary, request additional information or respond using voice or touch controls. The device sends this user feedback to the server, which is then used to optimize future recommendations. A concrete example of this might be a prompt message such as, "Please suggest products based on the user's past purchase history."

[0702] In this way, the system can provide information tailored to each visitor, enabling efficient and effective service delivery.

[0703] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0704] Step 1:

[0705] The terminal acquires visitor identification data. This data is obtained through facial recognition via sensors and cameras, QR code scanning, or RFID reading. This data serves as input. The terminal then sends this data to the server to initiate the identification process.

[0706] Step 2:

[0707] The server queries the activity history stored on the recording medium based on the received identification data. The server uses a database management system to search for and retrieve past history information that matches the identification data. This becomes the output. This history information includes the visitor's past purchase history and event participation history.

[0708] Step 3:

[0709] The server uses the acquired historical information to perform analysis with a generating AI model. This analysis identifies the visitor's preferences and interests, and generates personalized information suggestions. The input is historical information, and the output is suggestion information. The suggestion information includes recommended products and event announcements.

[0710] Step 4:

[0711] The server sends the generated suggestion information to the terminal. The terminal visually displays the received suggestions on a display device. To present the suggestions in a way that is easy for the user to see and understand, the display device uses a touch panel or monitor.

[0712] Step 5:

[0713] Users can review suggestions displayed on the display device and provide feedback via voice or touch. For example, they can request further details or react to the suggestions. This feedback constitutes input.

[0714] Step 6:

[0715] The device sends user feedback to the server. The server records this feedback and uses it in an optimization process to improve the accuracy of future suggestions using a generative AI model. The output is training data that will be used for future suggestions.

[0716] (Application Example 1)

[0717] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0718] To improve the visitor's purchasing experience, it is crucial to provide information in real time based on individual preferences and behavioral history. However, conventional systems have struggled to effectively utilize visitor feedback and optimize recommendations. Furthermore, there has been a lack of methods to present information to visitors using both visual and auditory means, providing an interactive user experience.

[0719] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0720] In this invention, the server includes means for receiving visitor identification information, means for referring to a behavioral history database based on the received identification information and extracting relevant history data, means for analyzing the extracted history data and generating personalized information suggestions, means for outputting the generated suggestions visually and audibly to present the information, and means for recording visitor feedback and using a generating AI model to learn and optimize future suggestions. This enables the provision of dynamic and interactive information to visitors, thereby improving the purchasing experience.

[0721] "Visitor identification information" refers to data used by the system to identify each individual, and includes information such as smartphone IDs and facial recognition data.

[0722] A "behavioral history database" is a database that stores data related to visitors' behavior, such as past purchase history and event participation records.

[0723] "Personalized information suggestions" are suggestions generated to recommend specific information or products individually based on the visitor's preferences and needs.

[0724] "Visual and audible output" refers to the use of displays and speakers to communicate information visually and audibly.

[0725] A "generative AI model" refers to an artificial intelligence technology used to analyze data and generate personalized information.

[0726] "Dynamic and interactive information delivery" refers to a method of providing information that is instantly updated and adjusted in response to user feedback and actions, and delivered through dialogue.

[0727] The system implementing this invention is based on a network system including a server and terminals to receive visitor identification information and generate personalized information suggestions. The server receives identification information transmitted from a smartphone or in-store terminal when a visitor enters the facility and uses this information to access a behavioral history database. This database stores the visitor's past purchase history and event participation records, and relevant data can be extracted.

[0728] The server uses AI algorithms to analyze extracted historical data and generate personalized information suggestions based on the visitor's preferences and interests. These suggestions include visual and auditory presentations, delivered through displays and speakers. The generated suggestions are dynamic and interactive, forming the basis for optimizing subsequent information delivery based on visitor feedback.

[0729] The terminal serves to facilitate interaction with visitors, allowing them to provide feedback through voice or touch operations. This feedback is then sent back to the server and analyzed using a generative AI model to continuously optimize the suggestions.

[0730] As a concrete example, in one bookstore, if analysis indicates that a customer is interested in mystery novels, information about ongoing "mystery novel fairs" and new releases is visually displayed on the screen, along with audio recommendations. Based on this information, the user can also interact with an avatar to learn more details.

[0731] An example of a prompt message would be: "Use the user's past purchase history data to generate personalized information suggestions. In particular, create suggestions that are appealing to the user, including promotional information for new products."

[0732] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0733] Step 1:

[0734] The server receives visitor identification information from a smartphone or in-store terminal. This identification information is used to identify the visitor. The data entered includes an identification ID and the visitor's location information. Based on this, the server verifies the visitor's uniqueness and references it to the database.

[0735] Step 2:

[0736] The server references the behavioral history database based on the received identification information and extracts the relevant historical data. The input here is the visitor's identification information, and the output includes data such as the visitor's past purchase history and event participation records. This data is used as foundational data in subsequent analyses.

[0737] Step 3:

[0738] The server analyzes the extracted historical data using an AI algorithm. This algorithm uses a generative AI model to analyze the visitor's preferences and interests. The input is behavioral history data, and the output generates suggested information related to the visitor's areas of interest. This analysis is performed based on prompt messages to determine the content of the suggestions.

[0739] Step 4:

[0740] The server sends the generated personalized information suggestions to the terminal. The terminal provides the suggested information to the visitor visually and audibly. The input here is the suggested information, and the output is the display on the screen and audio guidance via the speaker. The visitor reviews the presented information and chooses an action based on its content.

[0741] Step 5:

[0742] Users provide feedback to the device using voice or touch controls. This feedback is sent to the server as data to help the generative AI model improve its suggestions. The input is the user's actions and selections, and the output is improvement suggestions based on that information.

[0743] Step 6:

[0744] The server receives feedback and continuously optimizes its suggestions. This learning process using the data improves the accuracy of subsequent suggestions. The input here is feedback data, and the output is optimized suggestion information. This enables the continuous provision of high-quality information to visitors.

[0745] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0746] This invention is a system for providing personalized information to visitors, and by combining it with an emotion engine, it recognizes the visitor's emotional state and provides information and experiences appropriate to that emotional state. This system acquires the visitor's identification information, extracts relevant information from a behavioral history database based on that information, and analyzes it. The system comprehensively judges the analysis results and the emotions recognized by the emotion engine to generate the most appropriate information suggestions for the visitor. These suggestions are then visually presented by an avatar through a display device.

[0747] Server operation

[0748] The server accesses a behavioral history database based on visitor identification information received from the terminal and extracts relevant historical data. An AI algorithm is applied to analyze the historical data to identify the visitor's preferences and interests. Furthermore, the visitor's emotional state, as determined by the emotion engine, is also incorporated into the analysis results. For example, the suggestions are adjusted to differ depending on whether the visitor is relaxed or stressed.

[0749] Terminal operation

[0750] The terminal outputs the suggested data received from the server to the display device. The avatar visually provides information, adjusting its tone and facial expressions according to the visitor's emotional state. In addition, the terminal accepts input from the visitor, enabling interaction through voice and touch controls. The received input is sent directly to the server as feedback.

[0751] User actions

[0752] Users can view the information displayed on the screen and obtain more detailed information through interaction with the avatar. For example, if the user is calm, the avatar will suggest relaxing restaurant information in a gentle tone. On the other hand, if the user is stressed, the avatar will provide information on refreshing activities in an encouraging tone.

[0753] Specific example

[0754] As a concrete example of use, suppose a user visits an information center at a travel destination. The device detects the user's face, and the emotion engine recognizes the emotion of "surprise." Based on past behavior history and this emotion information, the server decides to recommend "the spectacular view from the observation deck" to the user. The avatar speaks in an excited tone to keep the user interested. Through this entire process, the user can have an emotionally satisfying experience.

[0755] Thus, the present invention aims to realize a personalized experience that takes into account the emotional state of visitors, thereby improving facility services and dramatically increasing customer satisfaction.

[0756] The following describes the processing flow.

[0757] Step 1:

[0758] The terminal receives visitor identification information using sensors and camera technology.

[0759] Step 2:

[0760] The device transmits the visitor's facial expressions and voice to an emotion engine, recognizing emotions in real time.

[0761] Step 3:

[0762] The server accesses the behavioral history database based on the identification information sent from the terminal and extracts the corresponding history data.

[0763] Step 4:

[0764] The server analyzes the extracted historical data using an AI algorithm to identify the visitor's preferences.

[0765] Step 5:

[0766] The server integrates the analysis results with emotion data from the emotion engine to generate personalized information suggestions for visitors. These suggestions include information tailored to their emotional state.

[0767] Step 6:

[0768] The server sends the generated suggestions to the terminal.

[0769] Step 7:

[0770] The terminal outputs suggestions received from the server to a display device, providing visitors with information visually through an avatar. The avatar's tone and facial expressions are adjusted to match the recognized emotions.

[0771] Step 8:

[0772] The user reviews the displayed information and interacts with the avatar as needed. Interactions are conducted via voice or touch, and the input is recorded on the device.

[0773] Step 9:

[0774] The device sends user input and feedback to the server.

[0775] Step 10:

[0776] The server records the feedback it receives and stores it in a database. The server uses this information to learn from future suggestions and make improvements.

[0777] Step 11:

[0778] The server and terminal detect when the user leaves the signage area and terminate the information provision process.

[0779] (Example 2)

[0780] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0781] Providing visitors with personalized information and experiences immediately upon arrival at a facility has been difficult with conventional technology. In particular, there was a lack of systems to understand visitors' emotional states and provide the most appropriate services based on that information. Therefore, it is necessary to increase the satisfaction of each individual visitor and improve the overall service quality of the facility.

[0782] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0783] In this invention, the server includes means for accessing a behavioral history database based on visitor identification information and extracting relevant history information, means for recognizing the visitor's emotional state using an emotion analysis engine, and means for visually presenting personalized information suggestions to the visitor. This enables the provision of information and optimization of the experience according to the individual emotional state of each visitor.

[0784] A "visitor" refers to an individual or group visiting the facility, whose behavior and emotional state are observed by the system.

[0785] "Identification information" refers to data used to identify visitors, and may take the form of facial recognition or device ID.

[0786] A "behavioral history database" is an accessible information resource that stores information about visitors' past visit history and preferences.

[0787] An "emotion analysis engine" is a technology used to analyze the emotional state of visitors, recognizing emotions by analyzing facial expressions, tone of voice, and other factors.

[0788] "Information suggestions" are personalized suggestions of information and experiences generated based on a visitor's identification information and emotional state.

[0789] A "display means" is a device used to visually present information suggestions to visitors, and may consist of a screen or an avatar.

[0790] "Feedback information" refers to input or response information obtained from visitors, and is data used to optimize future proposal generation.

[0791] This invention is a system that provides personalized information to visitors and consists mainly of three elements: a server, a terminal, and a user. The core of this system is information suggestion based on the visitor's identification information and emotions.

[0792] The server receives visitor identification information transmitted from terminals via the facility's network. Based on this identification information, the server accesses a behavioral history database and extracts relevant historical information. This historical information is analyzed using AI algorithms to identify the visitor's preferences and interests. The server also uses an emotion analysis engine to recognize the visitor's emotional state from their facial expressions and voice. The resulting emotional state becomes an important element in generating information suggestions.

[0793] The terminal visually displays information suggestions received from the server. This display uses an avatar, which exhibits a tone and facial expression corresponding to the visitor's emotional state. The terminal can also accept voice input and touch input from the visitor. This allows the user to obtain detailed information through interaction with the avatar.

[0794] As a concrete example, consider a scenario where a user visits a tourist attraction. Suppose the device detects the user's face, and the emotion analysis engine recognizes the emotion of "surprise." Based on the user's behavioral history and emotional information, the server decides to suggest "a spectacular view from the observation deck." The avatar then conveys this information in an excited tone, piquing the user's interest.

[0795] An example of a prompt might be: "Please create a plan for a travel information system to provide personalized suggestions based on the visitor's emotions. Provide appropriate experiences to suggest when the visitor's emotional state is 'surprise'." This helps the generative AI model suggest services tailored to the individual emotions of each visitor.

[0796] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0797] Step 1:

[0798] The terminal acquires visitor identification information using sensors and cameras installed within the facility. The input includes the visitor's facial image and device ID. This information is encrypted and transmitted to the server via a secure protocol. The output is encrypted identification data.

[0799] Step 2:

[0800] The server accesses the behavioral history database based on the identification information received from the terminal. The visitor's identification information is provided as input. A database query is performed to extract the relevant history information. This outputs data about the visitor's past behavior and preferences.

[0801] Step 3:

[0802] The server uses an emotion analysis engine to analyze the visitor's emotional state. The input includes the visitor's facial expressions and voice tone. Natural language processing and speech recognition technologies are used to recognize emotions such as "surprise" and "stress." The output is the recognized emotion information.

[0803] Step 4:

[0804] The server integrates behavioral history and emotional information to generate personalized information suggestions. The input consists of extracted historical data and emotional information. A generative AI model is applied to calculate the most suitable information and experience for the visitor. The output is personalized information suggestion data.

[0805] Step 5:

[0806] The terminal provides visitors with information suggestions received from the server via a display mechanism. The input is generated information suggestion data. An avatar is introduced, presenting information with changes in tone and facial expression that correspond to the visitor's emotional state. The output is the visually presented information.

[0807] Step 6:

[0808] The user reviews the information presented by the avatar and requests further details as needed. Input is the information displayed on the device. The user interacts using voice input and touch gestures. Output is the detailed information obtained by the user.

[0809] Step 7:

[0810] The terminal sends user input to the server as feedback. This input consists of user operation data. The server analyzes and learns from this feedback to improve the accuracy of information suggestions. The output is the improved suggestion generation algorithm.

[0811] (Application Example 2)

[0812] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0813] In modern commercial facilities, providing more personalized services is required to increase visitor satisfaction. However, it is difficult to provide information that is appropriate to the different emotional states of each visitor. Furthermore, conventional technology is insufficient in suggesting products and services that match the visitor's emotions at that moment, making it difficult to maximize visitors' purchasing intent.

[0814] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0815] In this invention, the server includes means for receiving visitor identification information, means for accessing a set of visitor behavior history data based on the received identification information and extracting the relevant history information, and means for recognizing the visitor's emotional state and adjusting information suggestions based on that emotion. This makes it possible to suggest personalized products and services that correspond to the visitor's emotional state.

[0816] "Visitor identification information" refers to information used to identify individual people who visit a facility, and is data obtained through facial recognition, ID cards, etc.

[0817] A "behavioral history data set" is a collection of data that records visitors' past activities, preferences, and purchase history.

[0818] "Personalized information suggestions" refer to suggestions that provide information optimized according to the visitor's characteristics, preferences, and emotional state.

[0819] A "display device" is a device used to present information visually, and includes displays and smart glasses.

[0820] "Feedback" refers to data recorded from visitor responses and reactions, which is used to improve the system and optimize information suggestions.

[0821] "Emotional state" refers to the mental state a visitor is experiencing at a given time, and includes a variety of emotions such as relaxation, tension, and excitement.

[0822] A "visual display device" is a device used to provide visual information to users, and is particularly used to display emotion-based information and advertisements to visitors.

[0823] To implement this invention, a system using a server, a terminal, and a visual display device is required. The server receives visitor identification information, accesses a set of behavioral history data, and extracts relevant historical information. Furthermore, it analyzes the historical information using an AI algorithm and generates personalized information suggestions for the visitor. It also uses sentiment analysis software to recognize the visitor's emotional state and adjust the information suggestions accordingly. This makes it possible to suggest products and services that are appropriate to the visitor's emotions.

[0824] The terminal outputs information suggestions received from the server to a visual display device and presents them to the visitor. Smart glasses are used as the visual display device, transparently displaying the information. This allows visitors to obtain the necessary information without using their hands.

[0825] Users review the presented information and provide feedback as needed. This feedback is sent directly to the server and recorded in the database. The recorded feedback is used for learning to optimize future information suggestions.

[0826] As a concrete example, when a user enters a store, their emotional state is sensed through smart glasses. If the system determines the user is relaxed, it will gently guide them through information about new products. If the system determines the user is tense, it can provide promotional information about products with relaxing effects.

[0827] Example prompt: "Identify the product categories that customers are most interested in when they enter the store, and suggest appropriate product information for that situation."

[0828] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0829] Step 1:

[0830] The server receives the visitor's identification information. This identification information, such as the visitor's facial recognition data or ID card information, is passed to the server as input data. Based on the received information, the server accesses the corresponding visitor's activity history data set.

[0831] Step 2:

[0832] The server searches the behavioral history data set based on the received identification information and extracts the relevant historical information. Database queries are used for the search, and in this process, the historical data is used as input, and data calculations such as weighting and filtering are performed. Finally, historical information suitable for analysis is output.

[0833] Step 3:

[0834] The server analyzes the extracted historical information using an AI algorithm to generate personalized information suggestions for visitors. The AI ​​algorithm narrows down the candidate suggestions based on behavioral patterns and past preferences. The resulting information suggestions become the server's output.

[0835] Step 4:

[0836] The server uses sentiment analysis software to recognize the visitor's emotional state. Current facial expression data of the visitor is used as input, and the sentiment engine performs real-time analysis. The results, based on the emotional state, are used to adjust the information suggestions.

[0837] Step 5:

[0838] The terminal outputs information suggestions received from the server to smart glasses, which are a visual display device. The input here is information suggestion data, and the output is information that is visually presented to the visitor.

[0839] Step 6:

[0840] The user reviews the presented information on a visual display device and provides feedback. User feedback is input via voice or gestures, and the device sends this back to the server as input data.

[0841] Step 7:

[0842] The server records the received feedback in a historical database, accumulating data to optimize future information suggestions. The input is feedback data, and the output is an updated database.

[0843] These processing steps enable the provision of optimal information to visitors, tailored to their emotional state.

[0844] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0845] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0846] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0847] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0848] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0849] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0850] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0851] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0852] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0853] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0854] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0855] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0856] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0857] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0858] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0859] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0860] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0861] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0862] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0863] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0864] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0865] The following is further disclosed regarding the embodiments described above.

[0866] (Claim 1)

[0867] A means of receiving visitor identification information,

[0868] A means for accessing a visitor's behavior history database based on the received identification information and extracting the relevant history data,

[0869] A means of analyzing extracted historical data to generate personalized information suggestions for visitors,

[0870] A means of outputting the generated proposal to a display device and presenting the information to visitors,

[0871] A means of receiving visitor input and recording the feedback received,

[0872] A system that includes means for analyzing recorded feedback and learning to optimize future suggestions.

[0873] (Claim 2)

[0874] The system according to claim 1, which detects when a visitor enters or leaves a signage area and reflects that information in the operation of the system.

[0875] (Claim 3)

[0876] The system according to claim 1, which performs calculations to make optimal real-time suggestions, taking into account the visitor's current location information and schedule information.

[0877] "Example 1"

[0878] (Claim 1)

[0879] A means for receiving visitor identification data and identifying them using a recording medium,

[0880] A means for accessing a recording medium that stores the visitor's behavior history based on the received identification data and obtaining the corresponding history information,

[0881] A means for analyzing acquired historical information and applying a generation AI algorithm to generate personalized suggestion information for visitors,

[0882] A means for transmitting the generated proposal to a display device and presenting it visually to the visitor,

[0883] A means of receiving visitor input and saving the received feedback to a recording medium,

[0884] A system that includes means for analyzing recorded feedback and using a generative AI model to learn how to optimize future suggestions.

[0885] (Claim 2)

[0886] The system according to claim 1, which detects when a visitor enters or leaves the area of ​​a display device and reflects that information in the operation of the system.

[0887] (Claim 3)

[0888] The system according to claim 1, which references the visitor's current location information and schedule information and performs calculations to make optimized immediate suggestions.

[0889] "Application Example 1"

[0890] (Claim 1)

[0891] A means of receiving visitor identification information,

[0892] A means for accessing a visitor's behavior history database based on the received identification information and extracting the relevant history data,

[0893] A means of analyzing extracted historical data to generate personalized information suggestions for visitors,

[0894] A means for outputting the generated proposal to a display device and presenting the information to visitors visually and audibly,

[0895] A means of receiving visitor input and recording the feedback received,

[0896] A system that includes means for analyzing recorded feedback and using a generative AI model to learn how to optimize future suggestions.

[0897] (Claim 2)

[0898] The system according to claim 1, wherein when a visitor interacts with a display device, it accepts input using both voice and touch operations, and reflects the history of the interaction in the operation of the system.

[0899] (Claim 3)

[0900] The system according to claim 1, which combines the visitor's current location information and historical data, performs calculations to make adaptively optimal suggestions in real time, and dynamically updates the content of the suggestions presented.

[0901] "Example 2 of combining an emotion engine"

[0902] (Claim 1)

[0903] A means of receiving visitor identification information,

[0904] A means of accessing a behavioral history database based on the received identification information and extracting the relevant history information,

[0905] A means of analyzing extracted historical information and using an emotion analysis engine to recognize the visitor's emotional state,

[0906] A means for comprehensively evaluating recognized emotional states and historical information to generate personalized information suggestions for visitors,

[0907] A means of visually presenting the generated information proposal to visitors using a display means,

[0908] A means of receiving visitor input, obtaining and recording feedback information,

[0909] A system that includes means for analyzing recorded feedback information and performing learning to optimize the generation of information suggestions.

[0910] (Claim 2)

[0911] The system according to claim 1, which uses an avatar as a display means to provide information while adjusting the tone and facial expressions according to the emotional state of the visitor.

[0912] (Claim 3)

[0913] The system according to claim 1, which performs calculations to suggest a suitable experience in real time based on the emotional state of the visitor.

[0914] "Application example 2 when combining with an emotional engine"

[0915] (Claim 1)

[0916] A means of receiving visitor identification information,

[0917] A means for accessing a set of visitor behavior history data based on received identification information and extracting the relevant history information,

[0918] A means for analyzing extracted historical information to generate personalized information suggestions for visitors,

[0919] A means of outputting the generated proposal to a display device and presenting the information to visitors,

[0920] A means of receiving visitor input and recording the feedback received,

[0921] A means of analyzing recorded feedback and learning to optimize future suggestions,

[0922] A means of recognizing the emotional state of visitors and adjusting information suggestions based on those emotions,

[0923] A means of visually presenting recommended products and promotional information tailored to the visitor's emotional state through a visual display device worn by the visitor,

[0924] A system that includes this.

[0925] (Claim 2)

[0926] The system according to claim 1, which detects when a visitor enters or leaves a dynamic display area and reflects that information in the operation of the system.

[0927] (Claim 3)

[0928] The system according to claim 1, which performs processing to make optimal real-time suggestions, taking into account the visitor's current location information and schedule information. [Explanation of symbols]

[0929] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of receiving visitor identification information, A means for accessing a visitor's behavior history database based on the received identification information and extracting the relevant history data, A means of analyzing extracted historical data to generate personalized information suggestions for visitors, A means for outputting the generated proposal to a display device and presenting the information to visitors visually and audibly, A means of receiving visitor input and recording the feedback received, A system that includes means for analyzing recorded feedback and using a generative AI model to learn how to optimize future suggestions.

2. The system according to claim 1, wherein when a visitor interacts with a display device, it accepts input using both voice and touch operations, and reflects the history of the interaction in the operation of the system.

3. The system according to claim 1, which combines the visitor's current location information and historical data, performs calculations to make adaptively optimal suggestions in real time, and dynamically updates the content of the suggestions presented.

Images