system

Abstract

In large domes and stadiums, this system integrates advanced information systems, user interfaces, and real-time data to ensure that visitors are properly guided to their destinations (seats, restaurants, shops, etc.), can understand congestion levels, receive information on products and events tailored to their individual needs, and, in the face of the increasing adoption of unmanned stores, enable efficient inventory management and payment processing. [Solution] A system including a digital display device and means for the user to input a desired destination; a central control unit and means for receiving destination information from the digital display device and calculating the optimal route; means for providing the user with optimal route guidance using a geographic information system including real-time congestion information; and means for communication with the central control unit.

Description

Technical Field

[0004] , ,

[0005] , , ,

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In conventional large domes and stadiums, there were problems such as visitors not being properly guided to their destinations (seats, restaurants, shops, etc.), not being able to grasp the congestion situation, and not being able to provide information on products and events according to individual needs. Also, with the progress of the introduction of unmanned stores, efficient inventory management and settlement processing have become issues. To solve these problems, an advanced information system, user interface, and integration of real-time data are required. ​​​​​This invention provides a system that enhances the user experience through the linkage of a digital display device, a central control unit, and real-time data. Specifically, it includes a digital display device in which the user inputs their desired destination, a central control unit that receives destination information and calculates the optimal route, and means for providing optimal route guidance using a geographic information system that includes real-time congestion information. Furthermore, this system also includes a generative artificial intelligence that provides recommended product information based on the user's hobbies and preferences, a function to automatically display event announcements on the digital display device, and a function to scan products and process payments in unmanned stores. This makes it possible to provide efficient and personalized guidance and services in large domes and stadiums.

[0006] A "digital display device" is an electronically controlled device that receives information input by a user and displays various guidance information based on that information.

[0007] A "central control unit" is a control device that centrally manages data from multiple devices and systems, and processes and provides various types of information based on that data.

[0008] "Real-time congestion information" is data that shows the current congestion status of a location and route, and is updated in real time.

[0009] A Geographic Information System (GIS) is an information system used to collect, manage, and analyze information about geographical locations.

[0010] "Route guidance" is a service that provides users with the optimal route and steps to reach their destination.

[0011] "Generative artificial intelligence" is an artificial intelligence technology that analyzes past data and user behavior history to generate new information and content based on predictions.

[0012] An "unmanned store" is a store where goods are purchased and services are provided without the presence of store staff.

[0013] "Inventory information" refers to data that shows the current inventory status of a product.

[0014] "Payment processing" refers to a series of procedures necessary to complete the payment for a product.

[0015] "Event announcement" refers to notifying users of information about specific events or activities.

[0016] A "user" is an individual or group that uses this system to receive guidance or services. [Brief explanation of the drawing]

[0017] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Embodiment 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Embodiment 2 when the emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when the emotion engine is combined.

Mode for Carrying Out the Invention

[0018] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0021] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0022] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0023] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0025] [First Embodiment]

[0026] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0027] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0030] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0033] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0037] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0038] This invention is a system for efficiently guiding users to their desired destinations, recommending products, and announcing events. Specific embodiments for implementing this invention are described below.

[0039] System Overview

[0040] The system of the present invention provides users with optimal information by linking a digital display device, a central control unit, and real-time data. It includes a digital display device for users to select destinations and services, a central control unit that calculates the optimal route and recommended products based on the received information, and a system that manages congestion and inventory status in real time.

[0041] System processing and specific examples

[0042] 1. Route guidance processing

[0043] The user enters their desired seat number and destination on a digital display.

[0044] The terminal transmits that information to the central control unit.

[0045] The server (central control unit) refers to real-time congestion information and calculates the optimal route.

[0046] The terminal displays the calculated route to the user.

[0047] For example, if a user wants to know the location of "seat 102," they would enter the seat number "102" into the terminal. Based on congestion information, the central control unit sends specific route guidance to the terminal, such as "turn left down aisle B, go up the stairs, and turn right." The terminal then displays this information to the user as a map or text.

[0048] 2. Recommended Products

[0049] The user selects "recommended items" on a digital display device.

[0050] The terminal sends the request to the central control unit.

[0051] The server (central control unit) uses generative artificial intelligence to analyze the user's past purchase history and inventory information, and generates an optimal list of recommended products.

[0052] The device displays that information to the user.

[0053] For example, if a user enters "I want to know about recommended products" into the terminal, the server analyzes past purchase data and generates information such as, "This T-shirt is popular. You can purchase it at the unmanned store in Section A." The terminal then provides this information to the user.

[0054] 3. Event Announcement

[0055] The server constantly manages the latest event schedule and sends updated information to the digital display device.

[0056] The device automatically displays event information when the user is nearby or at a specific time.

[0057] As a concrete example, when a user approaches the device, the device will display a pop-up message saying, "A live event will begin at 3 PM on Stage C."

[0058] 4. Purchase processing at unmanned stores

[0059] Users select products in an unmanned store and scan the product barcodes using a digital display.

[0060] The terminal transmits barcode information to the central control unit to verify inventory information.

[0061] The user selects a payment method and enters their payment information.

[0062] The terminal sends payment information to the server and processes the payment.

[0063] The server updates the inventory and sends purchase completion information to the terminal.

[0064] The device displays a receipt and a purchase confirmation message to the user.

[0065] As a concrete example, a user selects a T-shirt at an unmanned store and scans the barcode at a terminal. The server checks the inventory information, and the user pays with a credit card. Once the payment is complete, a receipt is displayed on the terminal.

[0066] Integration of each component

[0067] The system of this invention improves user convenience through the close coordination of a digital display device, a central control unit, and real-time data. The central control unit centrally manages all data and updates information in real time, ensuring that users always receive the latest and most accurate information. Furthermore, by utilizing generative artificial intelligence, it is possible to provide users with personalized recommendation services.

[0068] As described above, the present invention provides an efficient information guidance system for realizing a comfortable user experience in large domes and stadiums.

[0069] The following describes the processing flow.

[0070] Route guidance processing

[0071] Step 1:

[0072] The user enters their destination (e.g., seat number "102") into the digital display.

[0073] Step 2:

[0074] The terminal receives the entered destination information and transmits it to the central control unit.

[0075] Step 3:

[0076] The server (central control unit) receives destination information and, simultaneously, calculates the optimal route by referencing real-time congestion information and a geographic information system (GIS).

[0077] Step 4:

[0078] The server transmits the calculated optimal route information to a digital display device.

[0079] Step 5:

[0080] The terminal displays the received optimal route information on the user interface and provides visual or audio guidance to the user.

[0081] Recommended Products

[0082] Step 1:

[0083] The user selects "Recommended Items" from the menu on the digital display device.

[0084] Step 2:

[0085] The terminal sends this request to the central control unit.

[0086] Step 3:

[0087] The server (central control unit) retrieves the user's past purchase history and current inventory information, and uses generative artificial intelligence to generate a list of recommended products best suited to the user.

[0088] Step 4:

[0089] The server sends the generated list of recommended items to the device.

[0090] Step 5:

[0091] The terminal displays the received product information on its user interface, visually presenting recommended products to the user.

[0092] Event Announcement

[0093] Step 1:

[0094] The server constantly manages the latest event information and sends updated information to the digital display device.

[0095] Step 2:

[0096] The device receives event information in real time and automatically sends event notifications based on specific times and conditions (e.g., when a user is near the device).

[0097] Step 3:

[0098] Users can view event information displayed on their devices and obtain detailed information to participate in events that interest them.

[0099] Purchase processing at unmanned stores

[0100] Step 1:

[0101] The user selects the items they want to purchase at the unmanned store and scans the barcodes of the items using a digital display device.

[0102] Step 2:

[0103] The terminal transmits barcode information to the central control unit to verify product inventory information.

[0104] Step 3:

[0105] The server (central control unit) updates the inventory status of products based on inventory information and prepares for payment processing.

[0106] Step 4:

[0107] The user selects a payment method and enters payment information, such as credit card details, into the terminal.

[0108] Step 5:

[0109] The terminal sends payment information to the server and executes the payment process.

[0110] Step 6:

[0111] The server confirms the completion of the payment and sends the purchase completion information to the terminal.

[0112] Step 7:

[0113] The device displays a purchase completion message and receipt to the user.

[0114] (Example 1)

[0115] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0116] Traditional navigation systems make it difficult for users to obtain the latest information about their destinations and recommended products. Furthermore, they suffer from inefficiencies in unmanned store purchasing processes and event announcements, resulting in a poor user experience.

[0117] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0118] In this invention, the server includes a digital display device including means for the user to input a destination, a central control device including means for receiving destination information from the digital display device and calculating the optimal route, and means for providing the user with optimal route guidance using a geographic information system including real-time congestion information. This makes it possible for the user to receive optimal route guidance based on real-time information.

[0119] Furthermore, the system includes means for analyzing the user's past purchase history and inventory information using generative artificial intelligence to generate recommended products, means for transmitting information based on the user's request to a central control unit, a digital display device including means for scanning product barcodes in an unmanned store and processing payments, means for transmitting payment information to the central control unit and processing settlements, and means including a central control unit that updates inventory information and notifies the user of purchase completion. This streamlines the user's purchasing process and provides a comfortable shopping experience.

[0120] Furthermore, the system includes means for automatically displaying event information when the user is nearby or at a specific time, and means for transmitting event schedule information to a digital display device. This ensures that users always have access to the latest event information, improving the convenience of event participation.

[0121] A "digital display device" is an electronic device that allows users to input destination and request information and then displays the results.

[0122] A "central control unit" is a computer system that processes data received from digital display devices and calculates necessary information.

[0123] "Real-time congestion information" refers to data that captures the current congestion situation in real time and provides it in an analyzable format.

[0124] A "Geographic Information System" is a system that manages and analyzes geographical data and provides functions such as route guidance and map display.

[0125] "Generative artificial intelligence" is an artificial intelligence technology that analyzes a user's past data to generate optimal information and recommended products.

[0126] "Barcode scanning" is the act of reading information from a product's barcode using an optical scanner.

[0127] "Payment processing" refers to the procedure for settling the payment for purchased goods.

[0128] "Event schedule information" refers to the timetable and detailed information of an event that takes place at a specific time and place.

[0129] An "unmanned store" is a store where there are no staff members present, and purchases and payments are handled through an automated system.

[0130] A "user" is someone who uses this system to receive services such as directions to a destination or the purchase of goods.

[0131] This invention is a system that efficiently provides users with destination guidance, recommended products, and event information. This system has a structure primarily consisting of a digital display device, a central control unit, and a real-time data linkage system, providing users with optimal information.

[0132] System Overview

[0133] The system consists of roughly three components. First, the user inputs destination and request information through a digital display device. Second, a central control unit processes the received information and calculates the optimal route and recommended products. Third, this information is displayed to the user through a terminal.

[0134] Hardware and software

[0135] This system utilizes the following key hardware and software components. Each element works in conjunction to ensure smooth information processing.

[0136] Digital display devices: These are terminals used by users to input destination and request information and display the results. Specifically, they include touch panel displays and barcode scanners.

[0137] Central Control Unit: This server analyzes received information and generates optimal routes and recommended products based on real-time data. This unit includes a generative artificial intelligence engine and a geographic information system (GIS).

[0138] Real-time data integration: This is a software interface for acquiring real-time congestion and inventory status data and using it for analysis.

[0139] System Processing Overview

[0140] Route guidance

[0141] The user enters their desired seat number and destination on a digital display. The entered information is transmitted to a central control unit via the terminal, and the server calculates the optimal route using real-time congestion information. The results are displayed on the terminal in map or text format.

[0142] Specific example: The user enters "seat 102," the server calculates a route such as "turn left down aisle B, go up the stairs, and proceed to the right," and the terminal displays this route to the user on a map.

[0143] Example of a prompt:

[0144] "Please tell me the way to seat 102."

[0145] Recommended Products

[0146] When a user selects a "recommended item" on a digital display, the terminal sends a request to a central control unit. The server uses generative artificial intelligence to analyze the user's past purchase history and inventory information to generate the most suitable recommended products. This information is then displayed to the user via the terminal.

[0147] Specific example: When a user enters "I want to know about recommended items," the server generates information such as "This T-shirt is popular. You can purchase it at the unmanned store in Section A," and the terminal displays this information to the user.

[0148] Example of a prompt:

[0149] "Please tell me your recommended products."

[0150] Event Announcement

[0151] The server manages the latest event schedule information and periodically transmits it to digital display devices. The terminal automatically displays event information when a user approaches or at specific times.

[0152] Specific example: When a user approaches the device, a pop-up appears saying, "The live event starts at 3 PM on Stage C."

[0153] Example of a prompt:

[0154] "Please tell me what events are happening at this time."

[0155] Purchase processing at unmanned stores

[0156] The user selects an item in an unmanned store and scans the barcode on a digital display. The barcode information is sent from the terminal to a central control unit, where inventory information is verified. The user selects a payment method and enters payment information, and the terminal sends this information to a server for payment processing. After payment is complete, the inventory information is updated, and a message indicating that the purchase is complete is displayed on the terminal.

[0157] Example: A user selects a T-shirt at an unmanned store, scans it, and pays with a credit card. A receipt is displayed after the payment is completed.

[0158] Example of a prompt:

[0159] "I would like to purchase this item, so I will scan the barcode."

[0160] In this way, each processing step works in conjunction, allowing users to receive optimized information in real time. The system achieves an efficient and comfortable user experience by closely coordinating digital display devices, a central control unit, and real-time data.

[0161] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0162] Route guidance processing

[0163] Step 1:

[0164] The user enters their desired destination (such as seat number) into the digital display device.

[0165] Input: Enter destination information, such as seat number "102," into the digital display.

[0166] Output: The user's input is registered in the system.

[0167] Step 2:

[0168] The terminal transmits the entered destination information to the central control unit (server).

[0169] Input: Data such as the seat number "102" entered by the user.

[0170] Output: Destination information is sent to the server.

[0171] Step 3:

[0172] The server refers to real-time congestion information and calculates the optimal route.

[0173] Input: Destination information and real-time congestion information.

[0174] Data processing / calculation: The server analyzes congestion information and applies an algorithm to calculate the optimal route.

[0175] Output: Calculated optimal route (e.g., turn left at passage B, go up the stairs and proceed to the right).

[0176] Step 4:

[0177] The terminal displays the calculated route information to the user.

[0178] Input: Optimal route information sent from the server.

[0179] Output: Route displayed on the device in map or text format.

[0180] ---

[0181] Recommended Products

[0182] Step 1:

[0183] The user selects "recommended items" on a digital display device.

[0184] Input: The user selects a "recommended item".

[0185] Output: Registered as a request in the system.

[0186] Step 2:

[0187] The terminal sends the request to the central control unit (server).

[0188] Input: User request information.

[0189] Output: Request information is sent to the server.

[0190] Step 3:

[0191] The server uses generative artificial intelligence to analyze the user's past purchase history and inventory information, and generates a list of optimal recommended products.

[0192] Input: User's past purchase history and inventory information.

[0193] Data processing / calculation: The server uses generative artificial intelligence to analyze this data and select the most suitable product.

[0194] Output: Recommended product list (e.g., "This T-shirt is popular").

[0195] Step 4:

[0196] The device displays a generated list of recommended products to the user.

[0197] Input: A list of recommended products sent from the server.

[0198] Output: The product list is displayed on the terminal.

[0199] ---

[0200] Event Announcement

[0201] Step 1:

[0202] The server constantly manages the latest event schedule and sends updated information to the digital display device.

[0203] Input: Latest event schedule information.

[0204] Output: Update information is sent to the digital display device.

[0205] Step 2:

[0206] The device automatically displays event information when the user is nearby or at a specific time.

[0207] Input: User proximity information, specific time.

[0208] Output: Pop-up display of event information (e.g., "The live event will start at Stage C at 3 PM").

[0209] ---

[0210] Purchase processing at unmanned stores

[0211] Step 1:

[0212] Users select products in an unmanned store and scan the product barcodes using a digital display.

[0213] Input: Product barcode information.

[0214] Output: Barcode information is registered in the system.

[0215] Step 2:

[0216] The terminal transmits barcode information to the central control unit (server) to verify inventory information.

[0217] Input: Barcode information.

[0218] Output: Inventory information is sent to the server.

[0219] Step 3:

[0220] The user selects a payment method and enters their payment information.

[0221] Input: User's payment information.

[0222] Output: Payment information is registered in the system.

[0223] Step 4:

[0224] The terminal sends payment information to the server and processes the payment.

[0225] Input: User's payment information.

[0226] Output: The payment result is sent to the server.

[0227] Step 5:

[0228] The server updates inventory information and sends purchase completion information to the terminal.

[0229] Input: Payment completion information.

[0230] Data processing / calculation: Updating inventory information.

[0231] Output: Purchase completion information is sent to the device.

[0232] Step 6:

[0233] The device displays a purchase confirmation message and receipt to the user.

[0234] Input: Purchase completion information.

[0235] Output: The receipt and purchase confirmation message will be displayed on the device.

[0236] (Application Example 1)

[0237] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0238] This invention relates to a system that provides a comfortable shopping experience for users in physical stores. In conventional physical stores, users often experience inconvenience due to the time it takes to find the desired product and to navigate the store while considering the crowd situation. Furthermore, there is a lack of recommended product information based on users' tastes and preferences, as well as timely event announcements. There is a desire to solve these problems and provide users with a comfortable and efficient shopping experience.

[0239] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0240] In this invention, the server includes a digital display device and means for the user to input their desired destination; a central control unit and means for receiving destination information from the digital display device and calculating the optimal route; means for providing the user with optimal route guidance using a geographic information system including real-time congestion information; and means for communicating with the central control unit and utilizing artificial intelligence to analyze the user's purchase history and inventory information and provide the user with optimal product recommendations; means for notifying the user of the latest event information in real time; and means for scanning product barcodes using the digital display device and processing payments. As a result, the user can efficiently move to their destination within the store, receive timely information on recommended products and events, and process payments quickly in unmanned stores.

[0241] A "digital display device" is a device equipped with an electronic display for users to input or receive information.

[0242] A "central control unit" is a device that oversees the operation of the entire system and calculates and provides the optimal route and recommended products based on information received from digital display devices.

[0243] "Real-time congestion information" refers to information that allows users to instantly grasp the current congestion situation and provide them with the most suitable route guidance.

[0244] A "Geographic Information System" is a system that manages and analyzes geospatial data and provides geographical information to users.

[0245] "Artificial intelligence" is a technology that uses machine learning and neural networks to analyze data and make decisions and predictions.

[0246] "User purchase history" refers to the historical data of products that a user has purchased in the past.

[0247] "Inventory information" refers to data about the quantity and status of products currently available in a store.

[0248] "Recommended product information" refers to providing information that recommends specific products based on the user's tastes, preferences, and purchase history.

[0249] "Event announcement" refers to informing users of the schedule and details of events taking place within the store.

[0250] An "unmanned store" is a store where there are no staff members present, and users can purchase goods automatically.

[0251] A "product barcode" is a barcode attached to a product that is used for product identification and inventory management.

[0252] "Payment processing" refers to the process of carrying out the necessary settlement procedures when purchasing goods.

[0253] The present invention provides a system that enables a comfortable shopping experience for users in physical stores. This system includes a digital display device, a central control unit, a geographic information system, a real-time data processing system, and generative artificial intelligence (AI). Details of embodiments are described below.

[0254] System Overview

[0255] The system allows users to input their desired destination using a digital display. A central control unit calculates the optimal route based on the destination information received from the digital display. Real-time congestion information is used in the route calculation to provide users with the most suitable route guidance. Furthermore, the system uses artificial intelligence to analyze the user's purchase history and inventory information, providing real-time notifications of optimal product recommendations and the latest event information.

[0256] Hardware and software to be used

[0257] Digital display devices: Smartphones, tablets, touch panel displays installed in stores, etc.

[0258] Central control unit: Servers, cloud computing environment (e.g., AWS®, Google® Cloud)

[0259] Geographic Information System: GIS software for managing map data (e.g., ArcGIS)

[0260] Real-time data processing system: Real-time data processing using Node.js and WebSocket

[0261] Generative artificial intelligence: AI models using TENSORFLOW® and PyTorch implemented in Python.

[0262] Data processing and data calculation

[0263] The server receives destination information transmitted from digital display devices and calculates the optimal route based on real-time congestion information. The Dijkstra algorithm is used for route calculation. Generative artificial intelligence analyzes the user's past purchase history and current inventory information to generate a personalized list of recommended products. Additionally, event information is retrieved from the database in real time and notified to the user based on their location and time.

[0264] For example, if a user enters "I'm looking for a silver ring" into the digital display, the server calculates the optimal route considering real-time congestion information and provides guidance such as, "Go straight from the entrance and proceed to the jewelry section on your right." Furthermore, based on past purchase history and inventory information, it also displays recommended product information such as, "New silver rings are on sale at a special price."

[0265] Example of a prompt

[0266] As an example of a prompt message that takes into account the user's interests, preferences, and behavioral history, the following data is generated:

[0267] "User ID: 12345, Current Location: Entrance, Destination: Jewelry Section, Past Purchase History: Necklace, Bracelet, Inventory Information: 15 'Silver Rings', Event Information: Next promotion starts at 3pm"

[0268] As described above, this system allows users to efficiently navigate to their destinations within the store, receive timely information on recommended products and events, and process payments quickly even in unmanned stores.

[0269] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0270] Step 1:

[0271] The user enters their desired destination into a digital display.

[0272] Input: Destination information input by the user to the digital display device.

[0273] Specific operation: The user inputs a destination such as "Jewelry section" to the touch screen or smartphone.

[0274] Output: The destination information is transmitted from the digital display device to the central control device.

[0275] Step 2:

[0276] The server receives the destination information from the digital display device and obtains the current congestion information.

[0277] Input: Destination information transmitted from the digital display device.

[0278] Specific operation: The server receives the destination information and obtains the congestion information updated in real time from the internal database.

[0279] Output: The destination information and the congestion information are stored in the server.

[0280] Step 3:

[0281] The server calculates the optimal route based on the destination information and the congestion information. …

[0282] Input: Destination information, congestion information. [[ID=…

[0283] Specific operation: The server uses the Dijkstra algorithm to calculate the optimal route considering the current congestion situation.

[0284] Output: The optimal route information is generated.

[0285] Step 4:

[0286] The server sends the calculated optimal route to the digital display device.

[0287] Input: Optimal route information.

[0288] Specific operation: The server generates a data packet for sending the optimal route to the digital display device and sends it using HTTPS communication.

[0289] Output: The optimal route information is displayed on the digital display device.

[0290] Step 5:

[0291] The user checks the optimal route displayed on the digital display device and starts moving.

[0292] Input: Optimal route information displayed on the digital display device.

[0293] Specific operation: The user checks the route guidance displayed on the smartphone or tablet and moves according to it.

[0294] Output: The user moves towards the destination.

[0295] Step 6:

[0296] The server analyzes the user's purchase history and inventory information and generates a list of recommended products.

[0297] Input: The user's purchase history data, current inventory information.

[0298] Specific operation: The server uses generative artificial intelligence to analyze the purchase history and inventory information and lists the optimal recommended products for the user. Input the prompt text into the AI model to obtain the results.

[0299] Output: A list of recommended products is generated.

[0300] Step 7:

[0301] The server sends a list of recommended products to the digital display device.

[0302] Input: Recommended product list.

[0303] Specific operation: Generate a data packet for sending the recommended product list generated by the server to the digital display device and send it using HTTPS communication.

[0304] Output: Recommended product information is displayed on the digital display device.

[0305] Step 8:

[0306] The server retrieves the latest event information from the database and notifies based on the user's location information.

[0307] Input: Event information database, user's location information.

[0308] Specific operation: The server extracts the latest event information from the database and generates appropriate event notifications based on the user's current location.

[0309] Output: Event notification information is generated.

[0310] Step 9:

[0311] The server sends the generated event notification information to the digital display device.

[0312] Input: Event notification information.

[0313] Specific operation: The server generates a data packet for sending the event notification information to the digital display device and sends it using HTTPS communication.

[0314] Output: Event notifications are displayed on the digital display device.

[0315] Step 10:

[0316] Users select products in an unmanned store and scan the product barcodes on a digital display.

[0317] Input: Product barcode information.

[0318] Specific operation: Scan the barcode of the product selected by the user onto the digital display device.

[0319] Output: Product barcode information is transmitted from the digital display device to the server.

[0320] Step 11:

[0321] The server receives the barcode information and verifies the inventory status.

[0322] Input: Product barcode information.

[0323] Specific operation: The server receives barcode information and retrieves inventory information for the corresponding product from its internal database.

[0324] Output: Inventory confirmation information is generated.

[0325] Step 12:

[0326] The user selects a payment method and enters payment information into the digital display.

[0327] Input: Payment information.

[0328] Specific operation: The user selects a payment method such as credit card or electronic money and enters the necessary information into the digital display device.

[0329] Output: Payment information is sent from the digital display device to the server.

[0330] Step 13:

[0331] The server processes the payment based on the payment information it receives.

[0332] Input: Payment information.

[0333] Specific operation: The server sends payment information to the payment system (e.g., Stripe or PayPal API) and processes the payment.

[0334] Output: Payment completion information is generated.

[0335] Step 14:

[0336] The server updates the inventory and notifies the digital display device of the purchase completion.

[0337] Input: Payment completion information, inventory information.

[0338] Specific operation: The server updates the inventory database and sends the purchase completion information to the digital display device.

[0339] Output: A purchase completion notification is displayed on the digital display.

[0340] These steps enable users to enjoy efficient and comfortable shopping at physical stores.

[0341] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0342] This invention provides a system that offers optimal information and services to users by linking a digital display device, a central control unit, real-time data, and an emotion engine. Specifically, it includes a digital display device for users to select destinations and services, a central control unit that calculates emotion recognition and optimal routes and recommended products based on received information, and a system that manages congestion status, inventory status, and the user's emotional state in real time.

[0343] System processing and specific examples

[0344] 1. Route guidance processing

[0345] The user enters their desired seat number and destination on a digital display.

[0346] The terminal transmits that information to the central control unit.

[0347] The server (central control unit) calculates the optimal route by referring to real-time congestion information and the user's emotional state.

[0348] The terminal displays the calculated route to the user.

[0349] For example, if a user wants to know the location of "seat 102," they enter the seat number into the terminal. The central control unit calculates a more comfortable route (for example, a route that avoids congestion) based on congestion information and the user's stress level, and sends it to the terminal. The terminal then displays that route to the user on a map or in text.

[0350] 2. Recommended Products

[0351] The user selects "recommended items" on a digital display device.

[0352] The terminal sends this request to the central control unit.

[0353] The server (central control unit) analyzes the user's past purchase history, current inventory information, and the user's emotional state as recognized by the emotion engine, and generates an optimal list of recommended products.

[0354] The device displays that information to the user.

[0355] As a concrete example, when a user enters "I want to know what products are recommended" into the terminal, the server analyzes past purchase data and sentiment data and presents "currently popular T-shirts" as products that are likely to please the user. The terminal then displays this information to the user.

[0356] 3. Event Announcement

[0357] The server constantly manages the latest event information and sends updated information to the digital display device.

[0358] The device receives event information in real time and automatically sends event notifications based on specific times and conditions (e.g., when a user is near the device).

[0359] As a specific example, when a user approaches a digital display device, the device detects the user's level of excitement from their facial expression and displays a pop-up message saying, "The live event starts at 3 PM on Stage C."

[0360] 4. Purchase processing at unmanned stores

[0361] The user selects the items they want to purchase at the unmanned store and scans the barcodes of the items using a digital display device.

[0362] The terminal transmits barcode information to the central control unit to verify product inventory information.

[0363] The server (central control unit) updates the inventory status of products based on inventory information and prepares for payment processing.

[0364] The user selects a payment method and enters payment information, such as credit card details, into the terminal.

[0365] The terminal sends payment information to the server and executes the payment process.

[0366] The server confirms the completion of the payment and sends the purchase completion information to the terminal.

[0367] The device displays a purchase completion message and receipt to the user.

[0368] As a concrete example, a user selects a T-shirt at an unmanned store and scans the barcode at a terminal. The server checks the inventory information, and the user pays with a credit card. Once the payment is complete, a receipt is displayed on the terminal.

[0369] 5. Utilizing the Emotional Engine

[0370] The device senses the user's facial expressions and voice, and analyzes them using an emotion engine.

[0371] The server receives information from the emotion engine and incorporates it into recommended products and route guidance information.

[0372] The device provides real-time alternatives and support information when the user expresses stress or dissatisfaction.

[0373] For example, when a user approaches a digital display device, the emotion engine detects stress from the user's facial expression, and the central control unit suggests less crowded routes or provides information about relaxation areas.

[0374] Integration of each component

[0375] The system of this invention enhances user convenience and experiential value through the close coordination of a digital display device, a central control unit, real-time data, and an emotion engine. The central control unit manages all data centrally and updates information in real time, ensuring that users always receive the latest and most accurate information. Furthermore, by utilizing the emotion engine, it is possible to provide user-optimized recommendation services.

[0376] As described above, the present invention provides an efficient information guidance system for realizing a comfortable user experience in large domes and stadiums.

[0377] The following describes the processing flow.

[0378] Route guidance processing combined with an emotion engine

[0379] Step 1:

[0380] The user enters their destination (e.g., seat number "102") into the digital display.

[0381] Step 2:

[0382] The terminal receives the entered destination information and transmits it to the central control unit.

[0383] Step 3:

[0384] The terminal analyzes the user's facial expressions and voice using an emotion engine and transmits the results to a central control unit.

[0385] Step 4:

[0386] The server (central control unit) receives destination information and sentiment information, while simultaneously referencing real-time congestion information and geographical information to calculate the optimal route.

[0387] Step 5:

[0388] The server transmits the calculated optimal route information to a digital display device.

[0389] Step 6:

[0390] The terminal displays the received optimal route information on the user interface and provides visual or audio guidance to the user.

[0391] A guide to recommended products that combine an emotional engine.

[0392] Step 1:

[0393] The user selects "Recommended Items" from the menu on the digital display device.

[0394] Step 2:

[0395] The terminal sends this request to the central control unit.

[0396] Step 3:

[0397] The terminal analyzes the user's facial expressions and voice using an emotion engine and transmits the results to a central control unit.

[0398] Step 4:

[0399] The server (central control unit) analyzes the user's past purchase history, inventory information, and the user's emotional state as recognized by the emotion engine, and generates an optimal list of recommended products.

[0400] Step 5:

[0401] The server sends the generated list of recommended items to a digital display device.

[0402] Step 6:

[0403] The terminal displays the received product information on its user interface, visually presenting recommended products to the user.

[0404] Event announcements combined with an emotional engine

[0405] Step 1:

[0406] The server constantly manages the latest event information and sends updated information to the digital display device.

[0407] Step 2:

[0408] The device receives event information in real time and activates the emotion engine based on specific times and conditions (e.g., when a user approaches the device).

[0409] Step 3:

[0410] The device analyzes the user's facial expressions and voice using an emotion engine and displays event notifications based on the results.

[0411] Step 4:

[0412] Users can view event information displayed on their devices and obtain detailed information to participate in events that interest them.

[0413] Purchase processing in unmanned stores using an emotion engine.

[0414] Step 1:

[0415] The user selects the items they want to purchase at the unmanned store and scans the barcodes of the items using a digital display device.

[0416] Step 2:

[0417] The terminal transmits barcode information to the central control unit to verify product inventory information.

[0418] Step 3:

[0419] The terminal analyzes the user's facial expressions and voice using an emotion engine and transmits the results to a central control unit.

[0420] Step 4:

[0421] The server (central control unit) updates the inventory status of products based on inventory information and prepares for payment processing.

[0422] Step 5:

[0423] The user selects a payment method and enters payment information, such as credit card details, into a digital display device.

[0424] Step 6:

[0425] The terminal transmits payment information to the central control unit and executes the payment process.

[0426] Step 7:

[0427] The server confirms the completion of the payment and sends the purchase completion information to the digital display device.

[0428] Step 8:

[0429] The device displays a purchase completion message and receipt to the user.

[0430] Real-time support utilizing an emotion engine

[0431] Step 1:

[0432] The device constantly analyzes the user's facial expressions and voice using an emotion engine to detect signs of stress and dissatisfaction.

[0433] Step 2:

[0434] The server receives information detected by the emotion engine and generates appropriate support information and alternatives in real time.

[0435] Step 3:

[0436] The device displays support information and alternatives tailored to the user's emotional state, and provides specific guidance.

[0437] This invention enables the realization of a system that analyzes the user's emotional state and provides optimal guidance and information based on that data.

[0438] (Example 2)

[0439] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0440] Conventional information guidance systems using digital display devices and central control units lack the ability to provide optimal information and services that take into account the user's emotional state, and there is a need to improve the user experience. Furthermore, it has been difficult to analyze the user's emotional state in real time and provide information and services accordingly, which has prevented a sufficient improvement in user convenience and satisfaction.

[0441] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0442] In this invention, the server includes means for using an emotion recognition engine that senses the user's facial expressions and voice using a digital display device and analyzes the user's emotional state; means for referencing the information from the emotion recognition engine in a central control unit and optimizing route guidance and product recommendations based on the user's emotional state; and means for providing recommended product information based on the user's hobbies and preferences. This enables the provision of optimal information and services that correspond to the user's emotional state.

[0443] A "digital display device" is an electronic device used by users to input information or to visually confirm information.

[0444] A "central control unit" is a key computer system that centrally manages data across the entire system, analyzes information, and provides optimal services.

[0445] An "emotion recognition engine" is software or hardware that analyzes a user's facial expressions and voice to determine the user's emotional state.

[0446] "Real-time congestion information" refers to the latest data showing the current congestion status of a specific location or route.

[0447] A "Geographic Information System" is a system that collects and analyzes data related to geospatial information and displays it visually.

[0448] "Generative artificial intelligence" is an artificial intelligence technology that analyzes large amounts of data and automatically generates useful information and recommendations.

[0449] "Recommended product information" refers to product information that is considered interesting to the user based on their hobbies, preferences, and past purchase history.

[0450] "Event schedule information" refers to data that includes detailed information such as the date, time, and location of various events.

[0451] An "unmanned store" is an automated store where there are no staff members present, and customers select products and complete the purchase process themselves.

[0452] A "generative AI model" is an artificial intelligence model that uses algorithms to generate useful patterns and predictions from large amounts of data.

[0453] A "prompt sentence" is an input sentence that prompts a generative AI model to generate a specific answer or information.

[0454] This invention relates to a system that provides users with optimal information and services by coordinating a digital display device, a central control unit, real-time data, and an emotion recognition engine. This system includes a digital display device for users to select destinations and services, a central control unit that calculates emotion recognition and optimal routes and recommended products based on received information, and a system that manages congestion levels, inventory status, and the user's emotional state in real time.

[0455] Route guidance processing

[0456] The digital display device provides an interface for users to input their desired seat number and destination. The terminal transmits the information entered by the user to the central control unit. The central control unit calculates the optimal route based on real-time congestion information and the user's emotional state, and transmits this information to the terminal. The terminal displays the calculated route to the user. Specifically, if a user wants to go to "seat 102," they enter that seat number into the terminal. The central control unit checks congestion information using Google Maps API and OpenStreetMap data and calculates a route that takes the user's stress level into consideration. The terminal then provides this route to the user in map or text format.

[0457] Recommended Products

[0458] When a user selects "recommended items" on a digital display, the terminal sends this request to a central control unit. The central control unit analyzes the user's past purchase history, current inventory information, and the user's emotional state provided by the emotion recognition engine to generate an optimal list of recommended items. The terminal then displays this information to the user. Specifically, if a user inputs "I want to know what recommended items are" into the terminal, the central control unit analyzes past purchase data and emotional data and recommends "popular T-shirts" as a product that is likely to please the user. The terminal then displays this information.

[0459] Event Announcement

[0460] The server constantly manages the latest event information and sends updates to the digital display device. The terminal receives event information in real time and automatically notifies the user of the event based on specific times and conditions (e.g., when a user approaches the terminal). For example, when a user approaches the digital display device, the terminal detects the user's level of excitement from their facial expression and displays a pop-up message saying, "The live event starts at 3 PM on Stage C."

[0461] Purchase processing at unmanned stores

[0462] The user selects the items they wish to purchase at the unmanned store and scans the barcodes of the items using a digital display. The terminal sends the barcode information to a central control unit to check the product's inventory. The central control unit updates the inventory information and prepares for payment processing. The user enters their credit card information into the terminal and selects a payment method, at which point the terminal sends the payment information to the central control unit for settlement processing. Once the payment is complete, the central control unit sends purchase completion information to the terminal, which then displays a purchase completion message and receipt.

[0463] Utilization of emotion recognition engines

[0464] The terminal senses the user's facial expressions and voice, and analyzes them using an emotion recognition engine. The central control unit receives information from the emotion recognition engine and incorporates it into recommended products and route guidance information. For example, if a user shows signs of stress when approaching a digital display, the central control unit provides information on less crowded routes and relaxation areas in real time.

[0465] Concrete examples of prompt sentences for generative AI models

[0466] 1. "If I want to get to seat 102, what's the best route?"

[0467] 2. "I'd like to know your recommended products. Please also consider my recent purchase history and stock availability."

[0468] 3. "Show me information about events happening nearby."

[0469] 4. "I want to buy this product. Please tell me the steps to scan the barcode and complete the payment."

[0470] 5. "Please suggest some activities or places to go when I'm feeling stressed."

[0471] This system, through the close cooperation of a digital display device, a central control unit, an emotion recognition engine, and a real-time data system, can improve the accuracy and convenience of information provided to users, thereby enhancing the user experience.

[0472] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0473] Route guidance processing

[0474] Step 1:

[0475] The user enters their destination (e.g., seat 102) into the digital display.

[0476] Input: Destination information (e.g., seat 102)

[0477] Output: Instruction to send input information to the terminal

[0478] Step 2:

[0479] The terminal transmits the destination information received from the user to the central control unit.

[0480] Input: Destination information entered by the user

[0481] Output: Transmission of destination information to the central control unit.

[0482] Operation: The terminal receives destination information entered by the user on the digital display device and communicates to transmit it to the central control unit.

[0483] Step 3:

[0484] Based on destination information received by the server (central control unit), the system analyzes real-time congestion information and emotion recognition engine data to calculate the optimal route.

[0485] Input: Destination information, real-time congestion information, emotion recognition engine data.

[0486] Output: Optimal route information

[0487] Operation: The central control unit obtains real-time congestion information from Google Maps API and OpenStreetMap, and calculates the optimal route to avoid congestion based on the user's emotional state provided by the emotion recognition engine.

[0488] Step 4:

[0489] The terminal displays the optimal route information received from the server to the user.

[0490] Input: Optimal route information from the server

[0491] Output: Optimal route information displayed to the user

[0492] Operation: The terminal displays the optimal route received from the server to the user in map or text format.

[0493] Recommended Products

[0494] Step 1:

[0495] The user selects "recommended items" on a digital display device.

[0496] Input: User selection information (e.g., "Recommended items")

[0497] Output: Instruction to send input information to the terminal

[0498] Step 2:

[0499] The terminal sends a request from the user to the central control unit.

[0500] Input: User's selected information

[0501] Output: Sending request information to the central control unit.

[0502] Operation: The terminal communicates to send the user's request for "recommended items" to the central control unit.

[0503] Step 3:

[0504] The server (central control unit) analyzes the user's past purchase history, current inventory information, and emotion recognition engine data based on the request information to generate an optimal list of recommended products.

[0505] Input: Request information, past purchase history, inventory information, emotion recognition engine data.

[0506] Output: Best Recommended Goods List

[0507] Operation: The central control unit retrieves the user's past purchase history from the database and selects and generates the most suitable products for the user based on current inventory information and data from the emotion recognition engine.

[0508] Step 4:

[0509] The device displays a list of recommended items received from the server to the user.

[0510] Input: Recommended goods list from the server

[0511] Output: Recommended product list displayed to the user

[0512] Operation: The terminal displays a list of recommended items received from the server to the user.

[0513] Event Announcement

[0514] Step 1:

[0515] The server manages the latest event information and transmits it to a digital display device.

[0516] Input: Event Information Database

[0517] Output: Sending event information to a digital display device.

[0518] Operation: The server constantly manages the latest event information and sends necessary updates to the digital display device.

[0519] Step 2:

[0520] The device automatically displays event information based on specific conditions.

[0521] Input: Event information from the server, proximity sensor information

[0522] Output: Event information displayed to the user

[0523] Operation: The device automatically displays event information as a pop-up based on the user's proximity and other specific conditions.

[0524] Purchase processing at unmanned stores

[0525] Step 1:

[0526] Users select products in an unmanned store and scan the product barcodes using a digital display.

[0527] Input: Barcode information

[0528] Output: Instruction to send barcode information to the terminal

[0529] Step 2:

[0530] The terminal transmits barcode information to the central control unit.

[0531] Input: Barcode information

[0532] Output: Transmission of barcode information to the central control unit.

[0533] Operation: The terminal communicates to transmit barcode information from the user to the central control unit.

[0534] Step 3:

[0535] The server (central control unit) checks inventory information and updates the inventory status.

[0536] Input: Barcode information, inventory database

[0537] Output: Inventory update information

[0538] Operation: The central control unit retrieves inventory information from the product database and performs necessary updates.

[0539] Step 4:

[0540] The user selects a payment method and enters payment information, such as credit card details, into the terminal.

[0541] Input: Payment information (e.g., credit card information)

[0542] Output: Notification that payment information has been successfully entered into the terminal.

[0543] Step 5:

[0544] The terminal sends payment information to the server and executes the payment process.

[0545] Input: Payment Information

[0546] Output: Sending payment information to the server

[0547] Operation: The terminal transmits the user's payment information to the central control unit and processes the payment via the payment API.

[0548] Step 6:

[0549] The server confirms the completion of the payment and sends the purchase completion information to the terminal.

[0550] Input: Payment Information

[0551] Output: Sending purchase completion information

[0552] Operation: The central control unit confirms the completion of the payment and sends the purchase completion information to the terminal.

[0553] Step 7:

[0554] The device displays a purchase completion message and receipt to the user.

[0555] Input: Purchase completion information

[0556] Output: Purchase completion message and electronic receipt displayed to the user.

[0557] Operation: The device displays a purchase completion message and an electronic receipt.

[0558] Utilization of emotion recognition engines

[0559] Step 1:

[0560] The device detects the user's facial expressions and voice, and analyzes them using an emotion recognition engine.

[0561] Input: User facial expression data, voice data

[0562] Output: Analyzed emotional state

[0563] Operation: The device uses a camera and microphone to sense the user's facial expressions and voice, and analyzes them in real time using an emotion recognition engine.

[0564] Step 2:

[0565] The server (central control unit) receives information from the emotion recognition engine and incorporates it into the recommendations.

[0566] Input: Analyzed emotional state

[0567] Output: Optimized recommendations

[0568] Operation: The central control unit generates optimal product and route guidance information that corresponds to the user's emotional state, based on information from the emotion recognition engine.

[0569] Step 3:

[0570] The device displays recommendations based on the user's emotional state.

[0571] Input: Optimized recommendations

[0572] Output: Recommended information displayed to the user

[0573] Operation: The terminal displays recommended information received from the central control unit to the user. For example, if the user is feeling stressed, it will display information about less crowded routes or relaxation areas.

[0574] Concrete examples of prompt sentences for generative AI models

[0575] 1. "If I want to get to seat 102, what's the best route?"

[0576] 2. "I'd like to know your recommended products. Please also consider my recent purchase history and stock availability."

[0577] 3. "Show me information about events happening nearby."

[0578] 4. "I want to buy this product. Please tell me the steps to scan the barcode and complete the payment."

[0579] 5. "Please suggest some activities or places to go when I'm feeling stressed."

[0580] (Application Example 2)

[0581] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0582] Traditional store guidance systems did not provide route guidance or product recommendations that took into account congestion levels or the user's emotional state. As a result, users were forced to take crowded routes or had difficulty finding products that matched their emotional state. Furthermore, event announcements and information about relaxation areas were not provided in real time. These problems led to a decline in the user's shopping experience.

[0583] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0584] In this invention, the server includes a digital display device and means for the user to input their desired destination; a central control unit and means for receiving destination information from the digital display device and calculating the optimal route; and means for providing the user with optimal route guidance using a geographic information system that includes real-time congestion information. This enables the user to take a comfortable route that avoids congestion.

[0585] Furthermore, the system includes means for analyzing the user's emotional state using an emotion recognition engine and providing optimal product and route guidance, means for analyzing past purchase data using generative artificial intelligence and providing recommended product information, means for automatically notifying users of real-time event information under specific conditions, and means for displaying guidance information in the user's field of view using a smart device. This improves the user's shopping experience and enables real-time relaxation and guidance based on event information.

[0586] A "digital display device" is an electronic device used to input and display information such as the destination a user wants to go to or the product information they have selected.

[0587] A "central control unit" is a device that calculates the optimal route and products based on destination information received from digital display devices, while also referencing real-time congestion information and the user's emotional state.

[0588] "Real-time congestion information" is data that continuously updates the current congestion situation and is used to consider user travel routes and usage patterns within stores.

[0589] A "geographic information system" refers to a system that provides destination and route information in the form of maps, and may include real-time updated congestion information.

[0590] An "emotion recognition engine" is an artificial intelligence technology that analyzes a user's facial expressions and voice to recognize their emotional state (for example, stress or joy).

[0591] A "smart device" is a device with advanced functions, such as a smartphone or smart glasses, that is used to display information within the user's field of vision.

[0592] "Generative artificial intelligence" refers to artificial intelligence models that analyze past data and patterns to generate new information and recommendations.

[0593] "Recommended product information" refers to product recommendations for users based on their past purchase history and current emotional state.

[0594] "Event notification" is a function that notifies users of event information in real time based on a specific time and the user's location.

[0595] An "unmanned store" is a store that does not have any staff and uses digital technology to sell products and process payments.

[0596] This invention is a system that combines a digital display device, a central control unit, real-time data, and an emotion recognition engine to provide users with optimal information and services. The following configuration is adopted to implement this invention.

[0597] 1. System Configuration

[0598] The system consists of a digital display device, a central control unit, a geographic information system including real-time data, an emotion recognition engine, and smart devices (smartphones, smart glasses). This system provides users with destination guidance, product recommendations, event notifications, and emotion-based assistance.

[0599] 2. Digital display device

[0600] The digital display device allows users to search for destinations and products, and displays related information. When a user enters their desired destination into the digital display device, that information is transmitted to the central control unit.

[0601] 3. Central Control Unit

[0602] The central control unit receives destination information from the digital display device and calculates the optimal route based on real-time congestion information and data from the emotion recognition engine. Furthermore, it analyzes past purchase data and inventory information to generate recommended product information and transmits it to the digital display device.

[0603] 4. Geographic Information Systems Including Real-Time Data

[0604] The geographic information system provides geographic data, including real-time congestion information, which is used by a central control unit to provide optimal route guidance. This allows users to follow routes that avoid congestion.

[0605] 5. Emotion Recognition Engine

[0606] The emotion recognition engine analyzes the user's facial expressions and voice to identify their emotional state. This information is transmitted to a central control unit, which then suggests the most suitable products and services based on the user's emotional state.

[0607] 6. Smart devices

[0608] Smart devices (smartphones and smart glasses) are used to display guidance information directly within the user's field of vision. They also automatically display real-time event information and notify the user under specific conditions.

[0609] 7. Specific Examples and Prompts

[0610] Specific example

[0611] Assume a user is searching for products in a store. The user enters "Blue Jeans" into a digital display. This information is sent to a central control unit, which calculates the optimal route and a route that avoids congestion based on real-time congestion information and emotion recognition engine data. The results are displayed on the digital display or smart glasses, allowing the user to smoothly reach the location of Blue Jeans.

[0612] Example of a prompt

[0613] "A user searches for a product using a smartphone app. The user enters that they are looking for 'blue jeans.' Based on this information, generate a flow that suggests the optimal product location and the best route to avoid crowds."

[0614] This system will enable users to have a more comfortable and efficient shopping experience. Real-time information updates and emotion-based service delivery are also expected to improve user satisfaction.

[0615] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0616] Step 1:

[0617] The user enters destination and product information into a digital display device. The entered information is text data such as destination and product name. The digital display device then transmits the user's input to a central control unit.

[0618] Step 2:

[0619] The central control unit receives information transmitted from the user. Based on the received information, it communicates with the geographic information system, which includes real-time congestion information, to obtain current congestion status and route information. The acquired data is real-time congestion data from the geographic information system.

[0620] Step 3:

[0621] The central control unit works in conjunction with the emotion recognition engine, which analyzes the user's emotional state from facial expressions and voice data. The emotion recognition engine captures user data using hardware such as cameras and microphones and transmits the analysis results to the central control unit. The analysis results include the user's stress level and satisfaction level.

[0622] Step 4:

[0623] The central control unit calculates the optimal route based on received congestion information and emotional states. The calculation uses an algorithm that combines real-time congestion information and data from the emotion recognition engine to determine the most comfortable route for the user. The calculation result is the optimal route information.

[0624] Step 5:

[0625] The central control unit analyzes past purchase data and current inventory information, and uses generative artificial intelligence to list recommended products. The generative AI combines the user's past purchase history with inventory data to recommend products that the user is likely to be interested in. The analysis results in a list of recommended products.

[0626] Step 6:

[0627] The central control unit transmits optimal route information and a list of recommended products to digital display devices and smart devices. The receiving terminals then display this information to the user. Display methods include maps, text, and augmented reality (AR) displays.

[0628] Step 7:

[0629] When a user approaches a digital display device or smart device, the central control unit acquires real-time event information based on the location data and issues a notification under specific conditions. This notification is automatically generated based on the user's location data, and the displayed content is event notification information.

[0630] Step 8:

[0631] When a user scans an item in an unmanned store, the digital display device transmits the item's barcode information to a central control unit. The central control unit checks the inventory from the received barcode information and prepares for payment processing. The user selects a payment method and executes the payment process.

[0632] Step 9:

[0633] After payment is completed, the central control unit transmits purchase completion information to the digital display device, notifying the user of the fact. The display shows a purchase completion message and a receipt.

[0634] By following these steps, users can enjoy a comfortable and efficient shopping experience. Real-time information updates and emotion-based service delivery are expected to improve user satisfaction.

[0635] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0636] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0637] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0638] [Second Embodiment]

[0639] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0640] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0641] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0642] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0643] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0644] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0645] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0646] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0647] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0648] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0649] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0650] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0651] This invention is a system for efficiently guiding users to their desired destinations, recommending products, and announcing events. Specific embodiments for implementing this invention are described below.

[0652] System Overview

[0653] The system of the present invention provides users with optimal information by linking a digital display device, a central control unit, and real-time data. It includes a digital display device for users to select destinations and services, a central control unit that calculates the optimal route and recommended products based on the received information, and a system that manages congestion and inventory status in real time.

[0654] System processing and specific examples

[0655] 1. Route guidance processing

[0656] The user enters their desired seat number and destination on a digital display.

[0657] The terminal transmits that information to the central control unit.

[0658] The server (central control unit) refers to real-time congestion information and calculates the optimal route.

[0659] The terminal displays the calculated route to the user.

[0660] For example, if a user wants to know the location of "seat 102," they would enter the seat number "102" into the terminal. Based on congestion information, the central control unit sends specific route guidance to the terminal, such as "turn left down aisle B, go up the stairs, and turn right." The terminal then displays this information to the user as a map or text.

[0661] 2. Recommended Products

[0662] The user selects "recommended items" on a digital display device.

[0663] The terminal sends the request to the central control unit.

[0664] The server (central control unit) uses generative artificial intelligence to analyze the user's past purchase history and inventory information, and generates an optimal list of recommended products.

[0665] The device displays that information to the user.

[0666] For example, if a user enters "I want to know about recommended products" into the terminal, the server analyzes past purchase data and generates information such as, "This T-shirt is popular. You can purchase it at the unmanned store in Section A." The terminal then provides this information to the user.

[0667] 3. Event Announcement

[0668] The server constantly manages the latest event schedule and sends updated information to the digital display device.

[0669] The device automatically displays event information when the user is nearby or at a specific time.

[0670] As a concrete example, when a user approaches the device, the device will display a pop-up message saying, "A live event will begin at 3 PM on Stage C."

[0671] 4. Purchase processing at unmanned stores

[0672] Users select products in an unmanned store and scan the product barcodes using a digital display.

[0673] The terminal transmits barcode information to the central control unit to verify inventory information.

[0674] The user selects a payment method and enters their payment information.

[0675] The terminal sends payment information to the server and processes the payment.

[0676] The server updates the inventory and sends purchase completion information to the terminal.

[0677] The device displays a receipt and a purchase confirmation message to the user.

[0678] As a concrete example, a user selects a T-shirt at an unmanned store and scans the barcode at a terminal. The server checks the inventory information, and the user pays with a credit card. Once the payment is complete, a receipt is displayed on the terminal.

[0679] Integration of each component

[0680] The system of this invention improves user convenience through the close coordination of a digital display device, a central control unit, and real-time data. The central control unit centrally manages all data and updates information in real time, ensuring that users always receive the latest and most accurate information. Furthermore, by utilizing generative artificial intelligence, it is possible to provide users with personalized recommendation services.

[0681] As described above, the present invention provides an efficient information guidance system for realizing a comfortable user experience in large domes and stadiums.

[0682] The following describes the processing flow.

[0683] Route guidance processing

[0684] Step 1:

[0685] The user enters their destination (e.g., seat number "102") into the digital display.

[0686] Step 2:

[0687] The terminal receives the entered destination information and transmits it to the central control unit.

[0688] Step 3:

[0689] The server (central control unit) receives destination information and, simultaneously, calculates the optimal route by referencing real-time congestion information and a geographic information system (GIS).

[0690] Step 4:

[0691] The server transmits the calculated optimal route information to a digital display device.

[0692] Step 5:

[0693] The terminal displays the received optimal route information on the user interface and provides visual or audio guidance to the user.

[0694] Recommended Products

[0695] Step 1:

[0696] The user selects "Recommended Items" from the menu on the digital display device.

[0697] Step 2:

[0698] The terminal sends this request to the central control unit.

[0699] Step 3:

[0700] The server (central control unit) retrieves the user's past purchase history and current inventory information, and uses generative artificial intelligence to generate a list of recommended products best suited to the user.

[0701] Step 4:

[0702] The server sends the generated list of recommended items to the device.

[0703] Step 5:

[0704] The terminal displays the received product information on its user interface, visually presenting recommended products to the user.

[0705] Event Announcement

[0706] Step 1:

[0707] The server constantly manages the latest event information and sends updated information to the digital display device.

[0708] Step 2:

[0709] The device receives event information in real time and automatically sends event notifications based on specific times and conditions (e.g., when a user is near the device).

[0710] Step 3:

[0711] Users can view event information displayed on their devices and obtain detailed information to participate in events that interest them.

[0712] Purchase processing at unmanned stores

[0713] Step 1:

[0714] The user selects the items they want to purchase at the unmanned store and scans the barcodes of the items using a digital display device.

[0715] Step 2:

[0716] The terminal transmits barcode information to the central control unit to verify product inventory information.

[0717] Step 3:

[0718] The server (central control unit) updates the inventory status of products based on inventory information and prepares for payment processing.

[0719] Step 4:

[0720] The user selects a payment method and enters payment information, such as credit card details, into the terminal.

[0721] Step 5:

[0722] The terminal sends payment information to the server and executes the payment process.

[0723] Step 6:

[0724] The server confirms the completion of the payment and sends the purchase completion information to the terminal.

[0725] Step 7:

[0726] The device displays a purchase completion message and receipt to the user.

[0727] (Example 1)

[0728] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0729] Traditional navigation systems make it difficult for users to obtain the latest information about their destinations and recommended products. Furthermore, they suffer from inefficiencies in unmanned store purchasing processes and event announcements, resulting in a poor user experience.

[0730] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0731] In this invention, the server includes a digital display device including means for the user to input a destination, a central control device including means for receiving destination information from the digital display device and calculating the optimal route, and means for providing the user with optimal route guidance using a geographic information system including real-time congestion information. This makes it possible for the user to receive optimal route guidance based on real-time information.

[0732] Furthermore, the system includes means for analyzing the user's past purchase history and inventory information using generative artificial intelligence to generate recommended products, means for transmitting information based on the user's request to a central control unit, a digital display device including means for scanning product barcodes in an unmanned store and processing payments, means for transmitting payment information to the central control unit and processing settlements, and means including a central control unit that updates inventory information and notifies the user of purchase completion. This streamlines the user's purchasing process and provides a comfortable shopping experience.

[0733] Furthermore, the system includes means for automatically displaying event information when the user is nearby or at a specific time, and means for transmitting event schedule information to a digital display device. This ensures that users always have access to the latest event information, improving the convenience of event participation.

[0734] A "digital display device" is an electronic device that allows users to input destination and request information and then displays the results.

[0735] A "central control unit" is a computer system that processes data received from digital display devices and calculates necessary information.

[0736] "Real-time congestion information" refers to data that captures the current congestion situation in real time and provides it in an analyzable format.

[0737] A "Geographic Information System" is a system that manages and analyzes geographical data and provides functions such as route guidance and map display.

[0738] "Generative artificial intelligence" is an artificial intelligence technology that analyzes a user's past data to generate optimal information and recommended products.

[0739] "Barcode scanning" is the act of reading information from a product's barcode using an optical scanner.

[0740] "Payment processing" refers to the procedure for settling the payment for purchased goods.

[0741] "Event schedule information" refers to the timetable and detailed information of an event that takes place at a specific time and place.

[0742] An "unmanned store" is a store where there are no staff members present, and purchases and payments are handled through an automated system.

[0743] A "user" is someone who uses this system to receive services such as directions to a destination or the purchase of goods.

[0744] This invention is a system that efficiently provides users with destination guidance, recommended products, and event information. This system has a structure primarily consisting of a digital display device, a central control unit, and a real-time data linkage system, providing users with optimal information.

[0745] System Overview

[0746] The system consists of roughly three components. First, the user inputs destination and request information through a digital display device. Second, a central control unit processes the received information and calculates the optimal route and recommended products. Third, this information is displayed to the user through a terminal.

[0747] Hardware and software

[0748] This system utilizes the following key hardware and software components. Each element works in conjunction to ensure smooth information processing.

[0749] Digital display devices: These are terminals used by users to input destination and request information and display the results. Specifically, they include touch panel displays and barcode scanners.

[0750] Central Control Unit: This server analyzes received information and generates optimal routes and recommended products based on real-time data. This unit includes a generative artificial intelligence engine and a geographic information system (GIS).

[0751] Real-time data integration: This is a software interface for acquiring real-time congestion and inventory status data and using it for analysis.

[0752] System Processing Overview

[0753] Route guidance

[0754] The user enters their desired seat number and destination on a digital display. The entered information is transmitted to a central control unit via the terminal, and the server calculates the optimal route using real-time congestion information. The results are displayed on the terminal in map or text format.

[0755] Specific example: The user enters "seat 102," the server calculates a route such as "turn left down aisle B, go up the stairs, and proceed to the right," and the terminal displays this route to the user on a map.

[0756] Example of a prompt:

[0757] "Please tell me the way to seat 102."

[0758] Recommended Products

[0759] When a user selects a "recommended item" on a digital display, the terminal sends a request to a central control unit. The server uses generative artificial intelligence to analyze the user's past purchase history and inventory information to generate the most suitable recommended products. This information is then displayed to the user via the terminal.

[0760] Specific example: When a user enters "I want to know about recommended items," the server generates information such as "This T-shirt is popular. You can purchase it at the unmanned store in Section A," and the terminal displays this information to the user.

[0761] Example of a prompt:

[0762] "Please tell me your recommended products."

[0763] Event Announcement

[0764] The server manages the latest event schedule information and periodically transmits it to digital display devices. The terminal automatically displays event information when a user approaches or at specific times.

[0765] Specific example: When a user approaches the device, a pop-up appears saying, "The live event starts at 3 PM on Stage C."

[0766] Example of a prompt:

[0767] "Please tell me what events are happening at this time."

[0768] Purchase processing at unmanned stores

[0769] The user selects an item in an unmanned store and scans the barcode on a digital display. The barcode information is sent from the terminal to a central control unit, where inventory information is verified. The user selects a payment method and enters payment information, and the terminal sends this information to a server for payment processing. After payment is complete, the inventory information is updated, and a message indicating that the purchase is complete is displayed on the terminal.

[0770] Example: A user selects a T-shirt at an unmanned store, scans it, and pays with a credit card. A receipt is displayed after the payment is completed.

[0771] Example of a prompt:

[0772] "I would like to purchase this item, so I will scan the barcode."

[0773] In this way, each processing step works in conjunction, allowing users to receive optimized information in real time. The system achieves an efficient and comfortable user experience by closely coordinating digital display devices, a central control unit, and real-time data.

[0774] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0775] Route guidance processing

[0776] Step 1:

[0777] The user enters their desired destination (such as seat number) into the digital display device.

[0778] Input: Enter destination information, such as seat number "102," into the digital display.

[0779] Output: The user's input is registered in the system.

[0780] Step 2:

[0781] The terminal transmits the entered destination information to the central control unit (server).

[0782] Input: Data such as the seat number "102" entered by the user.

[0783] Output: Destination information is sent to the server.

[0784] Step 3:

[0785] The server refers to real-time congestion information and calculates the optimal route.

[0786] Input: Destination information and real-time congestion information.

[0787] Data processing / calculation: The server analyzes congestion information and applies an algorithm to calculate the optimal route.

[0788] Output: Calculated optimal route (e.g., turn left at passage B, go up the stairs and proceed to the right).

[0789] Step 4:

[0790] The terminal displays the calculated route information to the user.

[0791] Input: Optimal route information sent from the server.

[0792] Output: Route displayed on the device in map or text format.

[0793] ---

[0794] Recommended Products

[0795] Step 1:

[0796] The user selects "recommended items" on a digital display device.

[0797] Input: The user selects a "recommended item".

[0798] Output: Registered as a request in the system.

[0799] Step 2:

[0800] The terminal sends the request to the central control unit (server).

[0801] Input: User request information.

[0802] Output: Request information is sent to the server.

[0803] Step 3:

[0804] The server uses generative artificial intelligence to analyze the user's past purchase history and inventory information, and generates a list of optimal recommended products.

[0805] Input: User's past purchase history and inventory information.

[0806] Data processing / calculation: The server uses generative artificial intelligence to analyze this data and select the most suitable product.

[0807] Output: Recommended product list (e.g., "This T-shirt is popular").

[0808] Step 4:

[0809] The device displays a generated list of recommended products to the user.

[0810] Input: A list of recommended products sent from the server.

[0811] Output: The product list is displayed on the terminal.

[0812] ---

[0813] Event Announcement

[0814] Step 1:

[0815] The server constantly manages the latest event schedule and sends updated information to the digital display device.

[0816] Input: Latest event schedule information.

[0817] Output: Update information is sent to the digital display device.

[0818] Step 2:

[0819] The device automatically displays event information when the user is nearby or at a specific time.

[0820] Input: User proximity information, specific time.

[0821] Output: Pop-up display of event information (e.g., "The live event will start at Stage C at 3 PM").

[0822] ---

[0823] Purchase processing at unmanned stores

[0824] Step 1:

[0825] Users select products in an unmanned store and scan the product barcodes using a digital display.

[0826] Input: Product barcode information.

[0827] Output: Barcode information is registered in the system.

[0828] Step 2:

[0829] The terminal transmits barcode information to the central control unit (server) to verify inventory information.

[0830] Input: Barcode information.

[0831] Output: Inventory information is sent to the server.

[0832] Step 3:

[0833] The user selects a payment method and enters their payment information.

[0834] Input: User's payment information.

[0835] Output: Payment information is registered in the system.

[0836] Step 4:

[0837] The terminal sends payment information to the server and processes the payment.

[0838] Input: User's payment information.

[0839] Output: The payment result is sent to the server.

[0840] Step 5:

[0841] The server updates inventory information and sends purchase completion information to the terminal.

[0842] Input: Payment completion information.

[0843] Data processing / calculation: Updating inventory information.

[0844] Output: Purchase completion information is sent to the device.

[0845] Step 6:

[0846] The device displays a purchase confirmation message and receipt to the user.

[0847] Input: Purchase completion information.

[0848] Output: The receipt and purchase confirmation message will be displayed on the device.

[0849] (Application Example 1)

[0850] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0851] This invention relates to a system that provides a comfortable shopping experience for users in physical stores. In conventional physical stores, users often experience inconvenience due to the time it takes to find the desired product and to navigate the store while considering the crowd situation. Furthermore, there is a lack of recommended product information based on users' tastes and preferences, as well as timely event announcements. There is a desire to solve these problems and provide users with a comfortable and efficient shopping experience.

[0852] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0853] In this invention, the server includes a digital display device and means for the user to input their desired destination; a central control unit and means for receiving destination information from the digital display device and calculating the optimal route; means for providing the user with optimal route guidance using a geographic information system including real-time congestion information; and means for communicating with the central control unit and utilizing artificial intelligence to analyze the user's purchase history and inventory information and provide the user with optimal product recommendations; means for notifying the user of the latest event information in real time; and means for scanning product barcodes using the digital display device and processing payments. As a result, the user can efficiently move to their destination within the store, receive timely information on recommended products and events, and process payments quickly in unmanned stores.

[0854] A "digital display device" is a device equipped with an electronic display for users to input or receive information.

[0855] A "central control unit" is a device that oversees the operation of the entire system and calculates and provides the optimal route and recommended products based on information received from digital display devices.

[0856] "Real-time congestion information" refers to information that allows users to instantly grasp the current congestion situation and provide them with the most suitable route guidance.

[0857] A "Geographic Information System" is a system that manages and analyzes geospatial data and provides geographical information to users.

[0858] "Artificial intelligence" is a technology that uses machine learning and neural networks to analyze data and make decisions and predictions.

[0859] "User purchase history" refers to the historical data of products that a user has purchased in the past.

[0860] "Inventory information" refers to data about the quantity and status of products currently available in a store.

[0861] "Recommended product information" refers to providing information that recommends specific products based on the user's tastes, preferences, and purchase history.

[0862] "Event announcement" refers to informing users of the schedule and details of events taking place within the store.

[0863] An "unmanned store" is a store where there are no staff members present, and users can purchase goods automatically.

[0864] A "product barcode" is a barcode attached to a product that is used for product identification and inventory management.

[0865] "Payment processing" refers to the process of carrying out the necessary settlement procedures when purchasing goods.

[0866] The present invention provides a system that enables a comfortable shopping experience for users in physical stores. This system includes a digital display device, a central control unit, a geographic information system, a real-time data processing system, and generative artificial intelligence (AI). Details of embodiments are described below.

[0867] System Overview

[0868] The system allows users to input their desired destination using a digital display. A central control unit calculates the optimal route based on the destination information received from the digital display. Real-time congestion information is used in the route calculation to provide users with the most suitable route guidance. Furthermore, the system uses artificial intelligence to analyze the user's purchase history and inventory information, providing real-time notifications of optimal product recommendations and the latest event information.

[0869] Hardware and software to be used

[0870] Digital display devices: Smartphones, tablets, touch panel displays installed in stores, etc.

[0871] Central control unit: Servers, cloud computing environment (e.g., AWS, Google Cloud)

[0872] Geographic Information System: GIS software for managing map data (e.g., ArcGIS)

[0873] Real-time data processing system: Real-time data processing using Node.js and WebSocket

[0874] Generative artificial intelligence: AI models using TensorFlow and PyTorch implemented in Python

[0875] Data processing and data calculation

[0876] The server receives destination information transmitted from digital display devices and calculates the optimal route based on real-time congestion information. The Dijkstra algorithm is used for route calculation. Generative artificial intelligence analyzes the user's past purchase history and current inventory information to generate a personalized list of recommended products. Additionally, event information is retrieved from the database in real time and notified to the user based on their location and time.

[0877] For example, if a user enters "I'm looking for a silver ring" into the digital display, the server calculates the optimal route considering real-time congestion information and provides guidance such as, "Go straight from the entrance and proceed to the jewelry section on your right." Furthermore, based on past purchase history and inventory information, it also displays recommended product information such as, "New silver rings are on sale at a special price."

[0878] Example of a prompt

[0879] As an example of a prompt message that takes into account the user's interests, preferences, and behavioral history, the following data is generated:

[0880] "User ID: 12345, Current Location: Entrance, Destination: Jewelry Section, Past Purchase History: Necklace, Bracelet, Inventory Information: 15 'Silver Rings', Event Information: Next promotion starts at 3pm"

[0881] As described above, this system allows users to efficiently navigate to their destinations within the store, receive timely information on recommended products and events, and process payments quickly even in unmanned stores.

[0882] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0883] Step 1:

[0884] The user enters their desired destination into a digital display.

[0885] Input: Destination information entered by the user into the digital display device.

[0886] Specific operation: The user enters a destination, such as "Jewelry Section," on the touchscreen or smartphone.

[0887] Output: Destination information is transmitted from the digital display device to the central control unit.

[0888] Step 2:

[0889] The server receives destination information from a digital display device and obtains current congestion information.

[0890] Input: Destination information transmitted from a digital display device.

[0891] Specific operation: The server receives destination information and retrieves real-time updated congestion information from its internal database.

[0892] Output: Destination information and congestion information are stored on the server.

[0893] Step 3:

[0894] The server calculates the optimal route based on destination information and congestion information.

[0895] Input: Destination information, congestion information.

[0896] Specific operation: The server uses Dijkstra's algorithm to calculate the optimal route, taking into account the current congestion.

[0897] Output: Optimal route information is generated.

[0898] Step 4:

[0899] The server sends the calculated optimal route to a digital display device.

[0900] Input: Optimal route information.

[0901] Specific operation: The server generates a data packet to send the optimal route to the digital display device and transmits it using HTTPS communication.

[0902] Output: Optimal route information is displayed on a digital display device.

[0903] Step 5:

[0904] The user confirms the optimal route displayed on the digital display and begins to move.

[0905] Input: Optimal route information displayed on a digital display device.

[0906] Specific action: The user checks the route directions displayed on their smartphone or tablet and follows them.

[0907] Output: The user moves towards the destination.

[0908] Step 6:

[0909] The server analyzes the user's purchase history and inventory information to generate a list of recommended products.

[0910] Input: User purchase history data, current inventory information.

[0911] Specific operation: The server uses generative artificial intelligence to analyze purchase history and inventory information, and lists the most suitable recommended products for the user. Prompt messages are input into the AI ​​model to obtain the results.

[0912] Output: A list of recommended products is generated.

[0913] Step 7:

[0914] The server sends a list of recommended products to a digital display device.

[0915] Input: Recommended product list.

[0916] Specific operation: Generate a data packet to send the recommended product list generated by the server to a digital display device, and send it using HTTPS communication.

[0917] Output: Recommended product information is displayed on the digital display device.

[0918] Step 8:

[0919] The server retrieves the latest event information from the database and notifies the user based on their location.

[0920] Input: Event information database, user location information.

[0921] Specific operation: The server retrieves the latest event information from the database and generates appropriate event notifications based on the user's current location.

[0922] Output: Event notification information is generated.

[0923] Step 9:

[0924] The server sends the generated event notification information to the digital display device.

[0925] Input: Event notification information.

[0926] Specific operation: The server generates a data packet to send event notification information to a digital display device and transmits it using HTTPS communication.

[0927] Output: An event notification is displayed on the digital display device.

[0928] Step 10:

[0929] Users select products in an unmanned store and scan the product barcodes on a digital display.

[0930] Input: Product barcode information.

[0931] Specific operation: Scan the barcode of the product selected by the user onto the digital display device.

[0932] Output: Product barcode information is transmitted from the digital display device to the server.

[0933] Step 11:

[0934] The server receives the barcode information and verifies the inventory status.

[0935] Input: Product barcode information.

[0936] Specific operation: The server receives barcode information and retrieves inventory information for the corresponding product from its internal database.

[0937] Output: Inventory confirmation information is generated.

[0938] Step 12:

[0939] The user selects a payment method and enters payment information into the digital display.

[0940] Input: Payment information.

[0941] Specific operation: The user selects a payment method such as credit card or electronic money and enters the necessary information into the digital display device.

[0942] Output: Payment information is sent from the digital display device to the server.

[0943] Step 13:

[0944] The server processes the payment based on the payment information it receives.

[0945] Input: Payment information.

[0946] Specific operation: The server sends payment information to the payment system (e.g., Stripe or PayPal API) and processes the payment.

[0947] Output: Payment completion information is generated.

[0948] Step 14:

[0949] The server updates the inventory and notifies the digital display device of the purchase completion.

[0950] Input: Payment completion information, inventory information.

[0951] Specific operation: The server updates the inventory database and sends the purchase completion information to the digital display device.

[0952] Output: A purchase completion notification is displayed on the digital display.

[0953] These steps enable users to enjoy efficient and comfortable shopping at physical stores.

[0954] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0955] This invention provides a system that offers optimal information and services to users by linking a digital display device, a central control unit, real-time data, and an emotion engine. Specifically, it includes a digital display device for users to select destinations and services, a central control unit that calculates emotion recognition and optimal routes and recommended products based on received information, and a system that manages congestion status, inventory status, and the user's emotional state in real time.

[0956] System processing and specific examples

[0957] 1. Route guidance processing

[0958] The user enters their desired seat number and destination on a digital display.

[0959] The terminal transmits that information to the central control unit.

[0960] The server (central control unit) calculates the optimal route by referring to real-time congestion information and the user's emotional state.

[0961] The terminal displays the calculated route to the user.

[0962] For example, if a user wants to know the location of "seat 102," they enter the seat number into the terminal. The central control unit calculates a more comfortable route (for example, a route that avoids congestion) based on congestion information and the user's stress level, and sends it to the terminal. The terminal then displays that route to the user on a map or in text.

[0963] 2. Recommended Products

[0964] The user selects "recommended items" on a digital display device.

[0965] The terminal sends this request to the central control unit.

[0966] The server (central control unit) analyzes the user's past purchase history, current inventory information, and the user's emotional state as recognized by the emotion engine, and generates an optimal list of recommended products.

[0967] The device displays that information to the user.

[0968] As a concrete example, when a user enters "I want to know what products are recommended" into the terminal, the server analyzes past purchase data and sentiment data and presents "currently popular T-shirts" as products that are likely to please the user. The terminal then displays this information to the user.

[0969] 3. Event Announcement

[0970] The server constantly manages the latest event information and sends updated information to the digital display device.

[0971] The device receives event information in real time and automatically sends event notifications based on specific times and conditions (e.g., when a user is near the device).

[0972] As a specific example, when a user approaches a digital display device, the device detects the user's level of excitement from their facial expression and displays a pop-up message saying, "The live event starts at 3 PM on Stage C."

[0973] 4. Purchase processing at unmanned stores

[0974] The user selects the items they want to purchase at the unmanned store and scans the barcodes of the items using a digital display device.

[0975] The terminal transmits barcode information to the central control unit to verify product inventory information.

[0976] The server (central control unit) updates the inventory status of products based on inventory information and prepares for payment processing.

[0977] The user selects a payment method and enters payment information, such as credit card details, into the terminal.

[0978] The terminal sends payment information to the server and executes the payment process.

[0979] The server confirms the completion of the payment and sends the purchase completion information to the terminal.

[0980] The device displays a purchase completion message and receipt to the user.

[0981] As a concrete example, a user selects a T-shirt at an unmanned store and scans the barcode at a terminal. The server checks the inventory information, and the user pays with a credit card. Once the payment is complete, a receipt is displayed on the terminal.

[0982] 5. Utilizing the Emotional Engine

[0983] The device senses the user's facial expressions and voice, and analyzes them using an emotion engine.

[0984] The server receives information from the emotion engine and incorporates it into recommended products and route guidance information.

[0985] The device provides real-time alternatives and support information when the user expresses stress or dissatisfaction.

[0986] For example, when a user approaches a digital display device, the emotion engine detects stress from the user's facial expression, and the central control unit suggests less crowded routes or provides information about relaxation areas.

[0987] Integration of each component

[0988] The system of this invention enhances user convenience and experiential value through the close coordination of a digital display device, a central control unit, real-time data, and an emotion engine. The central control unit manages all data centrally and updates information in real time, ensuring that users always receive the latest and most accurate information. Furthermore, by utilizing the emotion engine, it is possible to provide user-optimized recommendation services.

[0989] As described above, the present invention provides an efficient information guidance system for realizing a comfortable user experience in large domes and stadiums.

[0990] The following describes the processing flow.

[0991] Route guidance processing combined with an emotion engine

[0992] Step 1:

[0993] The user enters their destination (e.g., seat number "102") into the digital display.

[0994] Step 2:

[0995] The terminal receives the entered destination information and transmits it to the central control unit.

[0996] Step 3:

[0997] The terminal analyzes the user's facial expressions and voice using an emotion engine and transmits the results to a central control unit.

[0998] Step 4:

[0999] The server (central control unit) receives destination information and sentiment information, while simultaneously referencing real-time congestion information and geographical information to calculate the optimal route.

[1000] Step 5:

[1001] The server transmits the calculated optimal route information to a digital display device.

[1002] Step 6:

[1003] The terminal displays the received optimal route information on the user interface and provides visual or audio guidance to the user.

[1004] A guide to recommended products that combine an emotional engine.

[1005] Step 1:

[1006] The user selects "Recommended Items" from the menu on the digital display device.

[1007] Step 2:

[1008] The terminal sends this request to the central control unit.

[1009] Step 3:

[1010] The terminal analyzes the user's facial expressions and voice using an emotion engine and transmits the results to a central control unit.

[1011] Step 4:

[1012] The server (central control unit) analyzes the user's past purchase history, inventory information, and the user's emotional state as recognized by the emotion engine, and generates an optimal list of recommended products.

[1013] Step 5:

[1014] The server sends the generated list of recommended items to a digital display device.

[1015] Step 6:

[1016] The terminal displays the received product information on its user interface, visually presenting recommended products to the user.

[1017] Event announcements combined with an emotional engine

[1018] Step 1:

[1019] The server constantly manages the latest event information and sends updated information to the digital display device.

[1020] Step 2:

[1021] The device receives event information in real time and activates the emotion engine based on specific times and conditions (e.g., when a user approaches the device).

[1022] Step 3:

[1023] The device analyzes the user's facial expressions and voice using an emotion engine and displays event notifications based on the results.

[1024] Step 4:

[1025] Users can view event information displayed on their devices and obtain detailed information to participate in events that interest them.

[1026] Purchase processing in unmanned stores using an emotion engine.

[1027] Step 1:

[1028] The user selects the items they want to purchase at the unmanned store and scans the barcodes of the items using a digital display device.

[1029] Step 2:

[1030] The terminal transmits barcode information to the central control unit to verify product inventory information.

[1031] Step 3:

[1032] The terminal analyzes the user's facial expressions and voice using an emotion engine and transmits the results to a central control unit.

[1033] Step 4:

[1034] The server (central control unit) updates the inventory status of products based on inventory information and prepares for payment processing.

[1035] Step 5:

[1036] The user selects a payment method and enters payment information, such as credit card details, into a digital display device.

[1037] Step 6:

[1038] The terminal transmits payment information to the central control unit and executes the payment process.

[1039] Step 7:

[1040] The server confirms the completion of the payment and sends the purchase completion information to the digital display device.

[1041] Step 8:

[1042] The device displays a purchase completion message and receipt to the user.

[1043] Real-time support utilizing an emotion engine

[1044] Step 1:

[1045] The device constantly analyzes the user's facial expressions and voice using an emotion engine to detect signs of stress and dissatisfaction.

[1046] Step 2:

[1047] The server receives information detected by the emotion engine and generates appropriate support information and alternatives in real time.

[1048] Step 3:

[1049] The device displays support information and alternatives tailored to the user's emotional state, and provides specific guidance.

[1050] This invention enables the realization of a system that analyzes the user's emotional state and provides optimal guidance and information based on that data.

[1051] (Example 2)

[1052] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[1053] Conventional information guidance systems using digital display devices and central control units lack the ability to provide optimal information and services that take into account the user's emotional state, and there is a need to improve the user experience. Furthermore, it has been difficult to analyze the user's emotional state in real time and provide information and services accordingly, which has prevented a sufficient improvement in user convenience and satisfaction.

[1054] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1055] In this invention, the server includes means for using an emotion recognition engine that senses the user's facial expressions and voice using a digital display device and analyzes the user's emotional state; means for referencing the information from the emotion recognition engine in a central control unit and optimizing route guidance and product recommendations based on the user's emotional state; and means for providing recommended product information based on the user's hobbies and preferences. This enables the provision of optimal information and services that correspond to the user's emotional state.

[1056] A "digital display device" is an electronic device used by users to input information or to visually confirm information.

[1057] A "central control unit" is a key computer system that centrally manages data across the entire system, analyzes information, and provides optimal services.

[1058] An "emotion recognition engine" is software or hardware that analyzes a user's facial expressions and voice to determine the user's emotional state.

[1059] "Real-time congestion information" refers to the latest data showing the current congestion status of a specific location or route.

[1060] A "Geographic Information System" is a system that collects and analyzes data related to geospatial information and displays it visually.

[1061] "Generative artificial intelligence" is an artificial intelligence technology that analyzes large amounts of data and automatically generates useful information and recommendations.

[1062] "Recommended product information" refers to product information that is considered interesting to the user based on their hobbies, preferences, and past purchase history.

[1063] "Event schedule information" refers to data that includes detailed information such as the date, time, and location of various events.

[1064] An "unmanned store" is an automated store where there are no staff members present, and customers select products and complete the purchase process themselves.

[1065] A "generative AI model" is an artificial intelligence model that uses algorithms to generate useful patterns and predictions from large amounts of data.

[1066] A "prompt sentence" is an input sentence that prompts a generative AI model to generate a specific answer or information.

[1067] This invention relates to a system that provides users with optimal information and services by coordinating a digital display device, a central control unit, real-time data, and an emotion recognition engine. This system includes a digital display device for users to select destinations and services, a central control unit that calculates emotion recognition and optimal routes and recommended products based on received information, and a system that manages congestion levels, inventory status, and the user's emotional state in real time.

[1068] Route guidance processing

[1069] The digital display device provides an interface for users to input their desired seat number and destination. The terminal transmits the information entered by the user to the central control unit. The central control unit calculates the optimal route based on real-time congestion information and the user's emotional state, and transmits this information to the terminal. The terminal displays the calculated route to the user. Specifically, if a user wants to go to "seat 102," they enter that seat number into the terminal. The central control unit checks congestion information using Google Maps API and OpenStreetMap data and calculates a route that takes the user's stress level into consideration. The terminal then provides this route to the user in map or text format.

[1070] Recommended Products

[1071] When a user selects "recommended items" on a digital display, the terminal sends this request to a central control unit. The central control unit analyzes the user's past purchase history, current inventory information, and the user's emotional state provided by the emotion recognition engine to generate an optimal list of recommended items. The terminal then displays this information to the user. Specifically, if a user inputs "I want to know what recommended items are" into the terminal, the central control unit analyzes past purchase data and emotional data and recommends "popular T-shirts" as a product that is likely to please the user. The terminal then displays this information.

[1072] Event Announcement

[1073] The server constantly manages the latest event information and sends updates to the digital display device. The terminal receives event information in real time and automatically notifies the user of the event based on specific times and conditions (e.g., when a user approaches the terminal). For example, when a user approaches the digital display device, the terminal detects the user's level of excitement from their facial expression and displays a pop-up message saying, "The live event starts at 3 PM on Stage C."

[1074] Purchase processing at unmanned stores

[1075] The user selects the items they wish to purchase at the unmanned store and scans the barcodes of the items using a digital display. The terminal sends the barcode information to a central control unit to check the product's inventory. The central control unit updates the inventory information and prepares for payment processing. The user enters their credit card information into the terminal and selects a payment method, at which point the terminal sends the payment information to the central control unit for settlement processing. Once the payment is complete, the central control unit sends purchase completion information to the terminal, which then displays a purchase completion message and receipt.

[1076] Utilization of emotion recognition engines

[1077] The terminal senses the user's facial expressions and voice, and analyzes them using an emotion recognition engine. The central control unit receives information from the emotion recognition engine and incorporates it into recommended products and route guidance information. For example, if a user shows signs of stress when approaching a digital display, the central control unit provides information on less crowded routes and relaxation areas in real time.

[1078] Concrete examples of prompt sentences for generative AI models

[1079] 1. "If I want to get to seat 102, what's the best route?"

[1080] 2. "I'd like to know your recommended products. Please also consider my recent purchase history and stock availability."

[1081] 3. "Show me information about events happening nearby."

[1082] 4. "I want to buy this product. Please tell me the steps to scan the barcode and complete the payment."

[1083] 5. "Please suggest some activities or places to go when I'm feeling stressed."

[1084] This system, through the close cooperation of a digital display device, a central control unit, an emotion recognition engine, and a real-time data system, can improve the accuracy and convenience of information provided to users, thereby enhancing the user experience.

[1085] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1086] Route guidance processing

[1087] Step 1:

[1088] The user enters their destination (e.g., seat 102) into the digital display.

[1089] Input: Destination information (e.g., seat 102)

[1090] Output: Instruction to send input information to the terminal

[1091] Step 2:

[1092] The terminal transmits the destination information received from the user to the central control unit.

[1093] Input: Destination information entered by the user

[1094] Output: Transmission of destination information to the central control unit.

[1095] Operation: The terminal receives destination information entered by the user on the digital display device and communicates to transmit it to the central control unit.

[1096] Step 3:

[1097] Based on destination information received by the server (central control unit), the system analyzes real-time congestion information and emotion recognition engine data to calculate the optimal route.

[1098] Input: Destination information, real-time congestion information, emotion recognition engine data.

[1099] Output: Optimal route information

[1100] Operation: The central control unit obtains real-time congestion information from Google Maps API and OpenStreetMap, and calculates the optimal route to avoid congestion based on the user's emotional state provided by the emotion recognition engine.

[1101] Step 4:

[1102] The terminal displays the optimal route information received from the server to the user.

[1103] Input: Optimal route information from the server

[1104] Output: Optimal route information displayed to the user

[1105] Operation: The terminal displays the optimal route received from the server to the user in map or text format.

[1106] Recommended Products

[1107] Step 1:

[1108] The user selects "recommended items" on a digital display device.

[1109] Input: User selection information (e.g., "Recommended items")

[1110] Output: Instruction to send input information to the terminal

[1111] Step 2:

[1112] The terminal sends a request from the user to the central control unit.

[1113] Input: User's selected information

[1114] Output: Sending request information to the central control unit.

[1115] Operation: The terminal communicates to send the user's request for "recommended items" to the central control unit.

[1116] Step 3:

[1117] The server (central control unit) analyzes the user's past purchase history, current inventory information, and emotion recognition engine data based on the request information to generate an optimal list of recommended products.

[1118] Input: Request information, past purchase history, inventory information, emotion recognition engine data.

[1119] Output: Best Recommended Goods List

[1120] Operation: The central control unit retrieves the user's past purchase history from the database and selects and generates the most suitable products for the user based on current inventory information and data from the emotion recognition engine.

[1121] Step 4:

[1122] The device displays a list of recommended items received from the server to the user.

[1123] Input: Recommended goods list from the server

[1124] Output: Recommended product list displayed to the user

[1125] Operation: The terminal displays a list of recommended items received from the server to the user.

[1126] Event Announcement

[1127] Step 1:

[1128] The server manages the latest event information and transmits it to a digital display device.

[1129] Input: Event Information Database

[1130] Output: Sending event information to a digital display device.

[1131] Operation: The server constantly manages the latest event information and sends necessary updates to the digital display device.

[1132] Step 2:

[1133] The device automatically displays event information based on specific conditions.

[1134] Input: Event information from the server, proximity sensor information

[1135] Output: Event information displayed to the user

[1136] Operation: The device automatically displays event information as a pop-up based on the user's proximity and other specific conditions.

[1137] Purchase processing at unmanned stores

[1138] Step 1:

[1139] Users select products in an unmanned store and scan the product barcodes using a digital display.

[1140] Input: Barcode information

[1141] Output: Instruction to send barcode information to the terminal

[1142] Step 2:

[1143] The terminal transmits barcode information to the central control unit.

[1144] Input: Barcode information

[1145] Output: Transmission of barcode information to the central control unit.

[1146] Operation: The terminal communicates to transmit barcode information from the user to the central control unit.

[1147] Step 3:

[1148] The server (central control unit) checks inventory information and updates the inventory status.

[1149] Input: Barcode information, inventory database

[1150] Output: Inventory update information

[1151] Operation: The central control unit retrieves inventory information from the product database and performs necessary updates.

[1152] Step 4:

[1153] The user selects a payment method and enters payment information, such as credit card details, into the terminal.

[1154] Input: Payment information (e.g., credit card information)

[1155] Output: Notification that payment information has been successfully entered into the terminal.

[1156] Step 5:

[1157] The terminal sends payment information to the server and executes the payment process.

[1158] Input: Payment Information

[1159] Output: Sending payment information to the server

[1160] Operation: The terminal transmits the user's payment information to the central control unit and processes the payment via the payment API.

[1161] Step 6:

[1162] The server confirms the completion of the payment and sends the purchase completion information to the terminal.

[1163] Input: Payment Information

[1164] Output: Sending purchase completion information

[1165] Operation: The central control unit confirms the completion of the payment and sends the purchase completion information to the terminal.

[1166] Step 7:

[1167] The device displays a purchase completion message and receipt to the user.

[1168] Input: Purchase completion information

[1169] Output: Purchase completion message and electronic receipt displayed to the user.

[1170] Operation: The device displays a purchase completion message and an electronic receipt.

[1171] Utilization of emotion recognition engines

[1172] Step 1:

[1173] The device detects the user's facial expressions and voice, and analyzes them using an emotion recognition engine.

[1174] Input: User facial expression data, voice data

[1175] Output: Analyzed emotional state

[1176] Operation: The device uses a camera and microphone to sense the user's facial expressions and voice, and analyzes them in real time using an emotion recognition engine.

[1177] Step 2:

[1178] The server (central control unit) receives information from the emotion recognition engine and incorporates it into the recommendations.

[1179] Input: Analyzed emotional state

[1180] Output: Optimized recommendations

[1181] Operation: The central control unit generates optimal product and route guidance information that corresponds to the user's emotional state, based on information from the emotion recognition engine.

[1182] Step 3:

[1183] The device displays recommendations based on the user's emotional state.

[1184] Input: Optimized recommendations

[1185] Output: Recommended information displayed to the user

[1186] Operation: The terminal displays recommended information received from the central control unit to the user. For example, if the user is feeling stressed, it will display information about less crowded routes or relaxation areas.

[1187] Concrete examples of prompt sentences for generative AI models

[1188] 1. "If I want to get to seat 102, what's the best route?"

[1189] 2. "I'd like to know your recommended products. Please also consider my recent purchase history and stock availability."

[1190] 3. "Show me information about events happening nearby."

[1191] 4. "I want to buy this product. Please tell me the steps to scan the barcode and complete the payment."

[1192] 5. "Please suggest some activities or places to go when I'm feeling stressed."

[1193] (Application Example 2)

[1194] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[1195] Traditional store guidance systems did not provide route guidance or product recommendations that took into account congestion levels or the user's emotional state. As a result, users were forced to take crowded routes or had difficulty finding products that matched their emotional state. Furthermore, event announcements and information about relaxation areas were not provided in real time. These problems led to a decline in the user's shopping experience.

[1196] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1197] In this invention, the server includes a digital display device and means for the user to input their desired destination; a central control unit and means for receiving destination information from the digital display device and calculating the optimal route; and means for providing the user with optimal route guidance using a geographic information system that includes real-time congestion information. This enables the user to take a comfortable route that avoids congestion.

[1198] Furthermore, the system includes means for analyzing the user's emotional state using an emotion recognition engine and providing optimal product and route guidance, means for analyzing past purchase data using generative artificial intelligence and providing recommended product information, means for automatically notifying users of real-time event information under specific conditions, and means for displaying guidance information in the user's field of view using a smart device. This improves the user's shopping experience and enables real-time relaxation and guidance based on event information.

[1199] A "digital display device" is an electronic device used to input and display information such as the destination a user wants to go to or the product information they have selected.

[1200] A "central control unit" is a device that calculates the optimal route and products based on destination information received from digital display devices, while also referencing real-time congestion information and the user's emotional state.

[1201] "Real-time congestion information" is data that continuously updates the current congestion situation and is used to consider user travel routes and usage patterns within stores.

[1202] A "geographic information system" refers to a system that provides destination and route information in the form of maps, and may include real-time updated congestion information.

[1203] An "emotion recognition engine" is an artificial intelligence technology that analyzes a user's facial expressions and voice to recognize their emotional state (for example, stress or joy).

[1204] A "smart device" is a device with advanced functions, such as a smartphone or smart glasses, that is used to display information within the user's field of vision.

[1205] "Generative artificial intelligence" refers to artificial intelligence models that analyze past data and patterns to generate new information and recommendations.

[1206] "Recommended product information" refers to product recommendations for users based on their past purchase history and current emotional state.

[1207] "Event notification" is a function that notifies users of event information in real time based on a specific time and the user's location.

[1208] An "unmanned store" is a store that does not have any staff and uses digital technology to sell products and process payments.

[1209] This invention is a system that combines a digital display device, a central control unit, real-time data, and an emotion recognition engine to provide users with optimal information and services. The following configuration is adopted to implement this invention.

[1210] 1. System Configuration

[1211] The system consists of a digital display device, a central control unit, a geographic information system including real-time data, an emotion recognition engine, and smart devices (smartphones, smart glasses). This system provides users with destination guidance, product recommendations, event notifications, and emotion-based assistance.

[1212] 2. Digital display device

[1213] The digital display device allows users to search for destinations and products, and displays related information. When a user enters their desired destination into the digital display device, that information is transmitted to the central control unit.

[1214] 3. Central Control Unit

[1215] The central control unit receives destination information from the digital display device and calculates the optimal route based on real-time congestion information and data from the emotion recognition engine. Furthermore, it analyzes past purchase data and inventory information to generate recommended product information and transmits it to the digital display device.

[1216] 4. Geographic Information Systems Including Real-Time Data

[1217] The geographic information system provides geographic data, including real-time congestion information, which is used by a central control unit to provide optimal route guidance. This allows users to follow routes that avoid congestion.

[1218] 5. Emotion Recognition Engine

[1219] The emotion recognition engine analyzes the user's facial expressions and voice to identify their emotional state. This information is transmitted to a central control unit, which then suggests the most suitable products and services based on the user's emotional state.

[1220] 6. Smart devices

[1221] Smart devices (smartphones and smart glasses) are used to display guidance information directly within the user's field of vision. They also automatically display real-time event information and notify the user under specific conditions.

[1222] 7. Specific Examples and Prompts

[1223] Specific example

[1224] Assume a user is searching for products in a store. The user enters "Blue Jeans" into a digital display. This information is sent to a central control unit, which calculates the optimal route and a route that avoids congestion based on real-time congestion information and emotion recognition engine data. The results are displayed on the digital display or smart glasses, allowing the user to smoothly reach the location of Blue Jeans.

[1225] Example of a prompt

[1226] "A user searches for a product using a smartphone app. The user enters that they are looking for 'blue jeans.' Based on this information, generate a flow that suggests the optimal product location and the best route to avoid crowds."

[1227] This system will enable users to have a more comfortable and efficient shopping experience. Real-time information updates and emotion-based service delivery are also expected to improve user satisfaction.

[1228] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1229] Step 1:

[1230] The user enters destination and product information into a digital display device. The entered information is text data such as destination and product name. The digital display device then transmits the user's input to a central control unit.

[1231] Step 2:

[1232] The central control unit receives information transmitted from the user. Based on the received information, it communicates with the geographic information system, which includes real-time congestion information, to obtain current congestion status and route information. The acquired data is real-time congestion data from the geographic information system.

[1233] Step 3:

[1234] The central control unit works in conjunction with the emotion recognition engine, which analyzes the user's emotional state from facial expressions and voice data. The emotion recognition engine captures user data using hardware such as cameras and microphones and transmits the analysis results to the central control unit. The analysis results include the user's stress level and satisfaction level.

[1235] Step 4:

[1236] The central control unit calculates the optimal route based on received congestion information and emotional states. The calculation uses an algorithm that combines real-time congestion information and data from the emotion recognition engine to determine the most comfortable route for the user. The calculation result is the optimal route information.

[1237] Step 5:

[1238] The central control unit analyzes past purchase data and current inventory information, and uses generative artificial intelligence to list recommended products. The generative AI combines the user's past purchase history with inventory data to recommend products that the user is likely to be interested in. The analysis results in a list of recommended products.

[1239] Step 6:

[1240] The central control unit transmits optimal route information and a list of recommended products to digital display devices and smart devices. The receiving terminals then display this information to the user. Display methods include maps, text, and augmented reality (AR) displays.

[1241] Step 7:

[1242] When a user approaches a digital display device or smart device, the central control unit acquires real-time event information based on the location data and issues a notification under specific conditions. This notification is automatically generated based on the user's location data, and the displayed content is event notification information.

[1243] Step 8:

[1244] When a user scans an item in an unmanned store, the digital display device transmits the item's barcode information to a central control unit. The central control unit checks the inventory from the received barcode information and prepares for payment processing. The user selects a payment method and executes the payment process.

[1245] Step 9:

[1246] After payment is completed, the central control unit transmits purchase completion information to the digital display device, notifying the user of the fact. The display shows a purchase completion message and a receipt.

[1247] By following these steps, users can enjoy a comfortable and efficient shopping experience. Real-time information updates and emotion-based service delivery are expected to improve user satisfaction.

[1248] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1249] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1250] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[1251] [Third Embodiment]

[1252] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[1253] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1254] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1255] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[1256] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1257] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1258] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1259] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1260] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1261] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1262] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1263] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[1264] This invention is a system for efficiently guiding users to their desired destinations, recommending products, and announcing events. Specific embodiments for implementing this invention are described below.

[1265] System Overview

[1266] The system of the present invention provides users with optimal information by linking a digital display device, a central control unit, and real-time data. It includes a digital display device for users to select destinations and services, a central control unit that calculates the optimal route and recommended products based on the received information, and a system that manages congestion and inventory status in real time.

[1267] System processing and specific examples

[1268] 1. Route guidance processing

[1269] The user enters their desired seat number and destination on a digital display.

[1270] The terminal transmits that information to the central control unit.

[1271] The server (central control unit) refers to real-time congestion information and calculates the optimal route.

[1272] The terminal displays the calculated route to the user.

[1273] For example, if a user wants to know the location of "seat 102," they would enter the seat number "102" into the terminal. Based on congestion information, the central control unit sends specific route guidance to the terminal, such as "turn left down aisle B, go up the stairs, and turn right." The terminal then displays this information to the user as a map or text.

[1274] 2. Recommended Products

[1275] The user selects "recommended items" on a digital display device.

[1276] The terminal sends the request to the central control unit.

[1277] The server (central control unit) uses generative artificial intelligence to analyze the user's past purchase history and inventory information, and generates an optimal list of recommended products.

[1278] The device displays that information to the user.

[1279] For example, if a user enters "I want to know about recommended products" into the terminal, the server analyzes past purchase data and generates information such as, "This T-shirt is popular. You can purchase it at the unmanned store in Section A." The terminal then provides this information to the user.

[1280] 3. Event Announcement

[1281] The server constantly manages the latest event schedule and sends updated information to the digital display device.

[1282] The device automatically displays event information when the user is nearby or at a specific time.

[1283] As a concrete example, when a user approaches the device, the device will display a pop-up message saying, "A live event will begin at 3 PM on Stage C."

[1284] 4. Purchase processing at unmanned stores

[1285] Users select products in an unmanned store and scan the product barcodes using a digital display.

[1286] The terminal transmits barcode information to the central control unit to verify inventory information.

[1287] The user selects a payment method and enters their payment information.

[1288] The terminal sends payment information to the server and processes the payment.

[1289] The server updates the inventory and sends purchase completion information to the terminal.

[1290] The device displays a receipt and a purchase confirmation message to the user.

[1291] As a concrete example, a user selects a T-shirt at an unmanned store and scans the barcode at a terminal. The server checks the inventory information, and the user pays with a credit card. Once the payment is complete, a receipt is displayed on the terminal.

[1292] Integration of each component

[1293] The system of this invention improves user convenience through the close coordination of a digital display device, a central control unit, and real-time data. The central control unit centrally manages all data and updates information in real time, ensuring that users always receive the latest and most accurate information. Furthermore, by utilizing generative artificial intelligence, it is possible to provide users with personalized recommendation services.

[1294] As described above, the present invention provides an efficient information guidance system for realizing a comfortable user experience in large domes and stadiums.

[1295] The following describes the processing flow.

[1296] Route guidance processing

[1297] Step 1:

[1298] The user enters their destination (e.g., seat number "102") into the digital display.

[1299] Step 2:

[1300] The terminal receives the entered destination information and transmits it to the central control unit.

[1301] Step 3:

[1302] The server (central control unit) receives destination information and, simultaneously, calculates the optimal route by referencing real-time congestion information and a geographic information system (GIS).

[1303] Step 4:

[1304] The server transmits the calculated optimal route information to a digital display device.

[1305] Step 5:

[1306] The terminal displays the received optimal route information on the user interface and provides visual or audio guidance to the user.

[1307] Recommended Products

[1308] Step 1:

[1309] The user selects "Recommended Items" from the menu on the digital display device.

[1310] Step 2:

[1311] The terminal sends this request to the central control unit.

[1312] Step 3:

[1313] The server (central control unit) retrieves the user's past purchase history and current inventory information, and uses generative artificial intelligence to generate a list of recommended products best suited to the user.

[1314] Step 4:

[1315] The server sends the generated list of recommended items to the device.

[1316] Step 5:

[1317] The terminal displays the received product information on its user interface, visually presenting recommended products to the user.

[1318] Event Announcement

[1319] Step 1:

[1320] The server constantly manages the latest event information and sends updated information to the digital display device.

[1321] Step 2:

[1322] The device receives event information in real time and automatically sends event notifications based on specific times and conditions (e.g., when a user is near the device).

[1323] Step 3:

[1324] Users can view event information displayed on their devices and obtain detailed information to participate in events that interest them.

[1325] Purchase processing at unmanned stores

[1326] Step 1:

[1327] The user selects the items they want to purchase at the unmanned store and scans the barcodes of the items using a digital display device.

[1328] Step 2:

[1329] The terminal transmits barcode information to the central control unit to verify product inventory information.

[1330] Step 3:

[1331] The server (central control unit) updates the inventory status of products based on inventory information and prepares for payment processing.

[1332] Step 4:

[1333] The user selects a payment method and enters payment information, such as credit card details, into the terminal.

[1334] Step 5:

[1335] The terminal sends payment information to the server and executes the payment process.

[1336] Step 6:

[1337] The server confirms the completion of the payment and sends the purchase completion information to the terminal.

[1338] Step 7:

[1339] The device displays a purchase completion message and receipt to the user.

[1340] (Example 1)

[1341] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1342] Traditional navigation systems make it difficult for users to obtain the latest information about their destinations and recommended products. Furthermore, they suffer from inefficiencies in unmanned store purchasing processes and event announcements, resulting in a poor user experience.

[1343] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1344] In this invention, the server includes a digital display device including means for the user to input a destination, a central control device including means for receiving destination information from the digital display device and calculating the optimal route, and means for providing the user with optimal route guidance using a geographic information system including real-time congestion information. This makes it possible for the user to receive optimal route guidance based on real-time information.

[1345] Furthermore, the system includes means for analyzing the user's past purchase history and inventory information using generative artificial intelligence to generate recommended products, means for transmitting information based on the user's request to a central control unit, a digital display device including means for scanning product barcodes in an unmanned store and processing payments, means for transmitting payment information to the central control unit and processing settlements, and means including a central control unit that updates inventory information and notifies the user of purchase completion. This streamlines the user's purchasing process and provides a comfortable shopping experience.

[1346] Furthermore, the system includes means for automatically displaying event information when the user is nearby or at a specific time, and means for transmitting event schedule information to a digital display device. This ensures that users always have access to the latest event information, improving the convenience of event participation.

[1347] A "digital display device" is an electronic device that allows users to input destination and request information and then displays the results.

[1348] A "central control unit" is a computer system that processes data received from digital display devices and calculates necessary information.

[1349] "Real-time congestion information" refers to data that captures the current congestion situation in real time and provides it in an analyzable format.

[1350] A "Geographic Information System" is a system that manages and analyzes geographical data and provides functions such as route guidance and map display.

[1351] "Generative artificial intelligence" is an artificial intelligence technology that analyzes a user's past data to generate optimal information and recommended products.

[1352] "Barcode scanning" is the act of reading information from a product's barcode using an optical scanner.

[1353] "Payment processing" refers to the procedure for settling the payment for purchased goods.

[1354] "Event schedule information" refers to the timetable and detailed information of an event that takes place at a specific time and place.

[1355] An "unmanned store" is a store where there are no staff members present, and purchases and payments are handled through an automated system.

[1356] A "user" is someone who uses this system to receive services such as directions to a destination or the purchase of goods.

[1357] This invention is a system that efficiently provides users with destination guidance, recommended products, and event information. This system has a structure primarily consisting of a digital display device, a central control unit, and a real-time data linkage system, providing users with optimal information.

[1358] System Overview

[1359] The system consists of roughly three components. First, the user inputs destination and request information through a digital display device. Second, a central control unit processes the received information and calculates the optimal route and recommended products. Third, this information is displayed to the user through a terminal.

[1360] Hardware and software

[1361] This system utilizes the following key hardware and software components. Each element works in conjunction to ensure smooth information processing.

[1362] Digital display devices: These are terminals used by users to input destination and request information and display the results. Specifically, they include touch panel displays and barcode scanners.

[1363] Central Control Unit: This server analyzes received information and generates optimal routes and recommended products based on real-time data. This unit includes a generative artificial intelligence engine and a geographic information system (GIS).

[1364] Real-time data integration: This is a software interface for acquiring real-time congestion and inventory status data and using it for analysis.

[1365] System Processing Overview

[1366] Route guidance

[1367] The user enters their desired seat number and destination on a digital display. The entered information is transmitted to a central control unit via the terminal, and the server calculates the optimal route using real-time congestion information. The results are displayed on the terminal in map or text format.

[1368] Specific example: The user enters "seat 102," the server calculates a route such as "turn left down aisle B, go up the stairs, and proceed to the right," and the terminal displays this route to the user on a map.

[1369] Example of a prompt:

[1370] "Please tell me the way to seat 102."

[1371] Recommended Products

[1372] When a user selects a "recommended item" on a digital display, the terminal sends a request to a central control unit. The server uses generative artificial intelligence to analyze the user's past purchase history and inventory information to generate the most suitable recommended products. This information is then displayed to the user via the terminal.

[1373] Specific example: When a user enters "I want to know about recommended items," the server generates information such as "This T-shirt is popular. You can purchase it at the unmanned store in Section A," and the terminal displays this information to the user.

[1374] Example of a prompt:

[1375] "Please tell me your recommended products."

[1376] Event Announcement

[1377] The server manages the latest event schedule information and periodically transmits it to digital display devices. The terminal automatically displays event information when a user approaches or at specific times.

[1378] Specific example: When a user approaches the device, a pop-up appears saying, "The live event starts at 3 PM on Stage C."

[1379] Example of a prompt:

[1380] "Please tell me what events are happening at this time."

[1381] Purchase processing at unmanned stores

[1382] The user selects an item in an unmanned store and scans the barcode on a digital display. The barcode information is sent from the terminal to a central control unit, where inventory information is verified. The user selects a payment method and enters payment information, and the terminal sends this information to a server for payment processing. After payment is complete, the inventory information is updated, and a message indicating that the purchase is complete is displayed on the terminal.

[1383] Example: A user selects a T-shirt at an unmanned store, scans it, and pays with a credit card. A receipt is displayed after the payment is completed.

[1384] Example of a prompt:

[1385] "I would like to purchase this item, so I will scan the barcode."

[1386] In this way, each processing step works in conjunction, allowing users to receive optimized information in real time. The system achieves an efficient and comfortable user experience by closely coordinating digital display devices, a central control unit, and real-time data.

[1387] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1388] Route guidance processing

[1389] Step 1:

[1390] The user enters their desired destination (such as seat number) into the digital display device.

[1391] Input: Enter destination information, such as seat number "102," into the digital display.

[1392] Output: The user's input is registered in the system.

[1393] Step 2:

[1394] The terminal transmits the entered destination information to the central control unit (server).

[1395] Input: Data such as the seat number "102" entered by the user.

[1396] Output: Destination information is sent to the server.

[1397] Step 3:

[1398] The server refers to real-time congestion information and calculates the optimal route.

[1399] Input: Destination information and real-time congestion information.

[1400] Data processing / calculation: The server analyzes congestion information and applies an algorithm to calculate the optimal route.

[1401] Output: Calculated optimal route (e.g., turn left at passage B, go up the stairs and proceed to the right).

[1402] Step 4:

[1403] The terminal displays the calculated route information to the user.

[1404] Input: Optimal route information sent from the server.

[1405] Output: Route displayed on the device in map or text format.

[1406] ---

[1407] Recommended Products

[1408] Step 1:

[1409] The user selects "recommended items" on a digital display device.

[1410] Input: The user selects a "recommended item".

[1411] Output: Registered as a request in the system.

[1412] Step 2:

[1413] The terminal sends the request to the central control unit (server).

[1414] Input: User request information.

[1415] Output: Request information is sent to the server.

[1416] Step 3:

[1417] The server uses generative artificial intelligence to analyze the user's past purchase history and inventory information, and generates a list of optimal recommended products.

[1418] Input: User's past purchase history and inventory information.

[1419] Data processing / calculation: The server uses generative artificial intelligence to analyze this data and select the most suitable product.

[1420] Output: Recommended product list (e.g., "This T-shirt is popular").

[1421] Step 4:

[1422] The device displays a generated list of recommended products to the user.

[1423] Input: A list of recommended products sent from the server.

[1424] Output: The product list is displayed on the terminal.

[1425] ---

[1426] Event Announcement

[1427] Step 1:

[1428] The server constantly manages the latest event schedule and sends updated information to the digital display device.

[1429] Input: Latest event schedule information.

[1430] Output: Update information is sent to the digital display device.

[1431] Step 2:

[1432] The device automatically displays event information when the user is nearby or at a specific time.

[1433] Input: User proximity information, specific time.

[1434] Output: Pop-up display of event information (e.g., "The live event will start at Stage C at 3 PM").

[1435] ---

[1436] Purchase processing at unmanned stores

[1437] Step 1:

[1438] Users select products in an unmanned store and scan the product barcodes using a digital display.

[1439] Input: Product barcode information.

[1440] Output: Barcode information is registered in the system.

[1441] Step 2:

[1442] The terminal transmits barcode information to the central control unit (server) to verify inventory information.

[1443] Input: Barcode information.

[1444] Output: Inventory information is sent to the server.

[1445] Step 3:

[1446] The user selects a payment method and enters their payment information.

[1447] Input: User's payment information.

[1448] Output: Payment information is registered in the system.

[1449] Step 4:

[1450] The terminal sends payment information to the server and processes the payment.

[1451] Input: User's payment information.

[1452] Output: The payment result is sent to the server.

[1453] Step 5:

[1454] The server updates inventory information and sends purchase completion information to the terminal.

[1455] Input: Payment completion information.

[1456] Data processing / calculation: Updating inventory information.

[1457] Output: Purchase completion information is sent to the device.

[1458] Step 6:

[1459] The device displays a purchase confirmation message and receipt to the user.

[1460] Input: Purchase completion information.

[1461] Output: The receipt and purchase confirmation message will be displayed on the device.

[1462] (Application Example 1)

[1463] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1464] This invention relates to a system that provides a comfortable shopping experience for users in physical stores. In conventional physical stores, users often experience inconvenience due to the time it takes to find the desired product and to navigate the store while considering the crowd situation. Furthermore, there is a lack of recommended product information based on users' tastes and preferences, as well as timely event announcements. There is a desire to solve these problems and provide users with a comfortable and efficient shopping experience.

[1465] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1466] In this invention, the server includes a digital display device and means for the user to input their desired destination; a central control unit and means for receiving destination information from the digital display device and calculating the optimal route; means for providing the user with optimal route guidance using a geographic information system including real-time congestion information; and means for communicating with the central control unit and utilizing artificial intelligence to analyze the user's purchase history and inventory information and provide the user with optimal product recommendations; means for notifying the user of the latest event information in real time; and means for scanning product barcodes using the digital display device and processing payments. As a result, the user can efficiently move to their destination within the store, receive timely information on recommended products and events, and process payments quickly in unmanned stores.

[1467] A "digital display device" is a device equipped with an electronic display for users to input or receive information.

[1468] A "central control unit" is a device that oversees the operation of the entire system and calculates and provides the optimal route and recommended products based on information received from digital display devices.

[1469] "Real-time congestion information" refers to information that allows users to instantly grasp the current congestion situation and provide them with the most suitable route guidance.

[1470] A "Geographic Information System" is a system that manages and analyzes geospatial data and provides geographical information to users.

[1471] "Artificial intelligence" is a technology that uses machine learning and neural networks to analyze data and make decisions and predictions.

[1472] "User purchase history" refers to the historical data of products that a user has purchased in the past.

[1473] "Inventory information" refers to data about the quantity and status of products currently available in a store.

[1474] "Recommended product information" refers to providing information that recommends specific products based on the user's tastes, preferences, and purchase history.

[1475] "Event announcement" refers to informing users of the schedule and details of events taking place within the store.

[1476] An "unmanned store" is a store where there are no staff members present, and users can purchase goods automatically.

[1477] A "product barcode" is a barcode attached to a product that is used for product identification and inventory management.

[1478] "Payment processing" refers to the process of carrying out the necessary settlement procedures when purchasing goods.

[1479] The present invention provides a system that enables a comfortable shopping experience for users in physical stores. This system includes a digital display device, a central control unit, a geographic information system, a real-time data processing system, and generative artificial intelligence (AI). Details of embodiments are described below.

[1480] System Overview

[1481] The system allows users to input their desired destination using a digital display. A central control unit calculates the optimal route based on the destination information received from the digital display. Real-time congestion information is used in the route calculation to provide users with the most suitable route guidance. Furthermore, the system uses artificial intelligence to analyze the user's purchase history and inventory information, providing real-time notifications of optimal product recommendations and the latest event information.

[1482] Hardware and software to be used

[1483] Digital display devices: Smartphones, tablets, touch panel displays installed in stores, etc.

[1484] Central control unit: Servers, cloud computing environment (e.g., AWS, Google Cloud)

[1485] Geographic Information System: GIS software for managing map data (e.g., ArcGIS)

[1486] Real-time data processing system: Real-time data processing using Node.js and WebSocket

[1487] Generative artificial intelligence: AI models using TensorFlow and PyTorch implemented in Python

[1488] Data processing and data calculation

[1489] The server receives destination information transmitted from digital display devices and calculates the optimal route based on real-time congestion information. The Dijkstra algorithm is used for route calculation. Generative artificial intelligence analyzes the user's past purchase history and current inventory information to generate a personalized list of recommended products. Additionally, event information is retrieved from the database in real time and notified to the user based on their location and time.

[1490] For example, if a user enters "I'm looking for a silver ring" into the digital display, the server calculates the optimal route considering real-time congestion information and provides guidance such as, "Go straight from the entrance and proceed to the jewelry section on your right." Furthermore, based on past purchase history and inventory information, it also displays recommended product information such as, "New silver rings are on sale at a special price."

[1491] Example of a prompt

[1492] As an example of a prompt message that takes into account the user's interests, preferences, and behavioral history, the following data is generated:

[1493] "User ID: 12345, Current Location: Entrance, Destination: Jewelry Section, Past Purchase History: Necklace, Bracelet, Inventory Information: 15 'Silver Rings', Event Information: Next promotion starts at 3pm"

[1494] As described above, this system allows users to efficiently navigate to their destinations within the store, receive timely information on recommended products and events, and process payments quickly even in unmanned stores.

[1495] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1496] Step 1:

[1497] The user enters their desired destination into a digital display.

[1498] Input: Destination information entered by the user into the digital display device.

[1499] Specific operation: The user enters a destination, such as "Jewelry Section," on the touchscreen or smartphone.

[1500] Output: Destination information is transmitted from the digital display device to the central control unit.

[1501] Step 2:

[1502] The server receives destination information from a digital display device and obtains current congestion information.

[1503] Input: Destination information transmitted from a digital display device.

[1504] Specific operation: The server receives destination information and retrieves real-time updated congestion information from its internal database.

[1505] Output: Destination information and congestion information are stored on the server.

[1506] Step 3:

[1507] The server calculates the optimal route based on destination information and congestion information.

[1508] Input: Destination information, congestion information.

[1509] Specific operation: The server uses Dijkstra's algorithm to calculate the optimal route, taking into account the current congestion.

[1510] Output: Optimal route information is generated.

[1511] Step 4:

[1512] The server sends the calculated optimal route to a digital display device.

[1513] Input: Optimal route information.

[1514] Specific operation: The server generates a data packet to send the optimal route to the digital display device and transmits it using HTTPS communication.

[1515] Output: Optimal route information is displayed on a digital display device.

[1516] Step 5:

[1517] The user confirms the optimal route displayed on the digital display and begins to move.

[1518] Input: Optimal route information displayed on a digital display device.

[1519] Specific action: The user checks the route directions displayed on their smartphone or tablet and follows them.

[1520] Output: The user moves towards the destination.

[1521] Step 6:

[1522] The server analyzes the user's purchase history and inventory information to generate a list of recommended products.

[1523] Input: User purchase history data, current inventory information.

[1524] Specific operation: The server uses generative artificial intelligence to analyze purchase history and inventory information, and lists the most suitable recommended products for the user. Prompt messages are input into the AI ​​model to obtain the results.

[1525] Output: A list of recommended products is generated.

[1526] Step 7:

[1527] The server sends a list of recommended products to a digital display device.

[1528] Input: Recommended product list.

[1529] Specific operation: Generate a data packet to send the recommended product list generated by the server to a digital display device, and send it using HTTPS communication.

[1530] Output: Recommended product information is displayed on the digital display device.

[1531] Step 8:

[1532] The server retrieves the latest event information from the database and notifies the user based on their location.

[1533] Input: Event information database, user location information.

[1534] Specific operation: The server retrieves the latest event information from the database and generates appropriate event notifications based on the user's current location.

[1535] Output: Event notification information is generated.

[1536] Step 9:

[1537] The server sends the generated event notification information to the digital display device.

[1538] Input: Event notification information.

[1539] Specific operation: The server generates a data packet to send event notification information to a digital display device and transmits it using HTTPS communication.

[1540] Output: An event notification is displayed on the digital display device.

[1541] Step 10:

[1542] Users select products in an unmanned store and scan the product barcodes on a digital display.

[1543] Input: Product barcode information.

[1544] Specific operation: Scan the barcode of the product selected by the user onto the digital display device.

[1545] Output: Product barcode information is transmitted from the digital display device to the server.

[1546] Step 11:

[1547] The server receives the barcode information and verifies the inventory status.

[1548] Input: Product barcode information.

[1549] Specific operation: The server receives barcode information and retrieves inventory information for the corresponding product from its internal database.

[1550] Output: Inventory confirmation information is generated.

[1551] Step 12:

[1552] The user selects a payment method and enters payment information into the digital display.

[1553] Input: Payment information.

[1554] Specific operation: The user selects a payment method such as credit card or electronic money and enters the necessary information into the digital display device.

[1555] Output: Payment information is sent from the digital display device to the server.

[1556] Step 13:

[1557] The server processes the payment based on the payment information it receives.

[1558] Input: Payment information.

[1559] Specific operation: The server sends payment information to the payment system (e.g., Stripe or PayPal API) and processes the payment.

[1560] Output: Payment completion information is generated.

[1561] Step 14:

[1562] The server updates the inventory and notifies the digital display device of the purchase completion.

[1563] Input: Payment completion information, inventory information.

[1564] Specific operation: The server updates the inventory database and sends the purchase completion information to the digital display device.

[1565] Output: A purchase completion notification is displayed on the digital display.

[1566] These steps enable users to enjoy efficient and comfortable shopping at physical stores.

[1567] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1568] This invention provides a system that offers optimal information and services to users by linking a digital display device, a central control unit, real-time data, and an emotion engine. Specifically, it includes a digital display device for users to select destinations and services, a central control unit that calculates emotion recognition and optimal routes and recommended products based on received information, and a system that manages congestion status, inventory status, and the user's emotional state in real time.

[1569] System processing and specific examples

[1570] 1. Route guidance processing

[1571] The user enters their desired seat number and destination on a digital display.

[1572] The terminal transmits that information to the central control unit.

[1573] The server (central control unit) calculates the optimal route by referring to real-time congestion information and the user's emotional state.

[1574] The terminal displays the calculated route to the user.

[1575] For example, if a user wants to know the location of "seat 102," they enter the seat number into the terminal. The central control unit calculates a more comfortable route (for example, a route that avoids congestion) based on congestion information and the user's stress level, and sends it to the terminal. The terminal then displays that route to the user on a map or in text.

[1576] 2. Recommended Products

[1577] The user selects "recommended items" on a digital display device.

[1578] The terminal sends this request to the central control unit.

[1579] The server (central control unit) analyzes the user's past purchase history, current inventory information, and the user's emotional state as recognized by the emotion engine, and generates an optimal list of recommended products.

[1580] The device displays that information to the user.

[1581] As a concrete example, when a user enters "I want to know what products are recommended" into the terminal, the server analyzes past purchase data and sentiment data and presents "currently popular T-shirts" as products that are likely to please the user. The terminal then displays this information to the user.

[1582] 3. Event Announcement

[1583] The server constantly manages the latest event information and sends updated information to the digital display device.

[1584] The device receives event information in real time and automatically sends event notifications based on specific times and conditions (e.g., when a user is near the device).

[1585] As a specific example, when a user approaches a digital display device, the device detects the user's level of excitement from their facial expression and displays a pop-up message saying, "The live event starts at 3 PM on Stage C."

[1586] 4. Purchase processing at unmanned stores

[1587] The user selects the items they want to purchase at the unmanned store and scans the barcodes of the items using a digital display device.

[1588] The terminal transmits barcode information to the central control unit to verify product inventory information.

[1589] The server (central control unit) updates the inventory status of products based on inventory information and prepares for payment processing.

[1590] The user selects a payment method and enters payment information, such as credit card details, into the terminal.

[1591] The terminal sends payment information to the server and executes the payment process.

[1592] The server confirms the completion of the payment and sends the purchase completion information to the terminal.

[1593] The device displays a purchase completion message and receipt to the user.

[1594] As a concrete example, a user selects a T-shirt at an unmanned store and scans the barcode at a terminal. The server checks the inventory information, and the user pays with a credit card. Once the payment is complete, a receipt is displayed on the terminal.

[1595] 5. Utilizing the Emotional Engine

[1596] The device senses the user's facial expressions and voice, and analyzes them using an emotion engine.

[1597] The server receives information from the emotion engine and incorporates it into recommended products and route guidance information.

[1598] The device provides real-time alternatives and support information when the user expresses stress or dissatisfaction.

[1599] For example, when a user approaches a digital display device, the emotion engine detects stress from the user's facial expression, and the central control unit suggests less crowded routes or provides information about relaxation areas.

[1600] Integration of each component

[1601] The system of this invention enhances user convenience and experiential value through the close coordination of a digital display device, a central control unit, real-time data, and an emotion engine. The central control unit manages all data centrally and updates information in real time, ensuring that users always receive the latest and most accurate information. Furthermore, by utilizing the emotion engine, it is possible to provide user-optimized recommendation services.

[1602] As described above, the present invention provides an efficient information guidance system for realizing a comfortable user experience in large domes and stadiums.

[1603] The following describes the processing flow.

[1604] Route guidance processing combined with an emotion engine

[1605] Step 1:

[1606] The user enters their destination (e.g., seat number "102") into the digital display.

[1607] Step 2:

[1608] The terminal receives the entered destination information and transmits it to the central control unit.

[1609] Step 3:

[1610] The terminal analyzes the user's facial expressions and voice using an emotion engine and transmits the results to a central control unit.

[1611] Step 4:

[1612] The server (central control unit) receives destination information and sentiment information, while simultaneously referencing real-time congestion information and geographical information to calculate the optimal route.

[1613] Step 5:

[1614] The server transmits the calculated optimal route information to a digital display device.

[1615] Step 6:

[1616] The terminal displays the received optimal route information on the user interface and provides visual or audio guidance to the user.

[1617] A guide to recommended products that combine an emotional engine.

[1618] Step 1:

[1619] The user selects "Recommended Items" from the menu on the digital display device.

[1620] Step 2:

[1621] The terminal sends this request to the central control unit.

[1622] Step 3:

[1623] The terminal analyzes the user's facial expressions and voice using an emotion engine and transmits the results to a central control unit.

[1624] Step 4:

[1625] The server (central control unit) analyzes the user's past purchase history, inventory information, and the user's emotional state as recognized by the emotion engine, and generates an optimal list of recommended products.

[1626] Step 5:

[1627] The server sends the generated list of recommended items to a digital display device.

[1628] Step 6:

[1629] The terminal displays the received product information on its user interface, visually presenting recommended products to the user.

[1630] Event announcements combined with an emotional engine

[1631] Step 1:

[1632] The server constantly manages the latest event information and sends updated information to the digital display device.

[1633] Step 2:

[1634] The device receives event information in real time and activates the emotion engine based on specific times and conditions (e.g., when a user approaches the device).

[1635] Step 3:

[1636] The device analyzes the user's facial expressions and voice using an emotion engine and displays event notifications based on the results.

[1637] Step 4:

[1638] Users can view event information displayed on their devices and obtain detailed information to participate in events that interest them.

[1639] Purchase processing in unmanned stores using an emotion engine.

[1640] Step 1:

[1641] The user selects the items they want to purchase at the unmanned store and scans the barcodes of the items using a digital display device.

[1642] Step 2:

[1643] The terminal transmits barcode information to the central control unit to verify product inventory information.

[1644] Step 3:

[1645] The terminal analyzes the user's facial expressions and voice using an emotion engine and transmits the results to a central control unit.

[1646] Step 4:

[1647] The server (central control unit) updates the inventory status of products based on inventory information and prepares for payment processing.

[1648] Step 5:

[1649] The user selects a payment method and enters payment information, such as credit card details, into a digital display device.

[1650] Step 6:

[1651] The terminal transmits payment information to the central control unit and executes the payment process.

[1652] Step 7:

[1653] The server confirms the completion of the payment and sends the purchase completion information to the digital display device.

[1654] Step 8:

[1655] The device displays a purchase completion message and receipt to the user.

[1656] Real-time support utilizing an emotion engine

[1657] Step 1:

[1658] The device constantly analyzes the user's facial expressions and voice using an emotion engine to detect signs of stress and dissatisfaction.

[1659] Step 2:

[1660] The server receives information detected by the emotion engine and generates appropriate support information and alternatives in real time.

[1661] Step 3:

[1662] The device displays support information and alternatives tailored to the user's emotional state, and provides specific guidance.

[1663] This invention enables the realization of a system that analyzes the user's emotional state and provides optimal guidance and information based on that data.

[1664] (Example 2)

[1665] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1666] Conventional information guidance systems using digital display devices and central control units lack the ability to provide optimal information and services that take into account the user's emotional state, and there is a need to improve the user experience. Furthermore, it has been difficult to analyze the user's emotional state in real time and provide information and services accordingly, which has prevented a sufficient improvement in user convenience and satisfaction.

[1667] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1668] In this invention, the server includes means for using an emotion recognition engine that senses the user's facial expressions and voice using a digital display device and analyzes the user's emotional state; means for referencing the information from the emotion recognition engine in a central control unit and optimizing route guidance and product recommendations based on the user's emotional state; and means for providing recommended product information based on the user's hobbies and preferences. This enables the provision of optimal information and services that correspond to the user's emotional state.

[1669] A "digital display device" is an electronic device used by users to input information or to visually confirm information.

[1670] A "central control unit" is a key computer system that centrally manages data across the entire system, analyzes information, and provides optimal services.

[1671] An "emotion recognition engine" is software or hardware that analyzes a user's facial expressions and voice to determine the user's emotional state.

[1672] "Real-time congestion information" refers to the latest data showing the current congestion status of a specific location or route.

[1673] A "Geographic Information System" is a system that collects and analyzes data related to geospatial information and displays it visually.

[1674] "Generative artificial intelligence" is an artificial intelligence technology that analyzes large amounts of data and automatically generates useful information and recommendations.

[1675] "Recommended product information" refers to product information that is considered interesting to the user based on their hobbies, preferences, and past purchase history.

[1676] "Event schedule information" refers to data that includes detailed information such as the date, time, and location of various events.

[1677] An "unmanned store" is an automated store where there are no staff members present, and customers select products and complete the purchase process themselves.

[1678] A "generative AI model" is an artificial intelligence model that uses algorithms to generate useful patterns and predictions from large amounts of data.

[1679] A "prompt sentence" is an input sentence that prompts a generative AI model to generate a specific answer or information.

[1680] This invention relates to a system that provides users with optimal information and services by coordinating a digital display device, a central control unit, real-time data, and an emotion recognition engine. This system includes a digital display device for users to select destinations and services, a central control unit that calculates emotion recognition and optimal routes and recommended products based on received information, and a system that manages congestion levels, inventory status, and the user's emotional state in real time.

[1681] Route guidance processing

[1682] The digital display device provides an interface for users to input their desired seat number and destination. The terminal transmits the information entered by the user to the central control unit. The central control unit calculates the optimal route based on real-time congestion information and the user's emotional state, and transmits this information to the terminal. The terminal displays the calculated route to the user. Specifically, if a user wants to go to "seat 102," they enter that seat number into the terminal. The central control unit checks congestion information using Google Maps API and OpenStreetMap data and calculates a route that takes the user's stress level into consideration. The terminal then provides this route to the user in map or text format.

[1683] Recommended Products

[1684] When a user selects "recommended items" on a digital display, the terminal sends this request to a central control unit. The central control unit analyzes the user's past purchase history, current inventory information, and the user's emotional state provided by the emotion recognition engine to generate an optimal list of recommended items. The terminal then displays this information to the user. Specifically, if a user inputs "I want to know what recommended items are" into the terminal, the central control unit analyzes past purchase data and emotional data and recommends "popular T-shirts" as a product that is likely to please the user. The terminal then displays this information.

[1685] Event Announcement

[1686] The server constantly manages the latest event information and sends updates to the digital display device. The terminal receives event information in real time and automatically notifies the user of the event based on specific times and conditions (e.g., when a user approaches the terminal). For example, when a user approaches the digital display device, the terminal detects the user's level of excitement from their facial expression and displays a pop-up message saying, "The live event starts at 3 PM on Stage C."

[1687] Purchase processing at unmanned stores

[1688] The user selects the items they wish to purchase at the unmanned store and scans the barcodes of the items using a digital display. The terminal sends the barcode information to a central control unit to check the product's inventory. The central control unit updates the inventory information and prepares for payment processing. The user enters their credit card information into the terminal and selects a payment method, at which point the terminal sends the payment information to the central control unit for settlement processing. Once the payment is complete, the central control unit sends purchase completion information to the terminal, which then displays a purchase completion message and receipt.

[1689] Utilization of emotion recognition engines

[1690] The terminal senses the user's facial expressions and voice, and analyzes them using an emotion recognition engine. The central control unit receives information from the emotion recognition engine and incorporates it into recommended products and route guidance information. For example, if a user shows signs of stress when approaching a digital display, the central control unit provides information on less crowded routes and relaxation areas in real time.

[1691] Concrete examples of prompt sentences for generative AI models

[1692] 1. "If I want to get to seat 102, what's the best route?"

[1693] 2. "I'd like to know your recommended products. Please also consider my recent purchase history and stock availability."

[1694] 3. "Show me information about events happening nearby."

[1695] 4. "I want to buy this product. Please tell me the steps to scan the barcode and complete the payment."

[1696] 5. "Please suggest some activities or places to go when I'm feeling stressed."

[1697] This system, through the close cooperation of a digital display device, a central control unit, an emotion recognition engine, and a real-time data system, can improve the accuracy and convenience of information provided to users, thereby enhancing the user experience.

[1698] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1699] Route guidance processing

[1700] Step 1:

[1701] The user enters their destination (e.g., seat 102) into the digital display.

[1702] Input: Destination information (e.g., seat 102)

[1703] Output: Instruction to send input information to the terminal

[1704] Step 2:

[1705] The terminal transmits the destination information received from the user to the central control unit.

[1706] Input: Destination information entered by the user

[1707] Output: Transmission of destination information to the central control unit.

[1708] Operation: The terminal receives destination information entered by the user on the digital display device and communicates to transmit it to the central control unit.

[1709] Step 3:

[1710] Based on destination information received by the server (central control unit), the system analyzes real-time congestion information and emotion recognition engine data to calculate the optimal route.

[1711] Input: Destination information, real-time congestion information, emotion recognition engine data.

[1712] Output: Optimal route information

[1713] Operation: The central control unit obtains real-time congestion information from Google Maps API and OpenStreetMap, and calculates the optimal route to avoid congestion based on the user's emotional state provided by the emotion recognition engine.

[1714] Step 4:

[1715] The terminal displays the optimal route information received from the server to the user.

[1716] Input: Optimal route information from the server

[1717] Output: Optimal route information displayed to the user

[1718] Operation: The terminal displays the optimal route received from the server to the user in map or text format.

[1719] Recommended Products

[1720] Step 1:

[1721] The user selects "recommended items" on a digital display device.

[1722] Input: User selection information (e.g., "Recommended items")

[1723] Output: Instruction to send input information to the terminal

[1724] Step 2:

[1725] The terminal sends a request from the user to the central control unit.

[1726] Input: User's selected information

[1727] Output: Sending request information to the central control unit.

[1728] Operation: The terminal communicates to send the user's request for "recommended items" to the central control unit.

[1729] Step 3:

[1730] The server (central control unit) analyzes the user's past purchase history, current inventory information, and emotion recognition engine data based on the request information to generate an optimal list of recommended products.

[1731] Input: Request information, past purchase history, inventory information, emotion recognition engine data.

[1732] Output: Best Recommended Goods List

[1733] Operation: The central control unit retrieves the user's past purchase history from the database and selects and generates the most suitable products for the user based on current inventory information and data from the emotion recognition engine.

[1734] Step 4:

[1735] The device displays a list of recommended items received from the server to the user.

[1736] Input: Recommended goods list from the server

[1737] Output: Recommended product list displayed to the user

[1738] Operation: The terminal displays a list of recommended items received from the server to the user.

[1739] Event Announcement

[1740] Step 1:

[1741] The server manages the latest event information and transmits it to a digital display device.

[1742] Input: Event Information Database

[1743] Output: Sending event information to a digital display device.

[1744] Operation: The server constantly manages the latest event information and sends necessary updates to the digital display device.

[1745] Step 2:

[1746] The device automatically displays event information based on specific conditions.

[1747] Input: Event information from the server, proximity sensor information

[1748] Output: Event information displayed to the user

[1749] Operation: The device automatically displays event information as a pop-up based on the user's proximity and other specific conditions.

[1750] Purchase processing at unmanned stores

[1751] Step 1:

[1752] Users select products in an unmanned store and scan the product barcodes using a digital display.

[1753] Input: Barcode information

[1754] Output: Instruction to send barcode information to the terminal

[1755] Step 2:

[1756] The terminal transmits barcode information to the central control unit.

[1757] Input: Barcode information

[1758] Output: Transmission of barcode information to the central control unit.

[1759] Operation: The terminal communicates to transmit barcode information from the user to the central control unit.

[1760] Step 3:

[1761] The server (central control unit) checks inventory information and updates the inventory status.

[1762] Input: Barcode information, inventory database

[1763] Output: Inventory update information

[1764] Operation: The central control unit retrieves inventory information from the product database and performs necessary updates.

[1765] Step 4:

[1766] The user selects a payment method and enters payment information, such as credit card details, into the terminal.

[1767] Input: Payment information (e.g., credit card information)

[1768] Output: Notification that payment information has been successfully entered into the terminal.

[1769] Step 5:

[1770] The terminal sends payment information to the server and executes the payment process.

[1771] Input: Payment Information

[1772] Output: Sending payment information to the server

[1773] Operation: The terminal transmits the user's payment information to the central control unit and processes the payment via the payment API.

[1774] Step 6:

[1775] The server confirms the completion of the payment and sends the purchase completion information to the terminal.

[1776] Input: Payment Information

[1777] Output: Sending purchase completion information

[1778] Operation: The central control unit confirms the completion of the payment and sends the purchase completion information to the terminal.

[1779] Step 7:

[1780] The device displays a purchase completion message and receipt to the user.

[1781] Input: Purchase completion information

[1782] Output: Purchase completion message and electronic receipt displayed to the user.

[1783] Operation: The device displays a purchase completion message and an electronic receipt.

[1784] Utilization of emotion recognition engines

[1785] Step 1:

[1786] The device detects the user's facial expressions and voice, and analyzes them using an emotion recognition engine.

[1787] Input: User facial expression data, voice data

[1788] Output: Analyzed emotional state

[1789] Operation: The device uses a camera and microphone to sense the user's facial expressions and voice, and analyzes them in real time using an emotion recognition engine.

[1790] Step 2:

[1791] The server (central control unit) receives information from the emotion recognition engine and incorporates it into the recommendations.

[1792] Input: Analyzed emotional state

[1793] Output: Optimized recommendations

[1794] Operation: The central control unit generates optimal product and route guidance information that corresponds to the user's emotional state, based on information from the emotion recognition engine.

[1795] Step 3:

[1796] The device displays recommendations based on the user's emotional state.

[1797] Input: Optimized recommendations

[1798] Output: Recommended information displayed to the user

[1799] Operation: The terminal displays recommended information received from the central control unit to the user. For example, if the user is feeling stressed, it will display information about less crowded routes or relaxation areas.

[1800] Concrete examples of prompt sentences for generative AI models

[1801] 1. "If I want to get to seat 102, what's the best route?"

[1802] 2. "I'd like to know your recommended products. Please also consider my recent purchase history and stock availability."

[1803] 3. "Show me information about events happening nearby."

[1804] 4. "I want to buy this product. Please tell me the steps to scan the barcode and complete the payment."

[1805] 5. "Please suggest some activities or places to go when I'm feeling stressed."

[1806] (Application Example 2)

[1807] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1808] Traditional store guidance systems did not provide route guidance or product recommendations that took into account congestion levels or the user's emotional state. As a result, users were forced to take crowded routes or had difficulty finding products that matched their emotional state. Furthermore, event announcements and information about relaxation areas were not provided in real time. These problems led to a decline in the user's shopping experience.

[1809] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1810] In this invention, the server includes a digital display device and means for the user to input their desired destination; a central control unit and means for receiving destination information from the digital display device and calculating the optimal route; and means for providing the user with optimal route guidance using a geographic information system that includes real-time congestion information. This enables the user to take a comfortable route that avoids congestion.

[1811] Furthermore, the system includes means for analyzing the user's emotional state using an emotion recognition engine and providing optimal product and route guidance, means for analyzing past purchase data using generative artificial intelligence and providing recommended product information, means for automatically notifying users of real-time event information under specific conditions, and means for displaying guidance information in the user's field of view using a smart device. This improves the user's shopping experience and enables real-time relaxation and guidance based on event information.

[1812] A "digital display device" is an electronic device used to input and display information such as the destination a user wants to go to or the product information they have selected.

[1813] A "central control unit" is a device that calculates the optimal route and products based on destination information received from digital display devices, while also referencing real-time congestion information and the user's emotional state.

[1814] "Real-time congestion information" is data that continuously updates the current congestion situation and is used to consider user travel routes and usage patterns within stores.

[1815] A "geographic information system" refers to a system that provides destination and route information in the form of maps, and may include real-time updated congestion information.

[1816] An "emotion recognition engine" is an artificial intelligence technology that analyzes a user's facial expressions and voice to recognize their emotional state (for example, stress or joy).

[1817] A "smart device" is a device with advanced functions, such as a smartphone or smart glasses, that is used to display information within the user's field of vision.

[1818] "Generative artificial intelligence" refers to artificial intelligence models that analyze past data and patterns to generate new information and recommendations.

[1819] "Recommended product information" refers to product recommendations for users based on their past purchase history and current emotional state.

[1820] "Event notification" is a function that notifies users of event information in real time based on a specific time and the user's location.

[1821] An "unmanned store" is a store that does not have any staff and uses digital technology to sell products and process payments.

[1822] This invention is a system that combines a digital display device, a central control unit, real-time data, and an emotion recognition engine to provide users with optimal information and services. The following configuration is adopted to implement this invention.

[1823] 1. System Configuration

[1824] The system consists of a digital display device, a central control unit, a geographic information system including real-time data, an emotion recognition engine, and smart devices (smartphones, smart glasses). This system provides users with destination guidance, product recommendations, event notifications, and emotion-based assistance.

[1825] 2. Digital display device

[1826] The digital display device allows users to search for destinations and products, and displays related information. When a user enters their desired destination into the digital display device, that information is transmitted to the central control unit.

[1827] 3. Central Control Unit

[1828] The central control unit receives destination information from the digital display device and calculates the optimal route based on real-time congestion information and data from the emotion recognition engine. Furthermore, it analyzes past purchase data and inventory information to generate recommended product information and transmits it to the digital display device.

[1829] 4. Geographic Information Systems Including Real-Time Data

[1830] The geographic information system provides geographic data, including real-time congestion information, which is used by a central control unit to provide optimal route guidance. This allows users to follow routes that avoid congestion.

[1831] 5. Emotion Recognition Engine

[1832] The emotion recognition engine analyzes the user's facial expressions and voice to identify their emotional state. This information is transmitted to a central control unit, which then suggests the most suitable products and services based on the user's emotional state.

[1833] 6. Smart devices

[1834] Smart devices (smartphones and smart glasses) are used to display guidance information directly within the user's field of vision. They also automatically display real-time event information and notify the user under specific conditions.

[1835] 7. Specific Examples and Prompts

[1836] Specific example

[1837] Assume a user is searching for products in a store. The user enters "Blue Jeans" into a digital display. This information is sent to a central control unit, which calculates the optimal route and a route that avoids congestion based on real-time congestion information and emotion recognition engine data. The results are displayed on the digital display or smart glasses, allowing the user to smoothly reach the location of Blue Jeans.

[1838] Example of a prompt

[1839] "A user searches for a product using a smartphone app. The user enters that they are looking for 'blue jeans.' Based on this information, generate a flow that suggests the optimal product location and the best route to avoid crowds."

[1840] This system will enable users to have a more comfortable and efficient shopping experience. Real-time information updates and emotion-based service delivery are also expected to improve user satisfaction.

[1841] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1842] Step 1:

[1843] The user enters destination and product information into a digital display device. The entered information is text data such as destination and product name. The digital display device then transmits the user's input to a central control unit.

[1844] Step 2:

[1845] The central control unit receives information transmitted from the user. Based on the received information, it communicates with the geographic information system, which includes real-time congestion information, to obtain current congestion status and route information. The acquired data is real-time congestion data from the geographic information system.

[1846] Step 3:

[1847] The central control unit works in conjunction with the emotion recognition engine, which analyzes the user's emotional state from facial expressions and voice data. The emotion recognition engine captures user data using hardware such as cameras and microphones and transmits the analysis results to the central control unit. The analysis results include the user's stress level and satisfaction level.

[1848] Step 4:

[1849] The central control unit calculates the optimal route based on received congestion information and emotional states. The calculation uses an algorithm that combines real-time congestion information and data from the emotion recognition engine to determine the most comfortable route for the user. The calculation result is the optimal route information.

[1850] Step 5:

[1851] The central control unit analyzes past purchase data and current inventory information, and uses generative artificial intelligence to list recommended products. The generative AI combines the user's past purchase history with inventory data to recommend products that the user is likely to be interested in. The analysis results in a list of recommended products.

[1852] Step 6:

[1853] The central control unit transmits optimal route information and a list of recommended products to digital display devices and smart devices. The receiving terminals then display this information to the user. Display methods include maps, text, and augmented reality (AR) displays.

[1854] Step 7:

[1855] When a user approaches a digital display device or smart device, the central control unit acquires real-time event information based on the location data and issues a notification under specific conditions. This notification is automatically generated based on the user's location data, and the displayed content is event notification information.

[1856] Step 8:

[1857] When a user scans an item in an unmanned store, the digital display device transmits the item's barcode information to a central control unit. The central control unit checks the inventory from the received barcode information and prepares for payment processing. The user selects a payment method and executes the payment process.

[1858] Step 9:

[1859] After payment is completed, the central control unit transmits purchase completion information to the digital display device, notifying the user of the fact. The display shows a purchase completion message and a receipt.

[1860] By following these steps, users can enjoy a comfortable and efficient shopping experience. Real-time information updates and emotion-based service delivery are expected to improve user satisfaction.

[1861] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1862] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1863] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1864] [Fourth Embodiment]

[1865] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1866] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1867] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1868] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1869] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1870] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1871] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1872] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1873] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1874] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1875] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1876] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1877] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1878] This invention is a system for efficiently guiding users to their desired destinations, recommending products, and announcing events. Specific embodiments for implementing this invention are described below.

[1879] System Overview

[1880] The system of the present invention provides users with optimal information by linking a digital display device, a central control unit, and real-time data. It includes a digital display device for users to select destinations and services, a central control unit that calculates the optimal route and recommended products based on the received information, and a system that manages congestion and inventory status in real time.

[1881] System processing and specific examples

[1882] 1. Route guidance processing

[1883] The user enters their desired seat number and destination on a digital display.

[1884] The terminal transmits that information to the central control unit.

[1885] The server (central control unit) refers to real-time congestion information and calculates the optimal route.

[1886] The terminal displays the calculated route to the user.

[1887] For example, if a user wants to know the location of "seat 102," they would enter the seat number "102" into the terminal. Based on congestion information, the central control unit sends specific route guidance to the terminal, such as "turn left down aisle B, go up the stairs, and turn right." The terminal then displays this information to the user as a map or text.

[1888] 2. Recommended Products

[1889] The user selects "recommended items" on a digital display device.

[1890] The terminal sends the request to the central control unit.

[1891] The server (central control unit) uses generative artificial intelligence to analyze the user's past purchase history and inventory information, and generates an optimal list of recommended products.

[1892] The device displays that information to the user.

[1893] For example, if a user enters "I want to know about recommended products" into the terminal, the server analyzes past purchase data and generates information such as, "This T-shirt is popular. You can purchase it at the unmanned store in Section A." The terminal then provides this information to the user.

[1894] 3. Event Announcement

[1895] The server constantly manages the latest event schedule and sends updated information to the digital display device.

[1896] The device automatically displays event information when the user is nearby or at a specific time.

[1897] As a concrete example, when a user approaches the device, the device will display a pop-up message saying, "A live event will begin at 3 PM on Stage C."

[1898] 4. Purchase processing at unmanned stores

[1899] Users select products in an unmanned store and scan the product barcodes using a digital display.

[1900] The terminal transmits barcode information to the central control unit to verify inventory information.

[1901] The user selects a payment method and enters their payment information.

[1902] The terminal sends payment information to the server and processes the payment.

[1903] The server updates the inventory and sends purchase completion information to the terminal.

[1904] The device displays a receipt and a purchase confirmation message to the user.

[1905] As a concrete example, a user selects a T-shirt at an unmanned store and scans the barcode at a terminal. The server checks the inventory information, and the user pays with a credit card. Once the payment is complete, a receipt is displayed on the terminal.

[1906] Integration of each component

[1907] The system of this invention improves user convenience through the close coordination of a digital display device, a central control unit, and real-time data. The central control unit centrally manages all data and updates information in real time, ensuring that users always receive the latest and most accurate information. Furthermore, by utilizing generative artificial intelligence, it is possible to provide users with personalized recommendation services.

[1908] As described above, the present invention provides an efficient information guidance system for realizing a comfortable user experience in large domes and stadiums.

[1909] The following describes the processing flow.

[1910] Route guidance processing

[1911] Step 1:

[1912] The user enters their destination (e.g., seat number "102") into the digital display.

[1913] Step 2:

[1914] The terminal receives the entered destination information and transmits it to the central control unit.

[1915] Step 3:

[1916] The server (central control unit) receives destination information and, simultaneously, calculates the optimal route by referencing real-time congestion information and a geographic information system (GIS).

[1917] Step 4:

[1918] The server transmits the calculated optimal route information to a digital display device.

[1919] Step 5:

[1920] The terminal displays the received optimal route information on the user interface and provides visual or audio guidance to the user.

[1921] Recommended Products

[1922] Step 1:

[1923] The user selects "Recommended Items" from the menu on the digital display device.

[1924] Step 2:

[1925] The terminal sends this request to the central control unit.

[1926] Step 3:

[1927] The server (central control unit) retrieves the user's past purchase history and current inventory information, and uses generative artificial intelligence to generate a list of recommended products best suited to the user.

[1928] Step 4:

[1929] The server sends the generated list of recommended items to the device.

[1930] Step 5:

[1931] The terminal displays the received product information on its user interface, visually presenting recommended products to the user.

[1932] Event Announcement

[1933] Step 1:

[1934] The server constantly manages the latest event information and sends updated information to the digital display device.

[1935] Step 2:

[1936] The device receives event information in real time and automatically sends event notifications based on specific times and conditions (e.g., when a user is near the device).

[1937] Step 3:

[1938] Users can view event information displayed on their devices and obtain detailed information to participate in events that interest them.

[1939] Purchase processing at unmanned stores

[1940] Step 1:

[1941] The user selects the items they want to purchase at the unmanned store and scans the barcodes of the items using a digital display device.

[1942] Step 2:

[1943] The terminal transmits barcode information to the central control unit to verify product inventory information.

[1944] Step 3:

[1945] The server (central control unit) updates the inventory status of products based on inventory information and prepares for payment processing.

[1946] Step 4:

[1947] The user selects a payment method and enters payment information, such as credit card details, into the terminal.

[1948] Step 5:

[1949] The terminal sends payment information to the server and executes the payment process.

[1950] Step 6:

[1951] The server confirms the completion of the payment and sends the purchase completion information to the terminal.

[1952] Step 7:

[1953] The device displays a purchase completion message and receipt to the user.

[1954] (Example 1)

[1955] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1956] Traditional navigation systems make it difficult for users to obtain the latest information about their destinations and recommended products. Furthermore, they suffer from inefficiencies in unmanned store purchasing processes and event announcements, resulting in a poor user experience.

[1957] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1958] In this invention, the server includes a digital display device including means for the user to input a destination, a central control device including means for receiving destination information from the digital display device and calculating the optimal route, and means for providing the user with optimal route guidance using a geographic information system including real-time congestion information. This makes it possible for the user to receive optimal route guidance based on real-time information.

[1959] Furthermore, the system includes means for analyzing the user's past purchase history and inventory information using generative artificial intelligence to generate recommended products, means for transmitting information based on the user's request to a central control unit, a digital display device including means for scanning product barcodes in an unmanned store and processing payments, means for transmitting payment information to the central control unit and processing settlements, and means including a central control unit that updates inventory information and notifies the user of purchase completion. This streamlines the user's purchasing process and provides a comfortable shopping experience.

[1960] Furthermore, the system includes means for automatically displaying event information when the user is nearby or at a specific time, and means for transmitting event schedule information to a digital display device. This ensures that users always have access to the latest event information, improving the convenience of event participation.

[1961] A "digital display device" is an electronic device that allows users to input destination and request information and then displays the results.

[1962] A "central control unit" is a computer system that processes data received from digital display devices and calculates necessary information.

[1963] "Real-time congestion information" refers to data that captures the current congestion situation in real time and provides it in an analyzable format.

[1964] A "Geographic Information System" is a system that manages and analyzes geographical data and provides functions such as route guidance and map display.

[1965] "Generative artificial intelligence" is an artificial intelligence technology that analyzes a user's past data to generate optimal information and recommended products.

[1966] "Barcode scanning" is the act of reading information from a product's barcode using an optical scanner.

[1967] "Payment processing" refers to the procedure for settling the payment for purchased goods.

[1968] "Event schedule information" refers to the timetable and detailed information of an event that takes place at a specific time and place.

[1969] An "unmanned store" is a store where there are no staff members present, and purchases and payments are handled through an automated system.

[1970] A "user" is someone who uses this system to receive services such as directions to a destination or the purchase of goods.

[1971] This invention is a system that efficiently provides users with destination guidance, recommended products, and event information. This system has a structure primarily consisting of a digital display device, a central control unit, and a real-time data linkage system, providing users with optimal information.

[1972] System Overview

[1973] The system consists of roughly three components. First, the user inputs destination and request information through a digital display device. Second, a central control unit processes the received information and calculates the optimal route and recommended products. Third, this information is displayed to the user through a terminal.

[1974] Hardware and software

[1975] This system utilizes the following key hardware and software components. Each element works in conjunction to ensure smooth information processing.

[1976] Digital display devices: These are terminals used by users to input destination and request information and display the results. Specifically, they include touch panel displays and barcode scanners.

[1977] Central Control Unit: This server analyzes received information and generates optimal routes and recommended products based on real-time data. This unit includes a generative artificial intelligence engine and a geographic information system (GIS).

[1978] Real-time data integration: This is a software interface for acquiring real-time congestion and inventory status data and using it for analysis.

[1979] System Processing Overview

[1980] Route guidance

[1981] The user enters their desired seat number and destination on a digital display. The entered information is transmitted to a central control unit via the terminal, and the server calculates the optimal route using real-time congestion information. The results are displayed on the terminal in map or text format.

[1982] Specific example: The user enters "seat 102," the server calculates a route such as "turn left down aisle B, go up the stairs, and proceed to the right," and the terminal displays this route to the user on a map.

[1983] Example of a prompt:

[1984] "Please tell me the way to seat 102."

[1985] Recommended Products

[1986] When a user selects a "recommended item" on a digital display, the terminal sends a request to a central control unit. The server uses generative artificial intelligence to analyze the user's past purchase history and inventory information to generate the most suitable recommended products. This information is then displayed to the user via the terminal.

[1987] Specific example: When a user enters "I want to know about recommended items," the server generates information such as "This T-shirt is popular. You can purchase it at the unmanned store in Section A," and the terminal displays this information to the user.

[1988] Example of a prompt:

[1989] "Please tell me your recommended products."

[1990] Event Announcement

[1991] The server manages the latest event schedule information and periodically transmits it to digital display devices. The terminal automatically displays event information when a user approaches or at specific times.

[1992] Specific example: When a user approaches the device, a pop-up appears saying, "The live event starts at 3 PM on Stage C."

[1993] Example of a prompt:

[1994] "Please tell me what events are happening at this time."

[1995] Purchase processing at unmanned stores

[1996] The user selects an item in an unmanned store and scans the barcode on a digital display. The barcode information is sent from the terminal to a central control unit, where inventory information is verified. The user selects a payment method and enters payment information, and the terminal sends this information to a server for payment processing. After payment is complete, the inventory information is updated, and a message indicating that the purchase is complete is displayed on the terminal.

[1997] Example: A user selects a T-shirt at an unmanned store, scans it, and pays with a credit card. A receipt is displayed after the payment is completed.

[1998] Example of a prompt:

[1999] "I would like to purchase this item, so I will scan the barcode."

[2000] In this way, each processing step works in conjunction, allowing users to receive optimized information in real time. The system achieves an efficient and comfortable user experience by closely coordinating digital display devices, a central control unit, and real-time data.

[2001] The flow of the specific processing in Example 1 will be explained using Figure 11.

[2002] Route guidance processing

[2003] Step 1:

[2004] The user enters their desired destination (such as seat number) into the digital display device.

[2005] Input: Enter destination information, such as seat number "102," into the digital display.

[2006] Output: The user's input is registered in the system.

[2007] Step 2:

[2008] The terminal transmits the entered destination information to the central control unit (server).

[2009] Input: Data such as the seat number "102" entered by the user.

[2010] Output: Destination information is sent to the server.

[2011] Step 3:

[2012] The server refers to real-time congestion information and calculates the optimal route.

[2013] Input: Destination information and real-time congestion information.

[2014] Data processing / calculation: The server analyzes congestion information and applies an algorithm to calculate the optimal route.

[2015] Output: Calculated optimal route (e.g., turn left at passage B, go up the stairs and proceed to the right).

[2016] Step 4:

[2017] The terminal displays the calculated route information to the user.

[2018] Input: Optimal route information sent from the server.

[2019] Output: Route displayed on the device in map or text format.

[2020] ---

[2021] Recommended Products

[2022] Step 1:

[2023] The user selects "recommended items" on a digital display device.

[2024] Input: The user selects a "recommended item".

[2025] Output: Registered as a request in the system.

[2026] Step 2:

[2027] The terminal sends the request to the central control unit (server).

[2028] Input: User request information.

[2029] Output: Request information is sent to the server.

[2030] Step 3:

[2031] The server uses generative artificial intelligence to analyze the user's past purchase history and inventory information, and generates a list of optimal recommended products.

[2032] Input: User's past purchase history and inventory information.

[2033] Data processing / calculation: The server uses generative artificial intelligence to analyze this data and select the most suitable product.

[2034] Output: Recommended product list (e.g., "This T-shirt is popular").

[2035] Step 4:

[2036] The device displays a generated list of recommended products to the user.

[2037] Input: A list of recommended products sent from the server.

[2038] Output: The product list is displayed on the terminal.

[2039] ---

[2040] Event Announcement

[2041] Step 1:

[2042] The server constantly manages the latest event schedule and sends updated information to the digital display device.

[2043] Input: Latest event schedule information.

[2044] Output: Update information is sent to the digital display device.

[2045] Step 2:

[2046] The device automatically displays event information when the user is nearby or at a specific time.

[2047] Input: User proximity information, specific time.

[2048] Output: Pop-up display of event information (e.g., "The live event will start at Stage C at 3 PM").

[2049] ---

[2050] Purchase processing at unmanned stores

[2051] Step 1:

[2052] Users select products in an unmanned store and scan the product barcodes using a digital display.

[2053] Input: Product barcode information.

[2054] Output: Barcode information is registered in the system.

[2055] Step 2:

[2056] The terminal transmits barcode information to the central control unit (server) to verify inventory information.

[2057] Input: Barcode information.

[2058] Output: Inventory information is sent to the server.

[2059] Step 3:

[2060] The user selects a payment method and enters their payment information.

[2061] Input: User's payment information.

[2062] Output: Payment information is registered in the system.

[2063] Step 4:

[2064] The terminal sends payment information to the server and processes the payment.

[2065] Input: User's payment information.

[2066] Output: The payment result is sent to the server.

[2067] Step 5:

[2068] The server updates inventory information and sends purchase completion information to the terminal.

[2069] Input: Payment completion information.

[2070] Data processing / calculation: Updating inventory information.

[2071] Output: Purchase completion information is sent to the device.

[2072] Step 6:

[2073] The device displays a purchase confirmation message and receipt to the user.

[2074] Input: Purchase completion information.

[2075] Output: The receipt and purchase confirmation message will be displayed on the device.

[2076] (Application Example 1)

[2077] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[2078] This invention relates to a system that provides a comfortable shopping experience for users in physical stores. In conventional physical stores, users often experience inconvenience due to the time it takes to find the desired product and to navigate the store while considering the crowd situation. Furthermore, there is a lack of recommended product information based on users' tastes and preferences, as well as timely event announcements. There is a desire to solve these problems and provide users with a comfortable and efficient shopping experience.

[2079] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[2080] In this invention, the server includes a digital display device and means for the user to input their desired destination; a central control unit and means for receiving destination information from the digital display device and calculating the optimal route; means for providing the user with optimal route guidance using a geographic information system including real-time congestion information; and means for communicating with the central control unit and utilizing artificial intelligence to analyze the user's purchase history and inventory information and provide the user with optimal product recommendations; means for notifying the user of the latest event information in real time; and means for scanning product barcodes using the digital display device and processing payments. As a result, the user can efficiently move to their destination within the store, receive timely information on recommended products and events, and process payments quickly in unmanned stores.

[2081] A "digital display device" is a device equipped with an electronic display for users to input or receive information.

[2082] A "central control unit" is a device that oversees the operation of the entire system and calculates and provides the optimal route and recommended products based on information received from digital display devices.

[2083] "Real-time congestion information" refers to information that allows users to instantly grasp the current congestion situation and provide them with the most suitable route guidance.

[2084] A "Geographic Information System" is a system that manages and analyzes geospatial data and provides geographical information to users.

[2085] "Artificial intelligence" is a technology that uses machine learning and neural networks to analyze data and make decisions and predictions.

[2086] "User purchase history" refers to the historical data of products that a user has purchased in the past.

[2087] "Inventory information" refers to data about the quantity and status of products currently available in a store.

[2088] "Recommended product information" refers to providing information that recommends specific products based on the user's tastes, preferences, and purchase history.

[2089] "Event announcement" refers to informing users of the schedule and details of events taking place within the store.

[2090] An "unmanned store" is a store where there are no staff members present, and users can purchase goods automatically.

[2091] A "product barcode" is a barcode attached to a product that is used for product identification and inventory management.

[2092] "Payment processing" refers to the process of carrying out the necessary settlement procedures when purchasing goods.

[2093] The present invention provides a system that enables a comfortable shopping experience for users in physical stores. This system includes a digital display device, a central control unit, a geographic information system, a real-time data processing system, and generative artificial intelligence (AI). Details of embodiments are described below.

[2094] System Overview

[2095] The system allows users to input their desired destination using a digital display. A central control unit calculates the optimal route based on the destination information received from the digital display. Real-time congestion information is used in the route calculation to provide users with the most suitable route guidance. Furthermore, the system uses artificial intelligence to analyze the user's purchase history and inventory information, providing real-time notifications of optimal product recommendations and the latest event information.

[2096] Hardware and software to be used

[2097] Digital display devices: Smartphones, tablets, touch panel displays installed in stores, etc.

[2098] Central control unit: Servers, cloud computing environment (e.g., AWS, Google Cloud)

[2099] Geographic Information System: GIS software for managing map data (e.g., ArcGIS)

[2100] Real-time data processing system: Real-time data processing using Node.js and WebSocket

[2101] Generative artificial intelligence: AI models using TensorFlow and PyTorch implemented in Python

[2102] Data processing and data calculation

[2103] The server receives destination information transmitted from digital display devices and calculates the optimal route based on real-time congestion information. The Dijkstra algorithm is used for route calculation. Generative artificial intelligence analyzes the user's past purchase history and current inventory information to generate a personalized list of recommended products. Additionally, event information is retrieved from the database in real time and notified to the user based on their location and time.

[2104] For example, if a user enters "I'm looking for a silver ring" into the digital display, the server calculates the optimal route considering real-time congestion information and provides guidance such as, "Go straight from the entrance and proceed to the jewelry section on your right." Furthermore, based on past purchase history and inventory information, it also displays recommended product information such as, "New silver rings are on sale at a special price."

[2105] Example of a prompt

[2106] As an example of a prompt message that takes into account the user's interests, preferences, and behavioral history, the following data is generated:

[2107] "User ID: 12345, Current Location: Entrance, Destination: Jewelry Section, Past Purchase History: Necklace, Bracelet, Inventory Information: 15 'Silver Rings', Event Information: Next promotion starts at 3pm"

[2108] As described above, this system allows users to efficiently navigate to their destinations within the store, receive timely information on recommended products and events, and process payments quickly even in unmanned stores.

[2109] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[2110] Step 1:

[2111] The user enters their desired destination into a digital display.

[2112] Input: Destination information entered by the user into the digital display device.

[2113] Specific operation: The user enters a destination, such as "Jewelry Section," on the touchscreen or smartphone.

[2114] Output: Destination information is transmitted from the digital display device to the central control unit.

[2115] Step 2:

[2116] The server receives destination information from a digital display device and obtains current congestion information.

[2117] Input: Destination information transmitted from a digital display device.

[2118] Specific operation: The server receives destination information and retrieves real-time updated congestion information from its internal database.

[2119] Output: Destination information and congestion information are stored on the server.

[2120] Step 3:

[2121] The server calculates the optimal route based on destination information and congestion information.

[2122] Input: Destination information, congestion information.

[2123] Specific operation: The server uses Dijkstra's algorithm to calculate the optimal route, taking into account the current congestion.

[2124] Output: Optimal route information is generated.

[2125] Step 4:

[2126] The server sends the calculated optimal route to a digital display device.

[2127] Input: Optimal route information.

[2128] Specific operation: The server generates a data packet to send the optimal route to the digital display device and transmits it using HTTPS communication.

[2129] Output: Optimal route information is displayed on a digital display device.

[2130] Step 5:

[2131] The user confirms the optimal route displayed on the digital display and begins to move.

[2132] Input: Optimal route information displayed on a digital display device.

[2133] Specific action: The user checks the route directions displayed on their smartphone or tablet and follows them.

[2134] Output: The user moves towards the destination.

[2135] Step 6:

[2136] The server analyzes the user's purchase history and inventory information to generate a list of recommended products.

[2137] Input: User purchase history data, current inventory information.

[2138] Specific operation: The server uses generative artificial intelligence to analyze purchase history and inventory information, and lists the most suitable recommended products for the user. Prompt messages are input into the AI ​​model to obtain the results.

[2139] Output: A list of recommended products is generated.

[2140] Step 7:

[2141] The server sends a list of recommended products to a digital display device.

[2142] Input: Recommended product list.

[2143] Specific operation: Generate a data packet to send the recommended product list generated by the server to a digital display device, and send it using HTTPS communication.

[2144] Output: Recommended product information is displayed on the digital display device.

[2145] Step 8:

[2146] The server retrieves the latest event information from the database and notifies the user based on their location.

[2147] Input: Event information database, user location information.

[2148] Specific operation: The server retrieves the latest event information from the database and generates appropriate event notifications based on the user's current location.

[2149] Output: Event notification information is generated.

[2150] Step 9:

[2151] The server sends the generated event notification information to the digital display device.

[2152] Input: Event notification information.

[2153] Specific operation: The server generates a data packet to send event notification information to a digital display device and transmits it using HTTPS communication.

[2154] Output: An event notification is displayed on the digital display device.

[2155] Step 10:

[2156] Users select products in an unmanned store and scan the product barcodes on a digital display.

[2157] Input: Product barcode information.

[2158] Specific operation: Scan the barcode of the product selected by the user onto the digital display device.

[2159] Output: Product barcode information is transmitted from the digital display device to the server.

[2160] Step 11:

[2161] The server receives the barcode information and verifies the inventory status.

[2162] Input: Product barcode information.

[2163] Specific operation: The server receives barcode information and retrieves inventory information for the corresponding product from its internal database.

[2164] Output: Inventory confirmation information is generated.

[2165] Step 12:

[2166] The user selects a payment method and enters payment information into the digital display.

[2167] Input: Payment information.

[2168] Specific operation: The user selects a payment method such as credit card or electronic money and enters the necessary information into the digital display device.

[2169] Output: Payment information is sent from the digital display device to the server.

[2170] Step 13:

[2171] The server processes the payment based on the payment information it receives.

[2172] Input: Payment information.

[2173] Specific operation: The server sends payment information to the payment system (e.g., Stripe or PayPal API) and processes the payment.

[2174] Output: Payment completion information is generated.

[2175] Step 14:

[2176] The server updates the inventory and notifies the digital display device of the purchase completion.

[2177] Input: Payment completion information, inventory information.

[2178] Specific operation: The server updates the inventory database and sends the purchase completion information to the digital display device.

[2179] Output: A purchase completion notification is displayed on the digital display.

[2180] These steps enable users to enjoy efficient and comfortable shopping at physical stores.

[2181] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[2182] This invention provides a system that offers optimal information and services to users by linking a digital display device, a central control unit, real-time data, and an emotion engine. Specifically, it includes a digital display device for users to select destinations and services, a central control unit that calculates emotion recognition and optimal routes and recommended products based on received information, and a system that manages congestion status, inventory status, and the user's emotional state in real time.

[2183] System processing and specific examples

[2184] 1. Route guidance processing

[2185] The user enters their desired seat number and destination on a digital display.

[2186] The terminal transmits that information to the central control unit.

[2187] The server (central control unit) calculates the optimal route by referring to real-time congestion information and the user's emotional state.

[2188] The terminal displays the calculated route to the user.

[2189] For example, if a user wants to know the location of "seat 102," they enter the seat number into the terminal. The central control unit calculates a more comfortable route (for example, a route that avoids congestion) based on congestion information and the user's stress level, and sends it to the terminal. The terminal then displays that route to the user on a map or in text.

[2190] 2. Recommended Products

[2191] The user selects "recommended items" on a digital display device.

[2192] The terminal sends this request to the central control unit.

[2193] The server (central control unit) analyzes the user's past purchase history, current inventory information, and the user's emotional state as recognized by the emotion engine, and generates an optimal list of recommended products.

[2194] The device displays that information to the user.

[2195] As a concrete example, when a user enters "I want to know what products are recommended" into the terminal, the server analyzes past purchase data and sentiment data and presents "currently popular T-shirts" as products that are likely to please the user. The terminal then displays this information to the user.

[2196] 3. Event Announcement

[2197] The server constantly manages the latest event information and sends updated information to the digital display device.

[2198] The device receives event information in real time and automatically sends event notifications based on specific times and conditions (e.g., when a user is near the device).

[2199] As a specific example, when a user approaches a digital display device, the device detects the user's level of excitement from their facial expression and displays a pop-up message saying, "The live event starts at 3 PM on Stage C."

[2200] 4. Purchase processing at unmanned stores

[2201] The user selects the items they want to purchase at the unmanned store and scans the barcodes of the items using a digital display device.

[2202] The terminal transmits barcode information to the central control unit to verify product inventory information.

[2203] The server (central control unit) updates the inventory status of products based on inventory information and prepares for payment processing.

[2204] The user selects a payment method and enters payment information, such as credit card details, into the terminal.

[2205] The terminal sends payment information to the server and executes the payment process.

[2206] The server confirms the completion of the payment and sends the purchase completion information to the terminal.

[2207] The device displays a purchase completion message and receipt to the user.

[2208] As a concrete example, a user selects a T-shirt at an unmanned store and scans the barcode at a terminal. The server checks the inventory information, and the user pays with a credit card. Once the payment is complete, a receipt is displayed on the terminal.

[2209] 5. Utilizing the Emotional Engine

[2210] The device senses the user's facial expressions and voice, and analyzes them using an emotion engine.

[2211] The server receives information from the emotion engine and incorporates it into recommended products and route guidance information.

[2212] The device provides real-time alternatives and support information when the user expresses stress or dissatisfaction.

[2213] For example, when a user approaches a digital display device, the emotion engine detects stress from the user's facial expression, and the central control unit suggests less crowded routes or provides information about relaxation areas.

[2214] Integration of each component

[2215] The system of this invention enhances user convenience and experiential value through the close coordination of a digital display device, a central control unit, real-time data, and an emotion engine. The central control unit manages all data centrally and updates information in real time, ensuring that users always receive the latest and most accurate information. Furthermore, by utilizing the emotion engine, it is possible to provide user-optimized recommendation services.

[2216] As described above, the present invention provides an efficient information guidance system for realizing a comfortable user experience in large domes and stadiums.

[2217] The following describes the processing flow.

[2218] Route guidance processing combined with an emotion engine

[2219] Step 1:

[2220] The user enters their destination (e.g., seat number "102") into the digital display.

[2221] Step 2:

[2222] The terminal receives the entered destination information and transmits it to the central control unit.

[2223] Step 3:

[2224] The terminal analyzes the user's facial expressions and voice using an emotion engine and transmits the results to a central control unit.

[2225] Step 4:

[2226] The server (central control unit) receives destination information and sentiment information, while simultaneously referencing real-time congestion information and geographical information to calculate the optimal route.

[2227] Step 5:

[2228] The server transmits the calculated optimal route information to a digital display device.

[2229] Step 6:

[2230] The terminal displays the received optimal route information on the user interface and provides visual or audio guidance to the user.

[2231] A guide to recommended products that combine an emotional engine.

[2232] Step 1:

[2233] The user selects "Recommended Items" from the menu on the digital display device.

[2234] Step 2:

[2235] The terminal sends this request to the central control unit.

[2236] Step 3:

[2237] The terminal analyzes the user's facial expressions and voice using an emotion engine and transmits the results to a central control unit.

[2238] Step 4:

[2239] The server (central control unit) analyzes the user's past purchase history, inventory information, and the user's emotional state as recognized by the emotion engine, and generates an optimal list of recommended products.

[2240] Step 5:

[2241] The server sends the generated list of recommended items to a digital display device.

[2242] Step 6:

[2243] The terminal displays the received product information on its user interface, visually presenting recommended products to the user.

[2244] Event announcements combined with an emotional engine

[2245] Step 1:

[2246] The server constantly manages the latest event information and sends updated information to the digital display device.

[2247] Step 2:

[2248] The device receives event information in real time and activates the emotion engine based on specific times and conditions (e.g., when a user approaches the device).

[2249] Step 3:

[2250] The device analyzes the user's facial expressions and voice using an emotion engine and displays event notifications based on the results.

[2251] Step 4:

[2252] Users can view event information displayed on their devices and obtain detailed information to participate in events that interest them.

[2253] Purchase processing in unmanned stores using an emotion engine.

[2254] Step 1:

[2255] The user selects the items they want to purchase at the unmanned store and scans the barcodes of the items using a digital display device.

[2256] Step 2:

[2257] The terminal transmits barcode information to the central control unit to verify product inventory information.

[2258] Step 3:

[2259] The terminal analyzes the user's facial expressions and voice using an emotion engine and transmits the results to a central control unit.

[2260] Step 4:

[2261] The server (central control unit) updates the inventory status of products based on inventory information and prepares for payment processing.

[2262] Step 5:

[2263] The user selects a payment method and enters payment information, such as credit card details, into a digital display device.

[2264] Step 6:

[2265] The terminal transmits payment information to the central control unit and executes the payment process.

[2266] Step 7:

[2267] The server confirms the completion of the payment and sends the purchase completion information to the digital display device.

[2268] Step 8:

[2269] The device displays a purchase completion message and receipt to the user.

[2270] Real-time support utilizing an emotion engine

[2271] Step 1:

[2272] The device constantly analyzes the user's facial expressions and voice using an emotion engine to detect signs of stress and dissatisfaction.

[2273] Step 2:

[2274] The server receives information detected by the emotion engine and generates appropriate support information and alternatives in real time.

[2275] Step 3:

[2276] The device displays support information and alternatives tailored to the user's emotional state, and provides specific guidance.

[2277] This invention enables the realization of a system that analyzes the user's emotional state and provides optimal guidance and information based on that data.

[2278] (Example 2)

[2279] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[2280] Conventional information guidance systems using digital display devices and central control units lack the ability to provide optimal information and services that take into account the user's emotional state, and there is a need to improve the user experience. Furthermore, it has been difficult to analyze the user's emotional state in real time and provide information and services accordingly, which has prevented a sufficient improvement in user convenience and satisfaction.

[2281] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[2282] In this invention, the server includes means for using an emotion recognition engine that senses the user's facial expressions and voice using a digital display device and analyzes the user's emotional state; means for referencing the information from the emotion recognition engine in a central control unit and optimizing route guidance and product recommendations based on the user's emotional state; and means for providing recommended product information based on the user's hobbies and preferences. This enables the provision of optimal information and services that correspond to the user's emotional state.

[2283] A "digital display device" is an electronic device used by users to input information or to visually confirm information.

[2284] A "central control unit" is a key computer system that centrally manages data across the entire system, analyzes information, and provides optimal services.

[2285] An "emotion recognition engine" is software or hardware that analyzes a user's facial expressions and voice to determine the user's emotional state.

[2286] "Real-time congestion information" refers to the latest data showing the current congestion status of a specific location or route.

[2287] A "Geographic Information System" is a system that collects and analyzes data related to geospatial information and displays it visually.

[2288] "Generative artificial intelligence" is an artificial intelligence technology that analyzes large amounts of data and automatically generates useful information and recommendations.

[2289] "Recommended product information" refers to product information that is considered interesting to the user based on ...

Claims

1. Including a digital display device, a means for the user to input the desired destination, Includes a central control unit, means for receiving destination information from the digital display device and calculating the optimal route, A means of providing users with optimal route guidance using a geographic information system that includes real-time congestion information, A system including means for communicating with the central control unit.

2. A means of providing recommended product information based on the user's tastes and preferences, The means includes a generative artificial intelligence that analyzes past purchase data and inventory information using the aforementioned central control unit, The aforementioned digital display device includes means for displaying recommended product information, The system according to claim 1, including the following:

3. A means of managing a database containing the latest event schedule information, A means of automatically displaying event announcements based on the user's proximity or specific conditions, A digital display device including means for scanning products and processing payments in an unmanned store, A means including a central control unit that updates inventory information and notifies the user of purchase completion information, The system according to claim 1, including the following:

Images