system

Abstract

We provide the system. [Solution] A means of collecting and storing users' payment history in a database, A method for analyzing payment history collected using machine learning algorithms and extracting user purchasing patterns, A means for generating product recommendations based on user-specific purchasing patterns, A means of collecting the latest market activity information and making the best possible proposals to users, A means of analyzing natural language input from users and providing appropriate information, A means of selecting and notifying market activity information when arriving at a specific facility using location information, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor and includes steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] With the rapid expansion of the cashless payment market, consumers are required to make optimal choices from a vast array of goods and services. However, it is not easy for individual consumers to make efficient and quick optimal purchasing decisions, which instead require a great deal of time and effort. In addition, it is also difficult for consumers to effectively utilize information and campaigns according to their own purchasing behaviors and preferences. Moreover, it is difficult for retailers to propose goods and services to consumers at an appropriate timing, which hinders sales growth. The object of this invention is to solve these problems faced by consumers and retailers.

Means for Solving the Problems

[0005] This invention provides a means for collecting and storing users' payment history in a database, utilizing technologies related to cashless payment methods. Next, it provides a means for identifying user-specific preferences by applying machine learning algorithms to this collected data and extracting user purchasing patterns. Furthermore, it includes a means for generating personalized product recommendations based on the results of this pattern analysis. It also incorporates a means for collecting the latest campaign information obtained from partner stores and service providers and providing it in a form optimized for the user's purchasing behavior. In addition, it solves the problem by including a means for using natural language processing technology to analyze natural language input from users and provide appropriate information immediately.

[0006] "User payment history" refers to records of transactions made by a user through a cashless payment system, including information such as purchased items, transaction amount, transaction date and time, and store information.

[0007] A "machine learning algorithm" is a set of mathematical procedures used to perform pattern recognition and prediction using data, and is a technology used to analyze user purchasing behavior and generate predictions.

[0008] "Purchase patterns" refer to the purchasing behaviors and preference tendencies that users have repeatedly shown in the past, and are indicators of future purchasing behavior and preferences that can be predicted based on these patterns.

[0009] "Personalized product recommendations" refer to information provision aimed at suggesting optimized products and services based on an individual user's past purchase history and preferences.

[0010] "Campaign information" refers to information offered by partner stores and service providers to promote purchases, such as discounts, benefits, and point rewards for a specific period.

[0011] "Natural language processing" is a technology that enables computers to understand, analyze, and generate language that humans use naturally, and is used for user interaction and understanding input. [Brief explanation of the drawing]

[0012] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of the data processing device and smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, when an emotion engine is combined. [Figure 14]It is a sequence diagram showing the processing flow of a data processing system in Application Example 2 when a sentiment engine is combined.

Embodiments for Carrying Out the Invention

[0013] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0014] First, the terms used in the following description will be explained.

[0015] In the following embodiments, a numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0016] In the following embodiments, a numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0017] In the following embodiments, a numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, and the like.

[0018] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0019] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0020] [First Embodiment]

[0021] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0022] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0023] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0024] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0025] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0026] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0027] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0028] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0029] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0030] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0031] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0032] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0033] This invention provides an AI agent that works in conjunction with a cashless payment system to support users' purchasing behavior. The system consists of the user's smartphone, servers located in the cloud or data center, and a communication system that interacts with them.

[0034] 1. Data collection and storage

[0035] Each time a user makes a cashless payment, the terminal retrieves payment data such as information about the purchased items, the amount, date and time, and the store where the transaction took place.

[0036] The device sends this data to the server via secure communication, and the server stores it in a database. The data is encrypted, and privacy is protected.

[0037] 2. Data Analysis and Recommendation Generation

[0038] The server periodically analyzes the accumulated data to model user purchasing patterns. This process utilizes machine learning algorithms such as collaborative filtering and content-based filtering.

[0039] The server generates personalized product recommendations and service suggestions based on the analysis results. These are individually optimized to reflect the user's preferences and purchase history.

[0040] 3. Integrating campaign information

[0041] The server collects and updates the latest campaign information sent from partner stores and service providers and stores it in a database.

[0042] The server matches the user's purchasing patterns, selects campaign information at the appropriate time, and sends it to the device as an optimized suggestion.

[0043] 4. Natural Language Interface

[0044] Users can input questions and requests verbally using voice assistants and chatbot functions.

[0045] The terminal sends these natural language inputs to the server, which uses natural language processing techniques to analyze them and generate appropriate responses.

[0046] The server-generated response is returned to the terminal and smoothly provided to the user.

[0047] 5. Operation of specific examples

[0048] In a real-world scenario, when a user purchases groceries at a supermarket, the terminal collects payment information and sends it to a server. Based on this information, the server provides product recommendations, combining discount information for frequently purchased items. This information is notified to the user from the terminal before the purchase at the store. Furthermore, during promotional periods, the system suggests ways to maximize point rewards.

[0049] Through this form, the present invention enhances the user's purchasing experience and simultaneously supports the marketing strategies of retailers.

[0050] The following describes the processing flow.

[0051] Step 1:

[0052] When a user makes a cashless payment, the terminal acquires payment-related data. This data includes details of the purchased items, the purchase amount, the store where the purchase was made, and the date and time.

[0053] Step 2:

[0054] The terminal encrypts the acquired payment data and transfers it to the server using a secure communication protocol. During this process, user privacy is guaranteed.

[0055] Step 3:

[0056] The server stores the received payment data in a database. The stored data is organized by user and prepared for later analysis.

[0057] Step 4:

[0058] The server periodically analyzes the stored data and uses machine learning algorithms to extract user purchasing patterns. This analysis is performed using a combination of collaborative filtering and content-based filtering.

[0059] Step 5:

[0060] The server generates personalized product and service recommendations for users based on the analysis results. These recommendations are designed to match the individual user's preferences.

[0061] Step 6:

[0062] The server collects campaign information from partner stores and service providers and updates its database. This information is used to identify the most relevant campaigns based on the user's purchase history and analysis results.

[0063] Step 7:

[0064] The server sends this optimized campaign information and product recommendations to the device, and the device notifies the user. The notification is immediately accessible to the user via push notifications or other means.

[0065] Step 8:

[0066] When a user enters a question or request through a voice assistant or chatbot function, the device sends that information to the server.

[0067] Step 9:

[0068] The server uses natural language processing techniques to analyze user input and generate appropriate responses. Care is taken to ensure that these responses address the user's questions.

[0069] Step 10:

[0070] The server generates the response, which is sent to the terminal, and the terminal then presents it to the user. This allows the user to obtain the necessary information in real time.

[0071] (Example 1)

[0072] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0073] Traditional cashless payment systems have been unable to effectively utilize users' purchase history, making it difficult to optimize the purchasing experience and provide appropriate product recommendations. Furthermore, there has been a lack of methods to effectively match campaign information with users' purchasing patterns, which has limited the provision of real-time information to users.

[0074] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0075] In this invention, the server includes means for collecting and storing the user's electronic payment history in a storage device; means for analyzing the collected electronic payment history using a machine learning algorithm and extracting the user's purchasing patterns; means for generating product recommendations based on the user's unique purchasing patterns using a generative AI model; means for obtaining the latest campaign information from data providers and making optimal suggestions to the user; and means for analyzing natural language input from the user and providing appropriate information based on prompts. This enables the provision of a personalized experience based on the user's purchasing history and the real-time provision of appropriate product recommendations and campaign information.

[0076] "User" refers to an individual or legal entity that uses a service or system.

[0077] "Electronic payment" refers to a transaction in which the price of goods or services is paid by electronic means.

[0078] A "storage device" refers to a hardware or software system for holding or storing information.

[0079] A "machine learning algorithm" refers to a computational method used to extract patterns from data and make predictions or decisions.

[0080] "Purchase patterns" refer to a series of purchasing behavioral trends derived from a user's past purchase history.

[0081] A "generative AI model" refers to an artificial intelligence computational model designed to make personalized recommendations and predictions based on data.

[0082] "Product recommendation" refers to products or services that suggest purchases based on the user's preferences and purchase history.

[0083] A "data provider" refers to an organization or company that provides information to the system.

[0084] "Campaign information" refers to information that includes details of discounts, benefits, and sales aimed at promoting sales.

[0085] "Natural language input" refers to an interface that allows users to input information into the system using their usual language.

[0086] A "prompt" refers to the text input that the generating AI uses for analysis based on user instructions or requests.

[0087] The system according to this invention provides comprehensive electronic payment support to improve the user's purchasing experience. This system mainly consists of a user's terminal, a server, and a communication system connecting them.

[0088] Data collection and storage

[0089] The terminal collects information about the purchased items, including the price, date and time, and store details, when a user makes an electronic payment. This is done using hardware such as NFC readers and barcode scanners. The terminal encrypts this data and sends it securely to the server, which then stores the data using a database management system (DBMS).

[0090] Data analysis and recommendation generation

[0091] The server uses programming languages ​​such as Python and R to execute machine learning algorithms and analyze user purchasing patterns. It then leverages generative AI models to automatically generate optimized product and service recommendations based on each user's unique purchase history.

[0092] Campaigns and Interactions

[0093] The server collects the latest campaign information from data providers via APIs and updates its database. It then combines this information with analyzed purchasing patterns to provide users with the most relevant information. Users can make natural language inquiries using voice assistants or text-based chatbots. The server interprets these inquiries using natural language processing technology and sends the most appropriate response back to the user's device.

[0094] Specific example

[0095] As a concrete example, consider a scenario where a user purchases groceries at a supermarket. The terminal uses an NFC reader to collect payment information and transfer it to a server. Based on this data, the server generates recommendations, including new discount information on items the user frequently purchases, and notifies the user via the terminal a few minutes before purchase. Additionally, if the user uses a voice assistant to input a prompt such as "Tell me about this weekend's sales," the server analyzes the prompt and returns appropriate recommendations to assist the user's purchasing plan.

[0096] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0097] Step 1: Data Collection

[0098] The terminal uses an NFC reader or barcode scanner to obtain information about the purchased items, amount, date and time, and store where the transaction took place when a user makes an electronic payment. It receives data about the user's payment behavior as input. This data is encrypted and transmitted to the server over the network. The output is formalized purchase data sent to the server.

[0099] Step 2: Save Data

[0100] The server stores the received data in a database. It receives encrypted data sent from the terminal as input. A database management system (DBMS) is used to store the data while maintaining its integrity. The output is the organized data stored in the DPMS for use in subsequent processing steps.

[0101] Step 3: Data Analysis

[0102] The server analyzes the stored data. It retrieves purchase history data stored in the database as input. Using Python or R, it applies machine learning algorithms to perform pattern recognition. By constructing purchase patterns, it reveals user preferences and trends and predicts expected purchasing behavior. The output is personalized purchase pattern information.

[0103] Step 4: Recommendation Generation

[0104] The server generates product recommendations using a generative AI model. The input is analyzed purchasing pattern information, which the generative AI model uses to provide optimal product recommendations to the user. This utilizes collaborative filtering and content-based filtering techniques. The output is optimized product recommendations designed to encourage future purchasing behavior.

[0105] Step 5: Integrating campaign information

[0106] The server retrieves the latest campaign information from data providers via API and updates the database. It receives campaign data from partners as input. This data is then adjusted to match purchasing patterns to enable optimal recommendations. The output is optimized campaign information for the user.

[0107] Step 6: User Interaction

[0108] Users ask questions in natural language through voice assistants or chatbots. The user's questions or requests are sent as input, either as text or voice. The device sends this to a server, which uses natural language processing techniques to interpret the prompt. An appropriate answer is generated and provided to the user as output. For example, in response to a prompt like "Tell me your recommended products for this weekend," the server might recommend a specific product list.

[0109] (Application Example 1)

[0110] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0111] With the widespread adoption of electronic payments in modern society, consumers are expected to be able to smoothly make the best choices from a vast array of products. However, providing personalized product and market activity information in real time amidst the enormous amount of data is challenging. Conventional purchasing support systems have limited contribution to improving the purchasing experience because product recommendations based on individual users' purchase history are not sufficiently personalized, and there is a lack of on-the-spot information provision utilizing market activity data from facilities.

[0112] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0113] In this invention, the server includes means for collecting and storing the user's payment history in a database, means for extracting the user's purchasing patterns using a machine learning algorithm, and means for selecting and notifying market activity information when the user arrives at a specific facility using location information. This makes it possible to provide product recommendations and market activity information tailored to the user's individual needs in real time, thereby assisting in purchasing decisions.

[0114] "User payment history" refers to information about all electronic transactions conducted by a user, including data such as purchased items, transaction amount, date and time, and the store where the transaction took place.

[0115] A "database" is an electronic information system used to store users' payment history, analysis results, and the latest market activity information.

[0116] A "machine learning algorithm" is a mathematical method that analyzes a user's past behavior history to extract patterns and use them for prediction and recommendation.

[0117] "Purchase patterns" refer to consumer behavior trends, such as what kinds of products a user buys and how often.

[0118] "Product recommendations" refer to a list of products and services suggested based on a user's purchase history and preferences, presenting consumers with suitable options.

[0119] "Latest market activity information" refers to timely sales promotion information such as promotions, special offers, and discount information provided by partner companies and retailers.

[0120] "Natural language input" refers to instructions and questions given by users using natural language, that is, everyday language, in the form of voice or text.

[0121] "Natural language processing technology" is an information technology that uses computers to analyze, understand, and generate human language.

[0122] "Location information" refers to data that indicates the physical location where a device is currently located, and is often obtained using GPS or beacon technology.

[0123] A "notification" is a method of communication sent from a system to a user, and is usually a message that appears as a pop-up on the device screen.

[0124] The system for realizing this invention consists of a user's smart device, a server in the cloud, and a communication network. When a user makes an electronic payment using their smart device, the device acquires payment history data and transmits it to the server in the cloud via secure communication. Cloud platforms such as Amazon Web Services (AWS®) and Microsoft Azure® can be used for the server.

[0125] The server receives this data and stores it in a database. This database is typically built using services such as Google Cloud Firestore or Amazon DynamoDB. The server implements machine learning algorithms using Python and Scikit-learn to analyze the user's purchase history and model their purchasing patterns.

[0126] When a user's smart device detects, via location services, that it has reached a specific location, the server generates optimal product recommendations and market activity information related to that location, based on purchase history and market activity data. Firebase Cloud Messaging or Apple Push Notification Service can be used to notify the user at this time.

[0127] When a user makes a query in natural language, the device sends this input to a server, which then parses it using Google Cloud Natural Language API or Amazon Comprehend, running in the cloud, to generate an appropriate response.

[0128] As a concrete example, when a user arrives at a supermarket, the server recommends discounted items and items with point rewards in real time. In this way, users can obtain accurate information before making a purchase, improving their shopping experience.

[0129] An example of a prompt for a generative AI model is, "Based on user B's recent purchase history, suggest available discounts and recommended products at the supermarket today."

[0130] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0131] Step 1:

[0132] When a user makes an electronic payment using a smart device, the device acquires transaction data (product information, amount, time, location, etc.). This is the input. This data is temporarily encrypted within the device and sent to a server in the cloud using secure communication; this is the output.

[0133] Step 2:

[0134] Transaction data received by the server is stored in a database such as Google Cloud Firestore or Amazon DynamoDB. The input is the transmitted transaction data, and the output is the stored data. The server encrypts and stores this data, ensuring its security.

[0135] Step 3:

[0136] The server analyzes data collected periodically. The input consists of a vast amount of stored payment history data. The server uses Python and Scikit-learn to execute machine learning algorithms and extract user purchasing patterns. The output is user-specific purchasing pattern data.

[0137] Step 4:

[0138] When a user arrives at a facility, the device determines its location and sends it to the server. The input is location data, and the output is facility information that the server searches for based on that location. The server then creates a feed containing the latest market activity information related to the facility.

[0139] Step 5:

[0140] The server generates product recommendations for individual users based on purchasing patterns and market activity information for the facility. The input consists of extracted purchasing pattern data and market activity information. The output is a set of user-optimized product recommendation lists and market activity information.

[0141] Step 6:

[0142] The server sends the generated product recommendation information to the smart device. The input is the recommendation information from the server, and the output is a notification displayed on the user's device. The device uses Firebase Cloud Messaging to send the notification to the user.

[0143] Step 7:

[0144] When a user makes a voice inquiry, the device sends this as voice data to the server. The input is the user's voice input. The server uses the Google Cloud Natural Language API to perform natural language processing and generate and output a response to the user's question. The response is then sent back to the device and provided to the user.

[0145] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0146] This invention efficiently supports users' purchasing behavior using an AI agent that works in conjunction with a cashless payment system. In particular, this invention incorporates an emotion engine to provide a sophisticated experience that also takes into account the user's emotions. Specific embodiments are shown below.

[0147] 1. Data collection and storage

[0148] Each time the terminal performs a cashless payment, it collects information such as the purchased items, transaction amount, transaction date and time, and the store where the transaction took place.

[0149] This data is transmitted from the terminal to the server via secure communication and stored in the database in an encrypted state.

[0150] 2. Analysis of purchasing patterns

[0151] The server analyzes payment history stored in the database using machine learning algorithms to model user purchasing patterns.

[0152] The generated purchasing patterns serve as fundamental data for providing affordable product recommendations.

[0153] 3. Emotion recognition by an emotion engine

[0154] The device sends data extracted from the user's voice and text to the server.

[0155] The server uses an emotion engine to recognize the user's emotions from the transmitted data and analyze their state. This emotional state is then reflected in the suggested product information and services.

[0156] 4. Optimization of product recommendations and campaign information

[0157] The server comprehensively considers information obtained from purchasing patterns and emotion recognition to generate product recommendations best suited to each individual user.

[0158] The latest campaign information will be added to this, maximizing the user's purchasing experience.

[0159] 5. Interfaces and Natural Language Processing

[0160] Users interact through voice assistants or text chat.

[0161] This natural language input is sent from the terminal to the server, which uses natural language processing technology to analyze the input and provide appropriate information tailored to the user's question.

[0162] For example, if a user feels anxious before making a purchase, the device detects this emotion, and the server uses this information to recommend products or services that will alleviate their stress. Relevant campaign information is also presented, allowing the user to understand the specific benefits and proceed with the purchase with confidence. This immersive experience enhances consumer satisfaction and enables effective marketing for retailers.

[0163] The following describes the processing flow.

[0164] Step 1:

[0165] When a user makes a purchase using a cashless payment system, the terminal collects data related to the purchase. This data includes the purchased items, price, store, and date and time.

[0166] Step 2:

[0167] The terminal encrypts the collected payment data and sends it to the server via a secure channel. The server receives it and securely stores it in its database.

[0168] Step 3:

[0169] The server periodically analyzes payment data in the database and uses machine learning algorithms to analyze user purchasing patterns. This analysis models user preferences and purchasing trends.

[0170] Step 4:

[0171] When a user expresses emotions through voice or text, the device collects the user's voice and text data and sends it to the server.

[0172] Step 5:

[0173] The server uses an emotion engine to analyze received audio and text data and recognize and evaluate the user's emotional state. For example, it estimates emotions such as joy, anxiety, and stress from the user's tone of voice and word choices.

[0174] Step 6:

[0175] The server comprehensively considers the results of the emotional state assessment and purchasing patterns to generate personalized product and service recommendations. In this process, product selection can be tailored to the user's emotions.

[0176] Step 7:

[0177] The server adds the latest campaign information, compiles it in a format optimized for emotional states and purchasing patterns, and sends it to the device.

[0178] Step 8:

[0179] The device notifies the user of the information it receives and presents it through lists, pop-ups, and voice suggestions. This allows the user to easily identify the products and services that are currently most relevant to them.

[0180] Step 9:

[0181] When a user enters a question in natural language, the terminal sends it to the server, which then analyzes it using natural language processing.

[0182] Step 10:

[0183] The server provides quick and accurate answers to user questions by generating appropriate responses and sending them back to the terminal. Users can then use this information to proceed with purchases or obtain additional information.

[0184] (Example 2)

[0185] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0186] In modern consumer behavior analysis, simply recommending products based on past purchase history is insufficient to capture consumers' psychological needs and temporary emotions, making it difficult to provide appropriate product suggestions. In particular, there is a demand for providing sophisticated purchasing experiences that take emotions and psychological states into account, and this challenge needs to be addressed.

[0187] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0188] In this invention, the server includes means for acquiring the user's payment history and storing it in a storage medium, means for analyzing the user's emotional data using an emotion recognition engine, and means for integrating purchase behavior and promotional information based on the emotion recognition results to generate optimal suggestions. This makes it possible to provide optimal product suggestions and promotional information that take into account not only the user's purchasing behavior but also their emotions and psychological state at the time.

[0189] "Payment history" refers to a record of all transactions a user has made, and is a collection of data including product information, amount, date and time, and location of the transaction.

[0190] A "storage medium" is a hardware or software component used to store and manage data and information.

[0191] A "learning model" is an algorithm or program that uses machine learning techniques to analyze data and discover specific patterns or rules.

[0192] "Purchase behavior" refers to data that shows the user's tendencies in selecting and purchasing goods and services, obtained by analyzing their past purchasing activities.

[0193] "Product recommendations" refer to recommendations for appropriate products and services tailored to individual users, based on their purchasing behavior and other information.

[0194] "Promotional information" refers to information that includes details of discounts, campaigns, and promotions designed to boost the sales of products and services.

[0195] "Emotional data" refers to information that indicates a user's psychological state or emotions, extracted from their voice, text, or behavior.

[0196] An "emotion recognition engine" is software or an algorithm used to analyze and identify a user's emotional state from voice, text, or other data.

[0197] "Natural language processing technology" refers to a group of technologies used by computers to understand, interpret, and generate natural human language.

[0198] "Information provision" refers to the act of communicating analysis results and recommendations to users in an appropriate format.

[0199] This invention provides a personalized product suggestion system that takes into account the user's purchasing behavior and emotions. This system is primarily realized through the cooperation of a terminal and a server.

[0200] The device retrieves payment information when a user purchases a product. This information includes the product name, price, date and time, and store of purchase. This data is transmitted to the server via a secure communication protocol (e.g., HTTPS) and securely stored on a storage medium.

[0201] Furthermore, sentiment data is extracted from the user's voice or text input. This process uses speech recognition software to convert speech to text. This data is then sent back to the server via a secure protocol.

[0202] The server analyzes stored payment history using a learning model to understand user purchasing behavior. This analysis utilizes programming languages ​​such as Python and machine learning libraries such as Scikit-learn and TENSORFLOW®. The purchase behavior data obtained through the analysis is integrated with the user's emotions, which are evaluated by an emotion recognition engine.

[0203] Emotion recognition utilizes an emotion recognition engine. This includes commonly available emotion recognition software that precisely identifies the user's emotional state. Based on this emotion data and purchasing patterns, the server uses a generative AI model to generate optimal product suggestions and promotional information.

[0204] The generated product suggestions and promotional information are sent to the user's device in the format that is easiest for them to understand. Natural language processing technology is used to generate prompts in a natural way. Users can receive the information and take necessary actions via voice assistants or text chat.

[0205] For example, if a user says to their device, "I've been feeling stressed lately. Do you have any product recommendations?", the device converts the voice into text and sends it to the server. The server then analyzes the prompt and, based on the user's emotions, suggests relaxation products and offers relevant campaigns to improve the user's purchasing experience.

[0206] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0207] Step 1:

[0208] The device acquires payment information such as product name, price, date and time, and store when a user purchases a product. This information is temporarily stored on the device. This generates a detailed dataset for each user transaction. The input is the user's purchase behavior, and the output is a dataset of transaction information.

[0209] Step 2:

[0210] The terminal transmits the collected transaction information to the server using a secure communication protocol (e.g., HTTPS). The transmitted data is encrypted and stored on the server's storage medium. In this scenario, the input is the transaction information from the terminal, and the output is the encrypted data stored in the server's database.

[0211] Step 3:

[0212] The server analyzes stored transaction history using machine learning algorithms. It extracts purchasing patterns using Python or Scikit-learn. The input is transaction history data, and the output is modeled user purchasing patterns. Specifically, it performs clustering and classification to identify patterns in the data.

[0213] Step 4:

[0214] The device acquires data entered by the user via voice or text. This data reflects the user's emotions. Specifically, when voice input is received, the device uses speech recognition software to convert it into text. The input is the user's voice data, and the output is the converted text data.

[0215] Step 5:

[0216] The terminal sends the converted text data to the server. The server uses an emotion recognition engine to analyze the user's emotions. In this process, the input is the text data from the terminal, and the output is the analyzed emotion information. Specifically, the emotion recognition algorithm identifies the user's emotional state.

[0217] Step 6:

[0218] The server uses a generative AI model to create product suggestions tailored to the user, based on purchase pattern data and emotional information. The input is purchase patterns and emotional information, and the output is customized product suggestions and promotional information. Specifically, it generates optimal product suggestions by comparing purchase history and emotional state.

[0219] Step 7:

[0220] The server sends the generated product suggestions to the terminal, and the user receives this information via voice assistant or text chat. The input is the generated product suggestions, and the output is the information presented to the user. Specifically, natural language processing is used to present the information in a way that is easy for the user to understand.

[0221] (Application Example 2)

[0222] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0223] In modern e-commerce, product recommendations and sales promotions are often based on general purchase history and do not always accurately reflect users' emotions or individual needs. Therefore, methods to improve the user experience are needed. Furthermore, there is a growing demand for interactive, natural language-based dialogue in user interfaces. This is expected to increase consumer satisfaction and enable more effective sales activities for businesses.

[0224] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0225] In this invention, the server includes means for collecting and storing the user's payment history in an information storage unit, means for analyzing the collected payment history using machine learning techniques and extracting the user's transaction patterns, and means for generating product recommendations based on the user's unique purchasing patterns and emotional state. This enables personalized product recommendations that take into account the user's emotional state.

[0226] "Payment history" is a record of information about all payment transactions a user has made in the past.

[0227] An "information storage unit" is a computer system or database for securely and systematically storing collected data and information over a long period of time.

[0228] "Machine learning techniques" are algorithms that automatically learn useful patterns and knowledge from data to perform predictions and classifications.

[0229] "Transaction patterns" refer to the regularities and trends in users' purchasing and payment behavior, and are characteristics of behavior extracted through analysis.

[0230] "Emotional state" refers to information that indicates the psychological and emotional situation a user is experiencing at a particular point in time.

[0231] "Product recommendation" refers to the act or process of presenting a specific product to a user based on their characteristics and behavioral history.

[0232] "Sales promotion information" refers to marketing data that includes campaigns and discount information designed to encourage users to purchase products.

[0233] The server stores payment history data transmitted from the user's terminal in its information storage unit and analyzes this data using machine learning techniques to extract the user's transaction patterns. This analysis allows for a clear understanding of regularities and trends in the user's purchasing behavior. Furthermore, the server recognizes the user's emotional state from voice and text data and integrates this information to provide product recommendations. The product recommendation process also takes into account the latest sales promotion information, enabling the provision of the most suitable products and campaign information for the user.

[0234] Users communicate bidirectionally with the server using a natural language interface via their device. The server processes user input in real time using natural language processing technology and quickly responds with information in response to user questions and requests.

[0235] For example, if a user enters "My budget for this month is limited, and I'm worried about whether I really need this product" into a smartphone application, the server will understand that sentiment and recommend relevant, cost-effective options.

[0236] An example of a prompt to input into a generative AI model is: "How can we recommend budget-friendly products when a user is feeling anxious? Which emotion engine and recommendation algorithm should we use?"

[0237] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0238] Step 1:

[0239] The terminal collects the user's payment history data and transmits it to the server using a secure communication method. The input consists of all payment details made by the user, and the output is encrypted payment history data.

[0240] Step 2:

[0241] The server stores the received payment history data in its information storage unit. This allows for centralized management of the information necessary for subsequent data analysis. The input is encrypted payment history data, and the output is the stored data state. The data is imported into a database and securely stored in an encrypted state.

[0242] Step 3:

[0243] The server analyzes payment history data stored in its information storage unit using machine learning techniques. The input is the stored payment history data, and the output is the user's transaction patterns. The server applies machine learning algorithms to this data to extract regularities and characteristics in the user's purchasing behavior.

[0244] Step 4:

[0245] The device collects user voice or text data and sends it to the server. The input for this sentiment data is voice or text input from the user, and the output is the data sent to the server.

[0246] Step 5:

[0247] The server recognizes the user's emotional state from the received voice or text data. The input is voice or text data, and the output is the recognized emotional state. The server applies an emotion recognition algorithm to analyze the user's psychological state.

[0248] Step 6:

[0249] The server generates product recommendations based on the user's trading patterns and emotional state, while also considering the latest sales promotion information. The inputs are trading patterns, emotional state, and sales promotion information, and the output is the most suitable product recommendation for the user. The server integrates this information to select products and deals that are most relevant to the user.

[0250] Step 7:

[0251] The user communicates bidirectionally with the server using a natural language interface through their device. Input is natural language input from the user, and output is corresponding information from the server. The server uses natural language processing technology to answer the user's questions in real time.

[0252] Each step works together to create a system that delivers an optimized shopping experience for the user.

[0253] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0254] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0255] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0256] [Second Embodiment]

[0257] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0258] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0259] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0260] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0261] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0262] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0263] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0264] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0265] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0266] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0267] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0268] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0269] This invention provides an AI agent that works in conjunction with a cashless payment system to support users' purchasing behavior. The system consists of the user's smartphone, servers located in the cloud or data center, and a communication system that interacts with them.

[0270] 1. Data collection and storage

[0271] Each time a user makes a cashless payment, the terminal retrieves payment data such as information about the purchased items, the amount, date and time, and the store where the transaction took place.

[0272] The device sends this data to the server via secure communication, and the server stores it in a database. The data is encrypted, and privacy is protected.

[0273] 2. Data Analysis and Recommendation Generation

[0274] The server periodically analyzes the accumulated data to model user purchasing patterns. This process utilizes machine learning algorithms such as collaborative filtering and content-based filtering.

[0275] The server generates personalized product recommendations and service suggestions based on the analysis results. These are individually optimized to reflect the user's preferences and purchase history.

[0276] 3. Integrating campaign information

[0277] The server collects and updates the latest campaign information sent from partner stores and service providers and stores it in a database.

[0278] The server matches the user's purchasing patterns, selects campaign information at the appropriate time, and sends it to the device as an optimized suggestion.

[0279] 4. Natural Language Interface

[0280] Users can input questions and requests verbally using voice assistants and chatbot functions.

[0281] The terminal sends these natural language inputs to the server, which uses natural language processing techniques to analyze them and generate appropriate responses.

[0282] The server-generated response is returned to the terminal and smoothly provided to the user.

[0283] 5. Operation of specific examples

[0284] As a practical scenario, when a user purchases food at a supermarket, the terminal collects payment information and sends it to the server. Based on this information, the server makes product recommendations by combining discount information on frequently purchased products. This information is notified to the user from the terminal before purchasing at the store. Furthermore, if it is during a campaign period, a method to receive the maximum point reduction is proposed.

[0285] Through this form, the present invention improves the user's purchase experience and at the same time supports the marketing strategy of retailers.

[0286] The following describes the processing flow.

[0287] Step 1:

[0288] When the user makes a cashless payment, the terminal acquires payment-related data. This data includes details of the purchased products, purchase amount, purchase store, and date and time.

[0289] Step 2:

[0290] The terminal encrypts the acquired payment data and transfers it to the server using a secure communication protocol. At this time, it is ensured that the user's privacy is protected.

[0291] Step 3:

[0292] The server stores the received payment data in the database. The stored data is organized for each user and prepared for later analysis.

[0293] Step 4:

[0294] The server periodically analyzes the stored data and extracts the user's purchase patterns using machine learning algorithms. This analysis is performed by combining collaborative filtering and content-based filtering.

[0295] Step 5:

[0296] The server generates personalized product and service recommendations for users based on the analysis results. These recommendations are designed to match the individual user's preferences.

[0297] Step 6:

[0298] The server collects campaign information from partner stores and service providers and updates its database. This information is used to identify the most relevant campaigns based on the user's purchase history and analysis results.

[0299] Step 7:

[0300] The server sends this optimized campaign information and product recommendations to the device, and the device notifies the user. The notification is immediately accessible to the user via push notifications or other means.

[0301] Step 8:

[0302] When a user enters a question or request through a voice assistant or chatbot function, the device sends that information to the server.

[0303] Step 9:

[0304] The server uses natural language processing techniques to analyze user input and generate appropriate responses. Care is taken to ensure that these responses address the user's questions.

[0305] Step 10:

[0306] The server generates the response, which is sent to the terminal, and the terminal then presents it to the user. This allows the user to obtain the necessary information in real time.

[0307] (Example 1)

[0308] Next, Example 1 will be described. In the following description, the data processing device 12 is referred to as a "server", and the smart glasses 214 are referred to as a "terminal".

[0309] In a conventional cashless payment system, it was difficult to effectively utilize the user's purchase history, optimize the purchase experience, and make appropriate product recommendations. Furthermore, there was a lack of a method to effectively match campaign information with the user's purchase patterns, and the real-time information provision to the user was limited, which was an issue.

[0310] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0311] In this invention, the server includes means for collecting the user's electronic payment history and storing it in a storage device, means for analyzing the collected electronic payment history using a machine learning algorithm and extracting the user's purchase pattern, means for generating product recommendations based on the user-specific purchase pattern using a generated AI model, means for obtaining the latest campaign information from a data provider and making an optimal proposal to the user, and means for analyzing the input in natural language from the user and providing appropriate information based on a prompt. Thereby, a personalized experience based on the user's purchase history can be provided, and real-time provision of appropriate product recommendations and campaign information becomes possible.

[0312] "User" refers to an individual or a corporation that uses a service or a system.

[0313] "Electronic payment" refers to a transaction in which the cost of goods or services is paid by electronic means.

[0314] "Storage device" refers to a hardware or software mechanism for holding or storing information.

[0315] A "machine learning algorithm" refers to a computational method used to extract patterns from data and make predictions or decisions.

[0316] "Purchase patterns" refer to a series of purchasing behavioral trends derived from a user's past purchase history.

[0317] A "generative AI model" refers to an artificial intelligence computational model designed to make personalized recommendations and predictions based on data.

[0318] "Product recommendation" refers to products or services that suggest purchases based on the user's preferences and purchase history.

[0319] A "data provider" refers to an organization or company that provides information to the system.

[0320] "Campaign information" refers to information that includes details of discounts, benefits, and sales aimed at promoting sales.

[0321] "Natural language input" refers to an interface that allows users to input information into the system using their usual language.

[0322] A "prompt" refers to the text input that the generating AI uses for analysis based on user instructions or requests.

[0323] The system according to this invention provides comprehensive electronic payment support to improve the user's purchasing experience. This system mainly consists of a user's terminal, a server, and a communication system connecting them.

[0324] Data collection and storage

[0325] The terminal collects information about the purchased items, including the price, date and time, and store details, when a user makes an electronic payment. This is done using hardware such as NFC readers and barcode scanners. The terminal encrypts this data and sends it securely to the server, which then stores the data using a database management system (DBMS).

[0326] Data analysis and recommendation generation

[0327] The server uses programming languages ​​such as Python and R to execute machine learning algorithms and analyze user purchasing patterns. It then leverages generative AI models to automatically generate optimized product and service recommendations based on each user's unique purchase history.

[0328] Campaigns and Interactions

[0329] The server collects the latest campaign information from data providers via APIs and updates its database. It then combines this information with analyzed purchasing patterns to provide users with the most relevant information. Users can make natural language inquiries using voice assistants or text-based chatbots. The server interprets these inquiries using natural language processing technology and sends the most appropriate response back to the user's device.

[0330] Specific example

[0331] As a concrete example, consider a scenario where a user purchases groceries at a supermarket. The terminal uses an NFC reader to collect payment information and transfer it to a server. Based on this data, the server generates recommendations, including new discount information on items the user frequently purchases, and notifies the user via the terminal a few minutes before purchase. Additionally, if the user uses a voice assistant to input a prompt such as "Tell me about this weekend's sales," the server analyzes the prompt and returns appropriate recommendations to assist the user's purchasing plan.

[0332] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0333] Step 1: Data Collection

[0334] The terminal uses an NFC reader or barcode scanner to obtain information about the purchased items, amount, date and time, and store where the transaction took place when a user makes an electronic payment. It receives data about the user's payment behavior as input. This data is encrypted and transmitted to the server over the network. The output is formalized purchase data sent to the server.

[0335] Step 2: Save Data

[0336] The server stores the received data in a database. It receives encrypted data sent from the terminal as input. A database management system (DBMS) is used to store the data while maintaining its integrity. The output is the organized data stored in the DPMS for use in subsequent processing steps.

[0337] Step 3: Data Analysis

[0338] The server analyzes the stored data. It retrieves purchase history data stored in the database as input. Using Python or R, it applies machine learning algorithms to perform pattern recognition. By constructing purchase patterns, it reveals user preferences and trends and predicts expected purchasing behavior. The output is personalized purchase pattern information.

[0339] Step 4: Recommendation Generation

[0340] The server generates product recommendations using a generative AI model. The input is analyzed purchasing pattern information, which the generative AI model uses to provide optimal product recommendations to the user. This utilizes collaborative filtering and content-based filtering techniques. The output is optimized product recommendations designed to encourage future purchasing behavior.

[0341] Step 5: Integrating campaign information

[0342] The server retrieves the latest campaign information from data providers via API and updates the database. It receives campaign data from partners as input. This data is then adjusted to match purchasing patterns to enable optimal recommendations. The output is optimized campaign information for the user.

[0343] Step 6: User Interaction

[0344] Users ask questions in natural language through voice assistants or chatbots. The user's questions or requests are sent as input, either as text or voice. The device sends this to a server, which uses natural language processing techniques to interpret the prompt. An appropriate answer is generated and provided to the user as output. For example, in response to a prompt like "Tell me your recommended products for this weekend," the server might recommend a specific product list.

[0345] (Application Example 1)

[0346] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0347] With the widespread adoption of electronic payments in modern society, consumers are expected to be able to smoothly make the best choices from a vast array of products. However, providing personalized product and market activity information in real time amidst the enormous amount of data is challenging. Conventional purchasing support systems have limited contribution to improving the purchasing experience because product recommendations based on individual users' purchase history are not sufficiently personalized, and there is a lack of on-the-spot information provision utilizing market activity data from facilities.

[0348] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0349] In this invention, the server includes means for collecting and storing the user's payment history in a database, means for extracting the user's purchasing patterns using a machine learning algorithm, and means for selecting and notifying market activity information when the user arrives at a specific facility using location information. This makes it possible to provide product recommendations and market activity information tailored to the user's individual needs in real time, thereby assisting in purchasing decisions.

[0350] "User payment history" refers to information about all electronic transactions conducted by a user, including data such as purchased items, transaction amount, date and time, and the store where the transaction took place.

[0351] A "database" is an electronic information system used to store users' payment history, analysis results, and the latest market activity information.

[0352] A "machine learning algorithm" is a mathematical method that analyzes a user's past behavior history to extract patterns and use them for prediction and recommendation.

[0353] "Purchase patterns" refer to consumer behavior trends, such as what kinds of products a user buys and how often.

[0354] "Product recommendations" refer to a list of products and services suggested based on a user's purchase history and preferences, presenting consumers with suitable options.

[0355] "Latest market activity information" refers to timely sales promotion information such as promotions, special offers, and discount information provided by partner companies and retailers.

[0356] "Natural language input" refers to instructions and questions given by users using natural language, that is, everyday language, in the form of voice or text.

[0357] "Natural language processing technology" is an information technology that uses computers to analyze, understand, and generate human language.

[0358] "Location information" refers to data that indicates the physical location where a device is currently located, and is often obtained using GPS or beacon technology.

[0359] A "notification" is a method of communication sent from a system to a user, and is usually a message that appears as a pop-up on the device screen.

[0360] The system for realizing this invention consists of a user's smart device, a server in the cloud, and a communication network. When a user makes an electronic payment using their smart device, the device acquires payment history data and transmits it to the server in the cloud via secure communication. Cloud platforms such as Amazon Web Services (AWS) and Microsoft Azure can be used for the server.

[0361] The server receives this data and stores it in a database. This database is typically built using services like Google Cloud Firestore or Amazon DynamoDB. The server uses Python and Scikit-learn to implement machine learning algorithms and analyze the user's purchase history to model their purchasing patterns.

[0362] When a user's smart device detects, via location services, that it has reached a specific location, the server generates optimal product recommendations and market activity information related to that location, based on purchase history and market activity data. Firebase Cloud Messaging or Apple Push Notification Service can be used to notify the user at this time.

[0363] When a user makes a query in natural language, the device sends this input to a server, which then parses it using Google Cloud Natural Language API or Amazon Comprehend, running in the cloud, to generate an appropriate response.

[0364] As a concrete example, when a user arrives at a supermarket, the server recommends discounted items and items with point rewards in real time. In this way, users can obtain accurate information before making a purchase, improving their shopping experience.

[0365] An example of a prompt for a generative AI model is, "Based on user B's recent purchase history, suggest available discounts and recommended products at the supermarket today."

[0366] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0367] Step 1:

[0368] When a user makes an electronic payment using a smart device, the device acquires transaction data (product information, amount, time, location, etc.). This is the input. This data is temporarily encrypted within the device and sent to a server in the cloud using secure communication; this is the output.

[0369] Step 2:

[0370] Transaction data received by the server is stored in a database such as Google Cloud Firestore or Amazon DynamoDB. The input is the transmitted transaction data, and the output is the stored data. The server encrypts and stores this data, ensuring its security.

[0371] Step 3:

[0372] The server analyzes data collected periodically. The input consists of a vast amount of stored payment history data. The server uses Python and Scikit-learn to execute machine learning algorithms and extract user purchasing patterns. The output is user-specific purchasing pattern data.

[0373] Step 4:

[0374] When a user arrives at a facility, the device determines its location and sends it to the server. The input is location data, and the output is facility information that the server searches for based on that location. The server then creates a feed containing the latest market activity information related to the facility.

[0375] Step 5:

[0376] The server generates product recommendations for individual users based on purchasing patterns and market activity information for the facility. The input consists of extracted purchasing pattern data and market activity information. The output is a set of user-optimized product recommendation lists and market activity information.

[0377] Step 6:

[0378] The server sends the generated product recommendation information to the smart device. The input is the recommendation information from the server, and the output is a notification displayed on the user's device. The device uses Firebase Cloud Messaging to send the notification to the user.

[0379] Step 7:

[0380] When a user makes a voice inquiry, the device sends this as voice data to the server. The input is the user's voice input. The server uses the Google Cloud Natural Language API to perform natural language processing and generate and output a response to the user's question. The response is then sent back to the device and provided to the user.

[0381] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0382] This invention efficiently supports users' purchasing behavior using an AI agent that works in conjunction with a cashless payment system. In particular, this invention incorporates an emotion engine to provide a sophisticated experience that also takes into account the user's emotions. Specific embodiments are shown below.

[0383] 1. Data collection and storage

[0384] Each time the terminal performs a cashless payment, it collects information such as the purchased items, transaction amount, transaction date and time, and the store where the transaction took place.

[0385] This data is transmitted from the terminal to the server via secure communication and stored in the database in an encrypted state.

[0386] 2. Analysis of purchasing patterns

[0387] The server analyzes payment history stored in the database using machine learning algorithms to model user purchasing patterns.

[0388] The generated purchasing patterns serve as fundamental data for providing affordable product recommendations.

[0389] 3. Emotion recognition by an emotion engine

[0390] The device sends data extracted from the user's voice and text to the server.

[0391] The server uses an emotion engine to recognize the user's emotions from the transmitted data and analyze their state. This emotional state is then reflected in the suggested product information and services.

[0392] 4. Optimization of product recommendations and campaign information

[0393] The server comprehensively considers information obtained from purchasing patterns and emotion recognition to generate product recommendations best suited to each individual user.

[0394] The latest campaign information will be added to this, maximizing the user's purchasing experience.

[0395] 5. Interfaces and Natural Language Processing

[0396] Users interact through voice assistants or text chat.

[0397] This natural language input is sent from the terminal to the server, which uses natural language processing technology to analyze the input and provide appropriate information tailored to the user's question.

[0398] For example, if a user feels anxious before making a purchase, the device detects this emotion, and the server uses this information to recommend products or services that will alleviate their stress. Relevant campaign information is also presented, allowing the user to understand the specific benefits and proceed with the purchase with confidence. This immersive experience enhances consumer satisfaction and enables effective marketing for retailers.

[0399] The following describes the processing flow.

[0400] Step 1:

[0401] When a user makes a purchase using a cashless payment system, the terminal collects data related to the purchase. This data includes the purchased items, price, store, and date and time.

[0402] Step 2:

[0403] The terminal encrypts the collected payment data and sends it to the server via a secure channel. The server receives it and securely stores it in its database.

[0404] Step 3:

[0405] The server periodically analyzes payment data in the database and uses machine learning algorithms to analyze user purchasing patterns. This analysis models user preferences and purchasing trends.

[0406] Step 4:

[0407] When a user expresses emotions through voice or text, the device collects the user's voice and text data and sends it to the server.

[0408] Step 5:

[0409] The server uses an emotion engine to analyze received audio and text data and recognize and evaluate the user's emotional state. For example, it estimates emotions such as joy, anxiety, and stress from the user's tone of voice and word choices.

[0410] Step 6:

[0411] The server comprehensively considers the results of the emotional state assessment and purchasing patterns to generate personalized product and service recommendations. In this process, product selection can be tailored to the user's emotions.

[0412] Step 7:

[0413] The server adds the latest campaign information, compiles it in a format optimized for emotional states and purchasing patterns, and sends it to the device.

[0414] Step 8:

[0415] The device notifies the user of the information it receives and presents it through lists, pop-ups, and voice suggestions. This allows the user to easily identify the products and services that are currently most relevant to them.

[0416] Step 9:

[0417] When a user enters a question in natural language, the terminal sends it to the server, which then analyzes it using natural language processing.

[0418] Step 10:

[0419] The server provides quick and accurate answers to user questions by generating appropriate responses and sending them back to the terminal. Users can then use this information to proceed with purchases or obtain additional information.

[0420] (Example 2)

[0421] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0422] In modern consumer behavior analysis, simply recommending products based on past purchase history is insufficient to capture consumers' psychological needs and temporary emotions, making it difficult to provide appropriate product suggestions. In particular, there is a demand for providing sophisticated purchasing experiences that take emotions and psychological states into account, and this challenge needs to be addressed.

[0423] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0424] In this invention, the server includes means for acquiring the user's payment history and storing it in a storage medium, means for analyzing the user's emotional data using an emotion recognition engine, and means for integrating purchase behavior and promotional information based on the emotion recognition results to generate optimal suggestions. This makes it possible to provide optimal product suggestions and promotional information that take into account not only the user's purchasing behavior but also their emotions and psychological state at the time.

[0425] "Payment history" refers to a record of all transactions a user has made, and is a collection of data including product information, amount, date and time, and location of the transaction.

[0426] A "storage medium" is a hardware or software component used to store and manage data and information.

[0427] A "learning model" is an algorithm or program that uses machine learning techniques to analyze data and discover specific patterns or rules.

[0428] "Purchase behavior" refers to data that shows the user's tendencies in selecting and purchasing goods and services, obtained by analyzing their past purchasing activities.

[0429] "Product recommendations" refer to recommendations for appropriate products and services tailored to individual users, based on their purchasing behavior and other information.

[0430] "Promotional information" refers to information that includes details of discounts, campaigns, and promotions designed to boost the sales of products and services.

[0431] "Emotional data" refers to information that indicates a user's psychological state or emotions, extracted from their voice, text, or behavior.

[0432] An "emotion recognition engine" is software or an algorithm used to analyze and identify a user's emotional state from voice, text, or other data.

[0433] "Natural language processing technology" refers to a group of technologies used by computers to understand, interpret, and generate natural human language.

[0434] "Information provision" refers to the act of communicating analysis results and recommendations to users in an appropriate format.

[0435] This invention provides a personalized product suggestion system that takes into account the user's purchasing behavior and emotions. This system is primarily realized through the cooperation of a terminal and a server.

[0436] The device retrieves payment information when a user purchases a product. This information includes the product name, price, date and time, and store of purchase. This data is transmitted to the server via a secure communication protocol (e.g., HTTPS) and securely stored on a storage medium.

[0437] Furthermore, sentiment data is extracted from the user's voice or text input. This process uses speech recognition software to convert speech to text. This data is then sent back to the server via a secure protocol.

[0438] The server analyzes stored payment history using a learning model to understand user purchasing behavior. This analysis utilizes programming languages ​​such as Python and machine learning libraries such as Scikit-learn and TensorFlow. The purchase behavior data obtained from the analysis is integrated with the user's emotions, which are evaluated by an emotion recognition engine.

[0439] Emotion recognition utilizes an emotion recognition engine. This includes commonly available emotion recognition software that precisely identifies the user's emotional state. Based on this emotion data and purchasing patterns, the server uses a generative AI model to generate optimal product suggestions and promotional information.

[0440] The generated product suggestions and promotional information are sent to the user's device in the format that is easiest for them to understand. Natural language processing technology is used to generate prompts in a natural way. Users can receive the information and take necessary actions via voice assistants or text chat.

[0441] For example, if a user says to their device, "I've been feeling stressed lately. Do you have any product recommendations?", the device converts the voice into text and sends it to the server. The server then analyzes the prompt and, based on the user's emotions, suggests relaxation products and offers relevant campaigns to improve the user's purchasing experience.

[0442] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0443] Step 1:

[0444] The device acquires payment information such as product name, price, date and time, and store when a user purchases a product. This information is temporarily stored on the device. This generates a detailed dataset for each user transaction. The input is the user's purchase behavior, and the output is a dataset of transaction information.

[0445] Step 2:

[0446] The terminal transmits the collected transaction information to the server using a secure communication protocol (e.g., HTTPS). The transmitted data is encrypted and stored on the server's storage medium. In this scenario, the input is the transaction information from the terminal, and the output is the encrypted data stored in the server's database.

[0447] Step 3:

[0448] The server analyzes stored transaction history using machine learning algorithms. It extracts purchasing patterns using Python or Scikit-learn. The input is transaction history data, and the output is modeled user purchasing patterns. Specifically, it performs clustering and classification to identify patterns in the data.

[0449] Step 4:

[0450] The device acquires data entered by the user via voice or text. This data reflects the user's emotions. Specifically, when voice input is received, the device uses speech recognition software to convert it into text. The input is the user's voice data, and the output is the converted text data.

[0451] Step 5:

[0452] The terminal sends the converted text data to the server. The server uses an emotion recognition engine to analyze the user's emotions. In this process, the input is the text data from the terminal, and the output is the analyzed emotion information. Specifically, the emotion recognition algorithm identifies the user's emotional state.

[0453] Step 6:

[0454] The server uses a generative AI model to create product suggestions tailored to the user, based on purchase pattern data and emotional information. The input is purchase patterns and emotional information, and the output is customized product suggestions and promotional information. Specifically, it generates optimal product suggestions by comparing purchase history and emotional state.

[0455] Step 7:

[0456] The server sends the generated product suggestions to the terminal, and the user receives this information via voice assistant or text chat. The input is the generated product suggestions, and the output is the information presented to the user. Specifically, natural language processing is used to present the information in a way that is easy for the user to understand.

[0457] (Application Example 2)

[0458] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 as the "terminal".

[0459] In modern e-commerce, product recommendations and sales promotions are often based on general purchase history and do not always accurately reflect users' emotions or individual needs. Therefore, methods to improve the user experience are needed. Furthermore, there is a growing demand for interactive, natural language-based dialogue in user interfaces. This is expected to increase consumer satisfaction and enable more effective sales activities for businesses.

[0460] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0461] In this invention, the server includes means for collecting and storing the user's payment history in an information storage unit, means for analyzing the collected payment history using machine learning techniques and extracting the user's transaction patterns, and means for generating product recommendations based on the user's unique purchasing patterns and emotional state. This enables personalized product recommendations that take into account the user's emotional state.

[0462] "Payment history" is a record of information about all payment transactions a user has made in the past.

[0463] An "information storage unit" is a computer system or database for securely and systematically storing collected data and information over a long period of time.

[0464] "Machine learning techniques" are algorithms that automatically learn useful patterns and knowledge from data to perform predictions and classifications.

[0465] "Transaction patterns" refer to the regularities and trends in users' purchasing and payment behavior, and are characteristics of behavior extracted through analysis.

[0466] "Emotional state" refers to information that indicates the psychological and emotional situation a user is experiencing at a particular point in time.

[0467] "Product recommendation" refers to the act or process of presenting a specific product to a user based on their characteristics and behavioral history.

[0468] "Sales promotion information" refers to marketing data that includes campaigns and discount information designed to encourage users to purchase products.

[0469] The server stores payment history data transmitted from the user's terminal in its information storage unit and analyzes this data using machine learning techniques to extract the user's transaction patterns. This analysis allows for a clear understanding of regularities and trends in the user's purchasing behavior. Furthermore, the server recognizes the user's emotional state from voice and text data and integrates this information to provide product recommendations. The product recommendation process also takes into account the latest sales promotion information, enabling the provision of the most suitable products and campaign information for the user.

[0470] Users communicate bidirectionally with the server using a natural language interface via their device. The server processes user input in real time using natural language processing technology and quickly responds with information in response to user questions and requests.

[0471] For example, if a user enters "My budget for this month is limited, and I'm worried about whether I really need this product" into a smartphone application, the server will understand that sentiment and recommend relevant, cost-effective options.

[0472] An example of a prompt to input into a generative AI model is: "How can we recommend budget-friendly products when a user is feeling anxious? Which emotion engine and recommendation algorithm should we use?"

[0473] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0474] Step 1:

[0475] The terminal collects the user's payment history data and transmits it to the server using a secure communication method. The input consists of all payment details made by the user, and the output is encrypted payment history data.

[0476] Step 2:

[0477] The server stores the received payment history data in its information storage unit. This allows for centralized management of the information necessary for subsequent data analysis. The input is encrypted payment history data, and the output is the stored data state. The data is imported into a database and securely stored in an encrypted state.

[0478] Step 3:

[0479] The server analyzes payment history data stored in its information storage unit using machine learning techniques. The input is the stored payment history data, and the output is the user's transaction patterns. The server applies machine learning algorithms to this data to extract regularities and characteristics in the user's purchasing behavior.

[0480] Step 4:

[0481] The device collects user voice or text data and sends it to the server. The input for this sentiment data is voice or text input from the user, and the output is the data sent to the server.

[0482] Step 5:

[0483] The server recognizes the user's emotional state from the received voice or text data. The input is voice or text data, and the output is the recognized emotional state. The server applies an emotion recognition algorithm to analyze the user's psychological state.

[0484] Step 6:

[0485] The server generates product recommendations based on the user's trading patterns and emotional state, while also considering the latest sales promotion information. The inputs are trading patterns, emotional state, and sales promotion information, and the output is the most suitable product recommendation for the user. The server integrates this information to select products and deals that are most relevant to the user.

[0486] Step 7:

[0487] The user communicates bidirectionally with the server using a natural language interface through their device. Input is natural language input from the user, and output is corresponding information from the server. The server uses natural language processing technology to answer the user's questions in real time.

[0488] Each step works together to create a system that delivers an optimized shopping experience for the user.

[0489] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0490] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0491] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0492] [Third Embodiment]

[0493] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0494] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0495] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0496] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0497] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0498] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0499] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0500] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0501] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0502] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0503] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0504] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0505] This invention provides an AI agent that works in conjunction with a cashless payment system to support users' purchasing behavior. The system consists of the user's smartphone, servers located in the cloud or data center, and a communication system that interacts with them.

[0506] 1. Data collection and storage

[0507] Each time a user makes a cashless payment, the terminal retrieves payment data such as information about the purchased items, the amount, date and time, and the store where the transaction took place.

[0508] The device sends this data to the server via secure communication, and the server stores it in a database. The data is encrypted, and privacy is protected.

[0509] 2. Data Analysis and Recommendation Generation

[0510] The server periodically analyzes the accumulated data to model user purchasing patterns. This process utilizes machine learning algorithms such as collaborative filtering and content-based filtering.

[0511] The server generates personalized product recommendations and service suggestions based on the analysis results. These are individually optimized to reflect the user's preferences and purchase history.

[0512] 3. Integrating campaign information

[0513] The server collects and updates the latest campaign information sent from partner stores and service providers and stores it in a database.

[0514] The server matches the user's purchasing patterns, selects campaign information at the appropriate time, and sends it to the device as an optimized suggestion.

[0515] 4. Natural Language Interface

[0516] Users can input questions and requests verbally using voice assistants and chatbot functions.

[0517] The terminal sends these natural language inputs to the server, which uses natural language processing techniques to analyze them and generate appropriate responses.

[0518] The server-generated response is returned to the terminal and smoothly provided to the user.

[0519] 5. Operation of specific examples

[0520] In a real-world scenario, when a user purchases groceries at a supermarket, the terminal collects payment information and sends it to a server. Based on this information, the server provides product recommendations, combining discount information for frequently purchased items. This information is notified to the user from the terminal before the purchase at the store. Furthermore, during promotional periods, the system suggests ways to maximize point rewards.

[0521] Through this form, the present invention enhances the user's purchasing experience and simultaneously supports the marketing strategies of retailers.

[0522] The following describes the processing flow.

[0523] Step 1:

[0524] When a user makes a cashless payment, the terminal acquires payment-related data. This data includes details of the purchased items, the purchase amount, the store where the purchase was made, and the date and time.

[0525] Step 2:

[0526] The terminal encrypts the acquired payment data and transfers it to the server using a secure communication protocol. During this process, user privacy is guaranteed.

[0527] Step 3:

[0528] The server stores the received payment data in a database. The stored data is organized by user and prepared for later analysis.

[0529] Step 4:

[0530] The server periodically analyzes the stored data and uses machine learning algorithms to extract user purchasing patterns. This analysis is performed using a combination of collaborative filtering and content-based filtering.

[0531] Step 5:

[0532] The server generates personalized product and service recommendations for users based on the analysis results. These recommendations are designed to match the individual user's preferences.

[0533] Step 6:

[0534] The server collects campaign information from partner stores and service providers and updates its database. This information is used to identify the most relevant campaigns based on the user's purchase history and analysis results.

[0535] Step 7:

[0536] The server sends this optimized campaign information and product recommendations to the device, and the device notifies the user. The notification is immediately accessible to the user via push notifications or other means.

[0537] Step 8:

[0538] When a user enters a question or request through a voice assistant or chatbot function, the device sends that information to the server.

[0539] Step 9:

[0540] The server uses natural language processing techniques to analyze user input and generate appropriate responses. Care is taken to ensure that these responses address the user's questions.

[0541] Step 10:

[0542] The server generates the response, which is sent to the terminal, and the terminal then presents it to the user. This allows the user to obtain the necessary information in real time.

[0543] (Example 1)

[0544] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0545] Traditional cashless payment systems have been unable to effectively utilize users' purchase history, making it difficult to optimize the purchasing experience and provide appropriate product recommendations. Furthermore, there has been a lack of methods to effectively match campaign information with users' purchasing patterns, which has limited the provision of real-time information to users.

[0546] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0547] In this invention, the server includes means for collecting and storing the user's electronic payment history in a storage device; means for analyzing the collected electronic payment history using a machine learning algorithm and extracting the user's purchasing patterns; means for generating product recommendations based on the user's unique purchasing patterns using a generative AI model; means for obtaining the latest campaign information from data providers and making optimal suggestions to the user; and means for analyzing natural language input from the user and providing appropriate information based on prompts. This enables the provision of a personalized experience based on the user's purchasing history and the real-time provision of appropriate product recommendations and campaign information.

[0548] "User" refers to an individual or legal entity that uses a service or system.

[0549] "Electronic payment" refers to a transaction in which the price of goods or services is paid by electronic means.

[0550] A "storage device" refers to a hardware or software system for holding or storing information.

[0551] A "machine learning algorithm" refers to a computational method used to extract patterns from data and make predictions or decisions.

[0552] "Purchase patterns" refer to a series of purchasing behavioral trends derived from a user's past purchase history.

[0553] A "generative AI model" refers to an artificial intelligence computational model designed to make personalized recommendations and predictions based on data.

[0554] "Product recommendation" refers to products or services that suggest purchases based on the user's preferences and purchase history.

[0555] A "data provider" refers to an organization or company that provides information to the system.

[0556] "Campaign information" refers to information that includes details of discounts, benefits, and sales aimed at promoting sales.

[0557] "Natural language input" refers to an interface that allows users to input information into the system using their usual language.

[0558] A "prompt" refers to the text input that the generating AI uses for analysis based on user instructions or requests.

[0559] The system according to this invention provides comprehensive electronic payment support to improve the user's purchasing experience. This system mainly consists of a user's terminal, a server, and a communication system connecting them.

[0560] Data collection and storage

[0561] The terminal collects information about the purchased items, including the price, date and time, and store details, when a user makes an electronic payment. This is done using hardware such as NFC readers and barcode scanners. The terminal encrypts this data and sends it securely to the server, which then stores the data using a database management system (DBMS).

[0562] Data analysis and recommendation generation

[0563] The server uses programming languages ​​such as Python and R to execute machine learning algorithms and analyze user purchasing patterns. It then leverages generative AI models to automatically generate optimized product and service recommendations based on each user's unique purchase history.

[0564] Campaigns and Interactions

[0565] The server collects the latest campaign information from data providers via APIs and updates its database. It then combines this information with analyzed purchasing patterns to provide users with the most relevant information. Users can make natural language inquiries using voice assistants or text-based chatbots. The server interprets these inquiries using natural language processing technology and sends the most appropriate response back to the user's device.

[0566] Specific example

[0567] As a concrete example, consider a scenario where a user purchases groceries at a supermarket. The terminal uses an NFC reader to collect payment information and transfer it to a server. Based on this data, the server generates recommendations, including new discount information on items the user frequently purchases, and notifies the user via the terminal a few minutes before purchase. Additionally, if the user uses a voice assistant to input a prompt such as "Tell me about this weekend's sales," the server analyzes the prompt and returns appropriate recommendations to assist the user's purchasing plan.

[0568] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0569] Step 1: Data Collection

[0570] The terminal uses an NFC reader or barcode scanner to obtain information about the purchased items, amount, date and time, and store where the transaction took place when a user makes an electronic payment. It receives data about the user's payment behavior as input. This data is encrypted and transmitted to the server over the network. The output is formalized purchase data sent to the server.

[0571] Step 2: Save Data

[0572] The server stores the received data in a database. It receives encrypted data sent from the terminal as input. A database management system (DBMS) is used to store the data while maintaining its integrity. The output is the organized data stored in the DPMS for use in subsequent processing steps.

[0573] Step 3: Data Analysis

[0574] The server analyzes the stored data. It retrieves purchase history data stored in the database as input. Using Python or R, it applies machine learning algorithms to perform pattern recognition. By constructing purchase patterns, it reveals user preferences and trends and predicts expected purchasing behavior. The output is personalized purchase pattern information.

[0575] Step 4: Recommendation Generation

[0576] The server generates product recommendations using a generative AI model. The input is analyzed purchasing pattern information, which the generative AI model uses to provide optimal product recommendations to the user. This utilizes collaborative filtering and content-based filtering techniques. The output is optimized product recommendations designed to encourage future purchasing behavior.

[0577] Step 5: Integrating campaign information

[0578] The server retrieves the latest campaign information from data providers via API and updates the database. It receives campaign data from partners as input. This data is then adjusted to match purchasing patterns to enable optimal recommendations. The output is optimized campaign information for the user.

[0579] Step 6: User Interaction

[0580] Users ask questions in natural language through voice assistants or chatbots. The user's questions or requests are sent as input, either as text or voice. The device sends this to a server, which uses natural language processing techniques to interpret the prompt. An appropriate answer is generated and provided to the user as output. For example, in response to a prompt like "Tell me your recommended products for this weekend," the server might recommend a specific product list.

[0581] (Application Example 1)

[0582] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0583] With the widespread adoption of electronic payments in modern society, consumers are expected to be able to smoothly make the best choices from a vast array of products. However, providing personalized product and market activity information in real time amidst the enormous amount of data is challenging. Conventional purchasing support systems have limited contribution to improving the purchasing experience because product recommendations based on individual users' purchase history are not sufficiently personalized, and there is a lack of on-the-spot information provision utilizing market activity data from facilities.

[0584] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0585] In this invention, the server includes means for collecting and storing the user's payment history in a database, means for extracting the user's purchasing patterns using a machine learning algorithm, and means for selecting and notifying market activity information when the user arrives at a specific facility using location information. This makes it possible to provide product recommendations and market activity information tailored to the user's individual needs in real time, thereby assisting in purchasing decisions.

[0586] "User payment history" refers to information about all electronic transactions conducted by a user, including data such as purchased items, transaction amount, date and time, and the store where the transaction took place.

[0587] A "database" is an electronic information system used to store users' payment history, analysis results, and the latest market activity information.

[0588] A "machine learning algorithm" is a mathematical method that analyzes a user's past behavior history to extract patterns and use them for prediction and recommendation.

[0589] "Purchase patterns" refer to consumer behavior trends, such as what kinds of products a user buys and how often.

[0590] "Product recommendations" refer to a list of products and services suggested based on a user's purchase history and preferences, presenting consumers with suitable options.

[0591] "Latest market activity information" refers to timely sales promotion information such as promotions, special offers, and discount information provided by partner companies and retailers.

[0592] "Natural language input" refers to instructions and questions given by users using natural language, that is, everyday language, in the form of voice or text.

[0593] "Natural language processing technology" is an information technology that uses computers to analyze, understand, and generate human language.

[0594] "Location information" refers to data that indicates the physical location where a device is currently located, and is often obtained using GPS or beacon technology.

[0595] A "notification" is a method of communication sent from a system to a user, and is usually a message that appears as a pop-up on the device screen.

[0596] The system for realizing this invention consists of a user's smart device, a server in the cloud, and a communication network. When a user makes an electronic payment using their smart device, the device acquires payment history data and transmits it to the server in the cloud via secure communication. Cloud platforms such as Amazon Web Services (AWS) and Microsoft Azure can be used for the server.

[0597] The server receives this data and stores it in a database. This database is typically built using services like Google Cloud Firestore or Amazon DynamoDB. The server uses Python and Scikit-learn to implement machine learning algorithms and analyze the user's purchase history to model their purchasing patterns.

[0598] When a user's smart device detects, via location services, that it has reached a specific location, the server generates optimal product recommendations and market activity information related to that location, based on purchase history and market activity data. Firebase Cloud Messaging or Apple Push Notification Service can be used to notify the user at this time.

[0599] When a user makes a query in natural language, the device sends this input to a server, which then parses it using Google Cloud Natural Language API or Amazon Comprehend, running in the cloud, to generate an appropriate response.

[0600] As a concrete example, when a user arrives at a supermarket, the server recommends discounted items and items with point rewards in real time. In this way, users can obtain accurate information before making a purchase, improving their shopping experience.

[0601] An example of a prompt for a generative AI model is, "Based on user B's recent purchase history, suggest available discounts and recommended products at the supermarket today."

[0602] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0603] Step 1:

[0604] When a user makes an electronic payment using a smart device, the device acquires transaction data (product information, amount, time, location, etc.). This is the input. This data is temporarily encrypted within the device and sent to a server in the cloud using secure communication; this is the output.

[0605] Step 2:

[0606] Transaction data received by the server is stored in a database such as Google Cloud Firestore or Amazon DynamoDB. The input is the transmitted transaction data, and the output is the stored data. The server encrypts and stores this data, ensuring its security.

[0607] Step 3:

[0608] The server analyzes data collected periodically. The input consists of a vast amount of stored payment history data. The server uses Python and Scikit-learn to execute machine learning algorithms and extract user purchasing patterns. The output is user-specific purchasing pattern data.

[0609] Step 4:

[0610] When a user arrives at a facility, the device determines its location and sends it to the server. The input is location data, and the output is facility information that the server searches for based on that location. The server then creates a feed containing the latest market activity information related to the facility.

[0611] Step 5:

[0612] The server generates product recommendations for individual users based on purchasing patterns and market activity information for the facility. The input consists of extracted purchasing pattern data and market activity information. The output is a set of user-optimized product recommendation lists and market activity information.

[0613] Step 6:

[0614] The server sends the generated product recommendation information to the smart device. The input is the recommendation information from the server, and the output is a notification displayed on the user's device. The device uses Firebase Cloud Messaging to send the notification to the user.

[0615] Step 7:

[0616] When a user makes a voice inquiry, the device sends this as voice data to the server. The input is the user's voice input. The server uses the Google Cloud Natural Language API to perform natural language processing and generate and output a response to the user's question. The response is then sent back to the device and provided to the user.

[0617] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0618] This invention efficiently supports users' purchasing behavior using an AI agent that works in conjunction with a cashless payment system. In particular, this invention incorporates an emotion engine to provide a sophisticated experience that also takes into account the user's emotions. Specific embodiments are shown below.

[0619] 1. Data collection and storage

[0620] Each time the terminal performs a cashless payment, it collects information such as the purchased items, transaction amount, transaction date and time, and the store where the transaction took place.

[0621] This data is transmitted from the terminal to the server via secure communication and stored in the database in an encrypted state.

[0622] 2. Analysis of purchasing patterns

[0623] The server analyzes payment history stored in the database using machine learning algorithms to model user purchasing patterns.

[0624] The generated purchasing patterns serve as fundamental data for providing affordable product recommendations.

[0625] 3. Emotion recognition by an emotion engine

[0626] The device sends data extracted from the user's voice and text to the server.

[0627] The server uses an emotion engine to recognize the user's emotions from the transmitted data and analyze their state. This emotional state is then reflected in the suggested product information and services.

[0628] 4. Optimization of product recommendations and campaign information

[0629] The server comprehensively considers information obtained from purchasing patterns and emotion recognition to generate product recommendations best suited to each individual user.

[0630] The latest campaign information will be added to this, maximizing the user's purchasing experience.

[0631] 5. Interfaces and Natural Language Processing

[0632] Users interact through voice assistants or text chat.

[0633] This natural language input is sent from the terminal to the server, which uses natural language processing technology to analyze the input and provide appropriate information tailored to the user's question.

[0634] For example, if a user feels anxious before making a purchase, the device detects this emotion, and the server uses this information to recommend products or services that will alleviate their stress. Relevant campaign information is also presented, allowing the user to understand the specific benefits and proceed with the purchase with confidence. This immersive experience enhances consumer satisfaction and enables effective marketing for retailers.

[0635] The following describes the processing flow.

[0636] Step 1:

[0637] When a user makes a purchase using a cashless payment system, the terminal collects data related to the purchase. This data includes the purchased items, price, store, and date and time.

[0638] Step 2:

[0639] The terminal encrypts the collected payment data and sends it to the server via a secure channel. The server receives it and securely stores it in its database.

[0640] Step 3:

[0641] The server periodically analyzes payment data in the database and uses machine learning algorithms to analyze user purchasing patterns. This analysis models user preferences and purchasing trends.

[0642] Step 4:

[0643] When a user expresses emotions through voice or text, the device collects the user's voice and text data and sends it to the server.

[0644] Step 5:

[0645] The server uses an emotion engine to analyze received audio and text data and recognize and evaluate the user's emotional state. For example, it estimates emotions such as joy, anxiety, and stress from the user's tone of voice and word choices.

[0646] Step 6:

[0647] The server comprehensively considers the results of the emotional state assessment and purchasing patterns to generate personalized product and service recommendations. In this process, product selection can be tailored to the user's emotions.

[0648] Step 7:

[0649] The server adds the latest campaign information, compiles it in a format optimized for emotional states and purchasing patterns, and sends it to the device.

[0650] Step 8:

[0651] The device notifies the user of the information it receives and presents it through lists, pop-ups, and voice suggestions. This allows the user to easily identify the products and services that are currently most relevant to them.

[0652] Step 9:

[0653] When a user enters a question in natural language, the terminal sends it to the server, which then analyzes it using natural language processing.

[0654] Step 10:

[0655] The server provides quick and accurate answers to user questions by generating appropriate responses and sending them back to the terminal. Users can then use this information to proceed with purchases or obtain additional information.

[0656] (Example 2)

[0657] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0658] In modern consumer behavior analysis, simply recommending products based on past purchase history is insufficient to capture consumers' psychological needs and temporary emotions, making it difficult to provide appropriate product suggestions. In particular, there is a demand for providing sophisticated purchasing experiences that take emotions and psychological states into account, and this challenge needs to be addressed.

[0659] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0660] In this invention, the server includes means for acquiring the user's payment history and storing it in a storage medium, means for analyzing the user's emotional data using an emotion recognition engine, and means for integrating purchase behavior and promotional information based on the emotion recognition results to generate optimal suggestions. This makes it possible to provide optimal product suggestions and promotional information that take into account not only the user's purchasing behavior but also their emotions and psychological state at the time.

[0661] "Payment history" refers to a record of all transactions a user has made, and is a collection of data including product information, amount, date and time, and location of the transaction.

[0662] A "storage medium" is a hardware or software component used to store and manage data and information.

[0663] A "learning model" is an algorithm or program that uses machine learning techniques to analyze data and discover specific patterns or rules.

[0664] "Purchase behavior" refers to data that shows the user's tendencies in selecting and purchasing goods and services, obtained by analyzing their past purchasing activities.

[0665] "Product recommendations" refer to recommendations for appropriate products and services tailored to individual users, based on their purchasing behavior and other information.

[0666] "Promotional information" refers to information that includes details of discounts, campaigns, and promotions designed to boost the sales of products and services.

[0667] "Emotional data" refers to information that indicates a user's psychological state or emotions, extracted from their voice, text, or behavior.

[0668] An "emotion recognition engine" is software or an algorithm used to analyze and identify a user's emotional state from voice, text, or other data.

[0669] "Natural language processing technology" refers to a group of technologies used by computers to understand, interpret, and generate natural human language.

[0670] "Information provision" refers to the act of communicating analysis results and recommendations to users in an appropriate format.

[0671] This invention provides a personalized product suggestion system that takes into account the user's purchasing behavior and emotions. This system is primarily realized through the cooperation of a terminal and a server.

[0672] The device retrieves payment information when a user purchases a product. This information includes the product name, price, date and time, and store of purchase. This data is transmitted to the server via a secure communication protocol (e.g., HTTPS) and securely stored on a storage medium.

[0673] Furthermore, sentiment data is extracted from the user's voice or text input. This process uses speech recognition software to convert speech to text. This data is then sent back to the server via a secure protocol.

[0674] The server analyzes stored payment history using a learning model to understand user purchasing behavior. This analysis utilizes programming languages ​​such as Python and machine learning libraries such as Scikit-learn and TensorFlow. The purchase behavior data obtained from the analysis is integrated with the user's emotions, which are evaluated by an emotion recognition engine.

[0675] Emotion recognition utilizes an emotion recognition engine. This includes commonly available emotion recognition software that precisely identifies the user's emotional state. Based on this emotion data and purchasing patterns, the server uses a generative AI model to generate optimal product suggestions and promotional information.

[0676] The generated product suggestions and promotional information are sent to the user's device in the format that is easiest for them to understand. Natural language processing technology is used to generate prompts in a natural way. Users can receive the information and take necessary actions via voice assistants or text chat.

[0677] For example, if a user says to their device, "I've been feeling stressed lately. Do you have any product recommendations?", the device converts the voice into text and sends it to the server. The server then analyzes the prompt and, based on the user's emotions, suggests relaxation products and offers relevant campaigns to improve the user's purchasing experience.

[0678] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0679] Step 1:

[0680] The device acquires payment information such as product name, price, date and time, and store when a user purchases a product. This information is temporarily stored on the device. This generates a detailed dataset for each user transaction. The input is the user's purchase behavior, and the output is a dataset of transaction information.

[0681] Step 2:

[0682] The terminal transmits the collected transaction information to the server using a secure communication protocol (e.g., HTTPS). The transmitted data is encrypted and stored on the server's storage medium. In this scenario, the input is the transaction information from the terminal, and the output is the encrypted data stored in the server's database.

[0683] Step 3:

[0684] The server analyzes stored transaction history using machine learning algorithms. It extracts purchasing patterns using Python or Scikit-learn. The input is transaction history data, and the output is modeled user purchasing patterns. Specifically, it performs clustering and classification to identify patterns in the data.

[0685] Step 4:

[0686] The device acquires data entered by the user via voice or text. This data reflects the user's emotions. Specifically, when voice input is received, the device uses speech recognition software to convert it into text. The input is the user's voice data, and the output is the converted text data.

[0687] Step 5:

[0688] The terminal sends the converted text data to the server. The server uses an emotion recognition engine to analyze the user's emotions. In this process, the input is the text data from the terminal, and the output is the analyzed emotion information. Specifically, the emotion recognition algorithm identifies the user's emotional state.

[0689] Step 6:

[0690] The server uses a generative AI model to create product suggestions tailored to the user, based on purchase pattern data and emotional information. The input is purchase patterns and emotional information, and the output is customized product suggestions and promotional information. Specifically, it generates optimal product suggestions by comparing purchase history and emotional state.

[0691] Step 7:

[0692] The server sends the generated product suggestions to the terminal, and the user receives this information via voice assistant or text chat. The input is the generated product suggestions, and the output is the information presented to the user. Specifically, natural language processing is used to present the information in a way that is easy for the user to understand.

[0693] (Application Example 2)

[0694] Next, we will explain Application Example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0695] In modern e-commerce, product recommendations and sales promotions are often based on general purchase history and do not always accurately reflect users' emotions or individual needs. Therefore, methods to improve the user experience are needed. Furthermore, there is a growing demand for interactive, natural language-based dialogue in user interfaces. This is expected to increase consumer satisfaction and enable more effective sales activities for businesses.

[0696] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0697] In this invention, the server includes means for collecting and storing the user's payment history in an information storage unit, means for analyzing the collected payment history using machine learning techniques and extracting the user's transaction patterns, and means for generating product recommendations based on the user's unique purchasing patterns and emotional state. This enables personalized product recommendations that take into account the user's emotional state.

[0698] "Payment history" is a record of information about all payment transactions a user has made in the past.

[0699] An "information storage unit" is a computer system or database for securely and systematically storing collected data and information over a long period of time.

[0700] "Machine learning techniques" are algorithms that automatically learn useful patterns and knowledge from data to perform predictions and classifications.

[0701] "Transaction patterns" refer to the regularities and trends in users' purchasing and payment behavior, and are characteristics of behavior extracted through analysis.

[0702] "Emotional state" refers to information that indicates the psychological and emotional situation a user is experiencing at a particular point in time.

[0703] "Product recommendation" refers to the act or process of presenting a specific product to a user based on their characteristics and behavioral history.

[0704] "Sales promotion information" refers to marketing data that includes campaigns and discount information designed to encourage users to purchase products.

[0705] The server stores payment history data transmitted from the user's terminal in its information storage unit and analyzes this data using machine learning techniques to extract the user's transaction patterns. This analysis allows for a clear understanding of regularities and trends in the user's purchasing behavior. Furthermore, the server recognizes the user's emotional state from voice and text data and integrates this information to provide product recommendations. The product recommendation process also takes into account the latest sales promotion information, enabling the provision of the most suitable products and campaign information for the user.

[0706] Users communicate bidirectionally with the server using a natural language interface via their device. The server processes user input in real time using natural language processing technology and quickly responds with information in response to user questions and requests.

[0707] For example, if a user enters "My budget for this month is limited, and I'm worried about whether I really need this product" into a smartphone application, the server will understand that sentiment and recommend relevant, cost-effective options.

[0708] An example of a prompt to input into a generative AI model is: "How can we recommend budget-friendly products when a user is feeling anxious? Which emotion engine and recommendation algorithm should we use?"

[0709] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0710] Step 1:

[0711] The terminal collects the user's payment history data and transmits it to the server using a secure communication method. The input consists of all payment details made by the user, and the output is encrypted payment history data.

[0712] Step 2:

[0713] The server stores the received payment history data in its information storage unit. This allows for centralized management of the information necessary for subsequent data analysis. The input is encrypted payment history data, and the output is the stored data state. The data is imported into a database and securely stored in an encrypted state.

[0714] Step 3:

[0715] The server analyzes payment history data stored in its information storage unit using machine learning techniques. The input is the stored payment history data, and the output is the user's transaction patterns. The server applies machine learning algorithms to this data to extract regularities and characteristics in the user's purchasing behavior.

[0716] Step 4:

[0717] The device collects user voice or text data and sends it to the server. The input for this sentiment data is voice or text input from the user, and the output is the data sent to the server.

[0718] Step 5:

[0719] The server recognizes the user's emotional state from the received voice or text data. The input is voice or text data, and the output is the recognized emotional state. The server applies an emotion recognition algorithm to analyze the user's psychological state.

[0720] Step 6:

[0721] The server generates product recommendations based on the user's trading patterns and emotional state, while also considering the latest sales promotion information. The inputs are trading patterns, emotional state, and sales promotion information, and the output is the most suitable product recommendation for the user. The server integrates this information to select products and deals that are most relevant to the user.

[0722] Step 7:

[0723] The user communicates bidirectionally with the server using a natural language interface through their device. Input is natural language input from the user, and output is corresponding information from the server. The server uses natural language processing technology to answer the user's questions in real time.

[0724] Each step works together to create a system that delivers an optimized shopping experience for the user.

[0725] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0726] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0727] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0728] [Fourth Embodiment]

[0729] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0730] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0731] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0732] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0733] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0734] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0735] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0736] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0737] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0738] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0739] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0740] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0741] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0742] This invention provides an AI agent that works in conjunction with a cashless payment system to support users' purchasing behavior. The system consists of the user's smartphone, servers located in the cloud or data center, and a communication system that interacts with them.

[0743] 1. Data collection and storage

[0744] Each time a user makes a cashless payment, the terminal retrieves payment data such as information about the purchased items, the amount, date and time, and the store where the transaction took place.

[0745] The device sends this data to the server via secure communication, and the server stores it in a database. The data is encrypted, and privacy is protected.

[0746] 2. Data Analysis and Recommendation Generation

[0747] The server periodically analyzes the accumulated data to model user purchasing patterns. This process utilizes machine learning algorithms such as collaborative filtering and content-based filtering.

[0748] The server generates personalized product recommendations and service suggestions based on the analysis results. These are individually optimized to reflect the user's preferences and purchase history.

[0749] 3. Integrating campaign information

[0750] The server collects and updates the latest campaign information sent from partner stores and service providers and stores it in a database.

[0751] The server matches the user's purchasing patterns, selects campaign information at the appropriate time, and sends it to the device as an optimized suggestion.

[0752] 4. Natural Language Interface

[0753] Users can input questions and requests verbally using voice assistants and chatbot functions.

[0754] The terminal sends these natural language inputs to the server, which uses natural language processing techniques to analyze them and generate appropriate responses.

[0755] The server-generated response is returned to the terminal and smoothly provided to the user.

[0756] 5. Operation of specific examples

[0757] In a real-world scenario, when a user purchases groceries at a supermarket, the terminal collects payment information and sends it to a server. Based on this information, the server provides product recommendations, combining discount information for frequently purchased items. This information is notified to the user from the terminal before the purchase at the store. Furthermore, during promotional periods, the system suggests ways to maximize point rewards.

[0758] Through this form, the present invention enhances the user's purchasing experience and simultaneously supports the marketing strategies of retailers.

[0759] The following describes the processing flow.

[0760] Step 1:

[0761] When a user makes a cashless payment, the terminal acquires payment-related data. This data includes details of the purchased items, the purchase amount, the store where the purchase was made, and the date and time.

[0762] Step 2:

[0763] The terminal encrypts the acquired payment data and transfers it to the server using a secure communication protocol. During this process, user privacy is guaranteed.

[0764] Step 3:

[0765] The server stores the received payment data in a database. The stored data is organized by user and prepared for later analysis.

[0766] Step 4:

[0767] The server periodically analyzes the stored data and uses machine learning algorithms to extract user purchasing patterns. This analysis is performed using a combination of collaborative filtering and content-based filtering.

[0768] Step 5:

[0769] The server generates personalized product and service recommendations for users based on the analysis results. These recommendations are designed to match the individual user's preferences.

[0770] Step 6:

[0771] The server collects campaign information from partner stores and service providers and updates its database. This information is used to identify the most relevant campaigns based on the user's purchase history and analysis results.

[0772] Step 7:

[0773] The server sends this optimized campaign information and product recommendations to the device, and the device notifies the user. The notification is immediately accessible to the user via push notifications or other means.

[0774] Step 8:

[0775] When a user enters a question or request through a voice assistant or chatbot function, the device sends that information to the server.

[0776] Step 9:

[0777] The server uses natural language processing techniques to analyze user input and generate appropriate responses. Care is taken to ensure that these responses address the user's questions.

[0778] Step 10:

[0779] The server generates the response, which is sent to the terminal, and the terminal then presents it to the user. This allows the user to obtain the necessary information in real time.

[0780] (Example 1)

[0781] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0782] Traditional cashless payment systems have been unable to effectively utilize users' purchase history, making it difficult to optimize the purchasing experience and provide appropriate product recommendations. Furthermore, there has been a lack of methods to effectively match campaign information with users' purchasing patterns, which has limited the provision of real-time information to users.

[0783] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0784] In this invention, the server includes means for collecting and storing the user's electronic payment history in a storage device; means for analyzing the collected electronic payment history using a machine learning algorithm and extracting the user's purchasing patterns; means for generating product recommendations based on the user's unique purchasing patterns using a generative AI model; means for obtaining the latest campaign information from data providers and making optimal suggestions to the user; and means for analyzing natural language input from the user and providing appropriate information based on prompts. This enables the provision of a personalized experience based on the user's purchasing history and the real-time provision of appropriate product recommendations and campaign information.

[0785] "User" refers to an individual or legal entity that uses a service or system.

[0786] "Electronic payment" refers to a transaction in which the price of goods or services is paid by electronic means.

[0787] A "storage device" refers to a hardware or software system for holding or storing information.

[0788] A "machine learning algorithm" refers to a computational method used to extract patterns from data and make predictions or decisions.

[0789] "Purchase patterns" refer to a series of purchasing behavioral trends derived from a user's past purchase history.

[0790] A "generative AI model" refers to an artificial intelligence computational model designed to make personalized recommendations and predictions based on data.

[0791] "Product recommendation" refers to products or services that suggest purchases based on the user's preferences and purchase history.

[0792] A "data provider" refers to an organization or company that provides information to the system.

[0793] "Campaign information" refers to information that includes details of discounts, benefits, and sales aimed at promoting sales.

[0794] "Natural language input" refers to an interface that allows users to input information into the system using their usual language.

[0795] A "prompt" refers to the text input that the generating AI uses for analysis based on user instructions or requests.

[0796] The system according to this invention provides comprehensive electronic payment support to improve the user's purchasing experience. This system mainly consists of a user's terminal, a server, and a communication system connecting them.

[0797] Data collection and storage

[0798] The terminal collects information about the purchased items, including the price, date and time, and store details, when a user makes an electronic payment. This is done using hardware such as NFC readers and barcode scanners. The terminal encrypts this data and sends it securely to the server, which then stores the data using a database management system (DBMS).

[0799] Data analysis and recommendation generation

[0800] The server uses programming languages ​​such as Python and R to execute machine learning algorithms and analyze user purchasing patterns. It then leverages generative AI models to automatically generate optimized product and service recommendations based on each user's unique purchase history.

[0801] Campaigns and Interactions

[0802] The server collects the latest campaign information from data providers via APIs and updates its database. It then combines this information with analyzed purchasing patterns to provide users with the most relevant information. Users can make natural language inquiries using voice assistants or text-based chatbots. The server interprets these inquiries using natural language processing technology and sends the most appropriate response back to the user's device.

[0803] Specific example

[0804] As a concrete example, consider a scenario where a user purchases groceries at a supermarket. The terminal uses an NFC reader to collect payment information and transfer it to a server. Based on this data, the server generates recommendations, including new discount information on items the user frequently purchases, and notifies the user via the terminal a few minutes before purchase. Additionally, if the user uses a voice assistant to input a prompt such as "Tell me about this weekend's sales," the server analyzes the prompt and returns appropriate recommendations to assist the user's purchasing plan.

[0805] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0806] Step 1: Data Collection

[0807] The terminal uses an NFC reader or barcode scanner to obtain information about the purchased items, amount, date and time, and store where the transaction took place when a user makes an electronic payment. It receives data about the user's payment behavior as input. This data is encrypted and transmitted to the server over the network. The output is formalized purchase data sent to the server.

[0808] Step 2: Save Data

[0809] The server stores the received data in a database. It receives encrypted data sent from the terminal as input. A database management system (DBMS) is used to store the data while maintaining its integrity. The output is the organized data stored in the DPMS for use in subsequent processing steps.

[0810] Step 3: Data Analysis

[0811] The server analyzes the stored data. It retrieves purchase history data stored in the database as input. Using Python or R, it applies machine learning algorithms to perform pattern recognition. By constructing purchase patterns, it reveals user preferences and trends and predicts expected purchasing behavior. The output is personalized purchase pattern information.

[0812] Step 4: Recommendation Generation

[0813] The server generates product recommendations using a generative AI model. The input is analyzed purchasing pattern information, which the generative AI model uses to provide optimal product recommendations to the user. This utilizes collaborative filtering and content-based filtering techniques. The output is optimized product recommendations designed to encourage future purchasing behavior.

[0814] Step 5: Integrating campaign information

[0815] The server retrieves the latest campaign information from data providers via API and updates the database. It receives campaign data from partners as input. This data is then adjusted to match purchasing patterns to enable optimal recommendations. The output is optimized campaign information for the user.

[0816] Step 6: User Interaction

[0817] Users ask questions in natural language through voice assistants or chatbots. The user's questions or requests are sent as input, either as text or voice. The device sends this to a server, which uses natural language processing techniques to interpret the prompt. An appropriate answer is generated and provided to the user as output. For example, in response to a prompt like "Tell me your recommended products for this weekend," the server might recommend a specific product list.

[0818] (Application Example 1)

[0819] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0820] With the widespread adoption of electronic payments in modern society, consumers are expected to be able to smoothly make the best choices from a vast array of products. However, providing personalized product and market activity information in real time amidst the enormous amount of data is challenging. Conventional purchasing support systems have limited contribution to improving the purchasing experience because product recommendations based on individual users' purchase history are not sufficiently personalized, and there is a lack of on-the-spot information provision utilizing market activity data from facilities.

[0821] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0822] In this invention, the server includes means for collecting and storing the user's payment history in a database, means for extracting the user's purchasing patterns using a machine learning algorithm, and means for selecting and notifying market activity information when the user arrives at a specific facility using location information. This makes it possible to provide product recommendations and market activity information tailored to the user's individual needs in real time, thereby assisting in purchasing decisions.

[0823] "User payment history" refers to information about all electronic transactions conducted by a user, including data such as purchased items, transaction amount, date and time, and the store where the transaction took place.

[0824] A "database" is an electronic information system used to store users' payment history, analysis results, and the latest market activity information.

[0825] A "machine learning algorithm" is a mathematical method that analyzes a user's past behavior history to extract patterns and use them for prediction and recommendation.

[0826] "Purchase patterns" refer to consumer behavior trends, such as what kinds of products a user buys and how often.

[0827] "Product recommendations" refer to a list of products and services suggested based on a user's purchase history and preferences, presenting consumers with suitable options.

[0828] "Latest market activity information" refers to timely sales promotion information such as promotions, special offers, and discount information provided by partner companies and retailers.

[0829] "Natural language input" refers to instructions and questions given by users using natural language, that is, everyday language, in the form of voice or text.

[0830] "Natural language processing technology" is an information technology that uses computers to analyze, understand, and generate human language.

[0831] "Location information" refers to data that indicates the physical location where a device is currently located, and is often obtained using GPS or beacon technology.

[0832] A "notification" is a method of communication sent from a system to a user, and is usually a message that appears as a pop-up on the device screen.

[0833] The system for realizing this invention consists of a user's smart device, a server in the cloud, and a communication network. When a user makes an electronic payment using their smart device, the device acquires payment history data and transmits it to the server in the cloud via secure communication. Cloud platforms such as Amazon Web Services (AWS) and Microsoft Azure can be used for the server.

[0834] The server receives this data and stores it in a database. This database is typically built using services like Google Cloud Firestore or Amazon DynamoDB. The server uses Python and Scikit-learn to implement machine learning algorithms and analyze the user's purchase history to model their purchasing patterns.

[0835] When a user's smart device detects, via location services, that it has reached a specific location, the server generates optimal product recommendations and market activity information related to that location, based on purchase history and market activity data. Firebase Cloud Messaging or Apple Push Notification Service can be used to notify the user at this time.

[0836] When a user makes a query in natural language, the device sends this input to a server, which then parses it using Google Cloud Natural Language API or Amazon Comprehend, running in the cloud, to generate an appropriate response.

[0837] As a concrete example, when a user arrives at a supermarket, the server recommends discounted items and items with point rewards in real time. In this way, users can obtain accurate information before making a purchase, improving their shopping experience.

[0838] An example of a prompt for a generative AI model is, "Based on user B's recent purchase history, suggest available discounts and recommended products at the supermarket today."

[0839] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0840] Step 1:

[0841] When a user makes an electronic payment using a smart device, the device acquires transaction data (product information, amount, time, location, etc.). This is the input. This data is temporarily encrypted within the device and sent to a server in the cloud using secure communication; this is the output.

[0842] Step 2:

[0843] Transaction data received by the server is stored in a database such as Google Cloud Firestore or Amazon DynamoDB. The input is the transmitted transaction data, and the output is the stored data. The server encrypts and stores this data, ensuring its security.

[0844] Step 3:

[0845] The server analyzes data collected periodically. The input consists of a vast amount of stored payment history data. The server uses Python and Scikit-learn to execute machine learning algorithms and extract user purchasing patterns. The output is user-specific purchasing pattern data.

[0846] Step 4:

[0847] When a user arrives at a facility, the device determines its location and sends it to the server. The input is location data, and the output is facility information that the server searches for based on that location. The server then creates a feed containing the latest market activity information related to the facility.

[0848] Step 5:

[0849] The server generates product recommendations for individual users based on purchasing patterns and market activity information for the facility. The input consists of extracted purchasing pattern data and market activity information. The output is a set of user-optimized product recommendation lists and market activity information.

[0850] Step 6:

[0851] The server sends the generated product recommendation information to the smart device. The input is the recommendation information from the server, and the output is a notification displayed on the user's device. The device uses Firebase Cloud Messaging to send the notification to the user.

[0852] Step 7:

[0853] When a user makes a voice inquiry, the device sends this as voice data to the server. The input is the user's voice input. The server uses the Google Cloud Natural Language API to perform natural language processing and generate and output a response to the user's question. The response is then sent back to the device and provided to the user.

[0854] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0855] This invention efficiently supports users' purchasing behavior using an AI agent that works in conjunction with a cashless payment system. In particular, this invention incorporates an emotion engine to provide a sophisticated experience that also takes into account the user's emotions. Specific embodiments are shown below.

[0856] 1. Data collection and storage

[0857] Each time the terminal performs a cashless payment, it collects information such as the purchased items, transaction amount, transaction date and time, and the store where the transaction took place.

[0858] This data is transmitted from the terminal to the server via secure communication and stored in the database in an encrypted state.

[0859] 2. Analysis of purchasing patterns

[0860] The server analyzes payment history stored in the database using machine learning algorithms to model user purchasing patterns.

[0861] The generated purchasing patterns serve as fundamental data for providing affordable product recommendations.

[0862] 3. Emotion recognition by an emotion engine

[0863] The device sends data extracted from the user's voice and text to the server.

[0864] The server uses an emotion engine to recognize the user's emotions from the transmitted data and analyze their state. This emotional state is then reflected in the suggested product information and services.

[0865] 4. Optimization of product recommendations and campaign information

[0866] The server comprehensively considers information obtained from purchasing patterns and emotion recognition to generate product recommendations best suited to each individual user.

[0867] The latest campaign information will be added to this, maximizing the user's purchasing experience.

[0868] 5. Interfaces and Natural Language Processing

[0869] Users interact through voice assistants or text chat.

[0870] This natural language input is sent from the terminal to the server, which uses natural language processing technology to analyze the input and provide appropriate information tailored to the user's question.

[0871] For example, if a user feels anxious before making a purchase, the device detects this emotion, and the server uses this information to recommend products or services that will alleviate their stress. Relevant campaign information is also presented, allowing the user to understand the specific benefits and proceed with the purchase with confidence. This immersive experience enhances consumer satisfaction and enables effective marketing for retailers.

[0872] The following describes the processing flow.

[0873] Step 1:

[0874] When a user makes a purchase using a cashless payment system, the terminal collects data related to the purchase. This data includes the purchased items, price, store, and date and time.

[0875] Step 2:

[0876] The terminal encrypts the collected payment data and sends it to the server via a secure channel. The server receives it and securely stores it in its database.

[0877] Step 3:

[0878] The server periodically analyzes payment data in the database and uses machine learning algorithms to analyze user purchasing patterns. This analysis models user preferences and purchasing trends.

[0879] Step 4:

[0880] When a user expresses emotions through voice or text, the device collects the user's voice and text data and sends it to the server.

[0881] Step 5:

[0882] The server uses an emotion engine to analyze received audio and text data and recognize and evaluate the user's emotional state. For example, it estimates emotions such as joy, anxiety, and stress from the user's tone of voice and word choices.

[0883] Step 6:

[0884] The server comprehensively considers the results of the emotional state assessment and purchasing patterns to generate personalized product and service recommendations. In this process, product selection can be tailored to the user's emotions.

[0885] Step 7:

[0886] The server adds the latest campaign information, compiles it in a format optimized for emotional states and purchasing patterns, and sends it to the device.

[0887] Step 8:

[0888] The device notifies the user of the information it receives and presents it through lists, pop-ups, and voice suggestions. This allows the user to easily identify the products and services that are currently most relevant to them.

[0889] Step 9:

[0890] When a user enters a question in natural language, the terminal sends it to the server, which then analyzes it using natural language processing.

[0891] Step 10:

[0892] The server provides quick and accurate answers to user questions by generating appropriate responses and sending them back to the terminal. Users can then use this information to proceed with purchases or obtain additional information.

[0893] (Example 2)

[0894] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0895] In modern consumer behavior analysis, simply recommending products based on past purchase history is insufficient to capture consumers' psychological needs and temporary emotions, making it difficult to provide appropriate product suggestions. In particular, there is a demand for providing sophisticated purchasing experiences that take emotions and psychological states into account, and this challenge needs to be addressed.

[0896] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0897] In this invention, the server includes means for acquiring the user's payment history and storing it in a storage medium, means for analyzing the user's emotional data using an emotion recognition engine, and means for integrating purchase behavior and promotional information based on the emotion recognition results to generate optimal suggestions. This makes it possible to provide optimal product suggestions and promotional information that take into account not only the user's purchasing behavior but also their emotions and psychological state at the time.

[0898] "Payment history" refers to a record of all transactions a user has made, and is a collection of data including product information, amount, date and time, and location of the transaction.

[0899] A "storage medium" is a hardware or software component used to store and manage data and information.

[0900] A "learning model" is an algorithm or program that uses machine learning techniques to analyze data and discover specific patterns or rules.

[0901] "Purchase behavior" refers to data that shows the user's tendencies in selecting and purchasing goods and services, obtained by analyzing their past purchasing activities.

[0902] "Product recommendations" refer to recommendations for appropriate products and services tailored to individual users, based on their purchasing behavior and other information.

[0903] "Promotional information" refers to information that includes details of discounts, campaigns, and promotions designed to boost the sales of products and services.

[0904] "Emotional data" refers to information that indicates a user's psychological state or emotions, extracted from their voice, text, or behavior.

[0905] An "emotion recognition engine" is software or an algorithm used to analyze and identify a user's emotional state from voice, text, or other data.

[0906] "Natural language processing technology" refers to a group of technologies used by computers to understand, interpret, and generate natural human language.

[0907] "Information provision" refers to the act of communicating analysis results and recommendations to users in an appropriate format.

[0908] This invention provides a personalized product suggestion system that takes into account the user's purchasing behavior and emotions. This system is primarily realized through the cooperation of a terminal and a server.

[0909] The device retrieves payment information when a user purchases a product. This information includes the product name, price, date and time, and store of purchase. This data is transmitted to the server via a secure communication protocol (e.g., HTTPS) and securely stored on a storage medium.

[0910] Furthermore, sentiment data is extracted from the user's voice or text input. This process uses speech recognition software to convert speech to text. This data is then sent back to the server via a secure protocol.

[0911] The server analyzes stored payment history using a learning model to understand user purchasing behavior. This analysis utilizes programming languages ​​such as Python and machine learning libraries such as Scikit-learn and TensorFlow. The purchase behavior data obtained from the analysis is integrated with the user's emotions, which are evaluated by an emotion recognition engine.

[0912] Emotion recognition utilizes an emotion recognition engine. This includes commonly available emotion recognition software that precisely identifies the user's emotional state. Based on this emotion data and purchasing patterns, the server uses a generative AI model to generate optimal product suggestions and promotional information.

[0913] The generated product suggestions and promotional information are sent to the user's device in the format that is easiest for them to understand. Natural language processing technology is used to generate prompts in a natural way. Users can receive the information and take necessary actions via voice assistants or text chat.

[0914] For example, if a user says to their device, "I've been feeling stressed lately. Do you have any product recommendations?", the device converts the voice into text and sends it to the server. The server then analyzes the prompt and, based on the user's emotions, suggests relaxation products and offers relevant campaigns to improve the user's purchasing experience.

[0915] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0916] Step 1:

[0917] The device acquires payment information such as product name, price, date and time, and store when a user purchases a product. This information is temporarily stored on the device. This generates a detailed dataset for each user transaction. The input is the user's purchase behavior, and the output is a dataset of transaction information.

[0918] Step 2:

[0919] The terminal transmits the collected transaction information to the server using a secure communication protocol (e.g., HTTPS). The transmitted data is encrypted and stored on the server's storage medium. In this scenario, the input is the transaction information from the terminal, and the output is the encrypted data stored in the server's database.

[0920] Step 3:

[0921] The server analyzes stored transaction history using machine learning algorithms. It extracts purchasing patterns using Python or Scikit-learn. The input is transaction history data, and the output is modeled user purchasing patterns. Specifically, it performs clustering and classification to identify patterns in the data.

[0922] Step 4:

[0923] The device acquires data entered by the user via voice or text. This data reflects the user's emotions. Specifically, when voice input is received, the device uses speech recognition software to convert it into text. The input is the user's voice data, and the output is the converted text data.

[0924] Step 5:

[0925] The terminal sends the converted text data to the server. The server uses an emotion recognition engine to analyze the user's emotions. In this process, the input is the text data from the terminal, and the output is the analyzed emotion information. Specifically, the emotion recognition algorithm identifies the user's emotional state.

[0926] Step 6:

[0927] The server uses a generative AI model to create product suggestions tailored to the user, based on purchase pattern data and emotional information. The input is purchase patterns and emotional information, and the output is customized product suggestions and promotional information. Specifically, it generates optimal product suggestions by comparing purchase history and emotional state.

[0928] Step 7:

[0929] The server sends the generated product suggestions to the terminal, and the user receives this information via voice assistant or text chat. The input is the generated product suggestions, and the output is the information presented to the user. Specifically, natural language processing is used to present the information in a way that is easy for the user to understand.

[0930] (Application Example 2)

[0931] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0932] In modern e-commerce, product recommendations and sales promotions are often based on general purchase history and do not always accurately reflect users' emotions or individual needs. Therefore, methods to improve the user experience are needed. Furthermore, there is a growing demand for interactive, natural language-based dialogue in user interfaces. This is expected to increase consumer satisfaction and enable more effective sales activities for businesses.

[0933] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0934] In this invention, the server includes means for collecting and storing the user's payment history in an information storage unit, means for analyzing the collected payment history using machine learning techniques and extracting the user's transaction patterns, and means for generating product recommendations based on the user's unique purchasing patterns and emotional state. This enables personalized product recommendations that take into account the user's emotional state.

[0935] "Payment history" is a record of information about all payment transactions a user has made in the past.

[0936] An "information storage unit" is a computer system or database for securely and systematically storing collected data and information over a long period of time.

[0937] "Machine learning techniques" are algorithms that automatically learn useful patterns and knowledge from data to perform predictions and classifications.

[0938] "Transaction patterns" refer to the regularities and trends in users' purchasing and payment behavior, and are characteristics of behavior extracted through analysis.

[0939] "Emotional state" refers to information that indicates the psychological and emotional situation a user is experiencing at a particular point in time.

[0940] "Product recommendation" refers to the act or process of presenting a specific product to a user based on their characteristics and behavioral history.

[0941] "Sales promotion information" refers to marketing data that includes campaigns and discount information designed to encourage users to purchase products.

[0942] The server stores payment history data transmitted from the user's terminal in its information storage unit and analyzes this data using machine learning techniques to extract the user's transaction patterns. This analysis allows for a clear understanding of regularities and trends in the user's purchasing behavior. Furthermore, the server recognizes the user's emotional state from voice and text data and integrates this information to provide product recommendations. The product recommendation process also takes into account the latest sales promotion information, enabling the provision of the most suitable products and campaign information for the user.

[0943] Users communicate bidirectionally with the server using a natural language interface via their device. The server processes user input in real time using natural language processing technology and quickly responds with information in response to user questions and requests.

[0944] For example, if a user enters "My budget for this month is limited, and I'm worried about whether I really need this product" into a smartphone application, the server will understand that sentiment and recommend relevant, cost-effective options.

[0945] An example of a prompt to input into a generative AI model is: "How can we recommend budget-friendly products when a user is feeling anxious? Which emotion engine and recommendation algorithm should we use?"

[0946] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0947] Step 1:

[0948] The terminal collects the user's payment history data and transmits it to the server using a secure communication method. The input consists of all payment details made by the user, and the output is encrypted payment history data.

[0949] Step 2:

[0950] The server stores the received payment history data in its information storage unit. This allows for centralized management of the information necessary for subsequent data analysis. The input is encrypted payment history data, and the output is the stored data state. The data is imported into a database and securely stored in an encrypted state.

[0951] Step 3:

[0952] The server analyzes payment history data stored in its information storage unit using machine learning techniques. The input is the stored payment history data, and the output is the user's transaction patterns. The server applies machine learning algorithms to this data to extract regularities and characteristics in the user's purchasing behavior.

[0953] Step 4:

[0954] The device collects user voice or text data and sends it to the server. The input for this sentiment data is voice or text input from the user, and the output is the data sent to the server.

[0955] Step 5:

[0956] The server recognizes the user's emotional state from the received voice or text data. The input is voice or text data, and the output is the recognized emotional state. The server applies an emotion recognition algorithm to analyze the user's psychological state.

[0957] Step 6:

[0958] The server generates product recommendations based on the user's trading patterns and emotional state, while also considering the latest sales promotion information. The inputs are trading patterns, emotional state, and sales promotion information, and the output is the most suitable product recommendation for the user. The server integrates this information to select products and deals that are most relevant to the user.

[0959] Step 7:

[0960] The user communicates bidirectionally with the server using a natural language interface through their device. Input is natural language input from the user, and output is corresponding information from the server. The server uses natural language processing technology to answer the user's questions in real time.

[0961] Each step works together to create a system that delivers an optimized shopping experience for the user.

[0962] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0963] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0964] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0965] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0966] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0967] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0968] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0969] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0970] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0971] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0972] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0973] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0974] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0975] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0976] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0977] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0978] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0979] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0980] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0981] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0982] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0983] The following is further disclosed regarding the embodiments described above.

[0984] (Claim 1)

[0985] A means of collecting and storing users' payment history in a database,

[0986] A method for analyzing payment history collected using machine learning algorithms and extracting user purchasing patterns,

[0987] A means for generating product recommendations based on user-specific purchasing patterns,

[0988] A means of gathering the latest campaign information and making the best suggestions to users,

[0989] A means of analyzing natural language input from users and providing appropriate information,

[0990] A system that includes this.

[0991] (Claim 2)

[0992] The system according to claim 1, comprising means for providing the user with optimal product recommendations and campaign information, taking into account the user's purchasing patterns and the latest campaign information.

[0993] (Claim 3)

[0994] The system according to claim 1, comprising means for analyzing the aforementioned natural language input using natural language processing technology and providing information to the user in real time.

[0995] "Example 1"

[0996] (Claim 1)

[0997] A means for collecting and storing a user's electronic payment history in a storage device,

[0998] A method for analyzing electronic payment history collected using machine learning algorithms and extracting user purchasing patterns,

[0999] A means of generating product recommendations based on user-specific purchasing patterns using a generative AI model,

[1000] A means of obtaining the latest campaign information from providers and making the most suitable suggestions to users,

[1001] A means for analyzing natural language input from the user and providing appropriate information based on prompts,

[1002] A system that includes this.

[1003] (Claim 2)

[1004] The system according to claim 1, comprising means for providing users with optimal product recommendations and campaign information using artificial intelligence, taking into account purchasing patterns and the latest campaign information.

[1005] (Claim 3)

[1006] The system according to claim 1, comprising means for analyzing natural language input using natural language processing technology and providing information to the user in real time.

[1007] "Application Example 1"

[1008] (Claim 1)

[1009] A means of collecting and storing users' payment history in a database,

[1010] A method for analyzing payment history collected using machine learning algorithms and extracting user purchasing patterns,

[1011] A means for generating product recommendations based on user-specific purchasing patterns,

[1012] A means of collecting the latest market activity information and making the best possible proposals to users,

[1013] A means of analyzing natural language input from users and providing appropriate information,

[1014] A means of selecting and notifying market activity information when arriving at a specific facility using location information,

[1015] A system that includes this.

[1016] (Claim 2)

[1017] The system according to claim 1, comprising means for providing the user with optimal product recommendations and market activity information, taking into account the user's purchasing patterns and the latest market activity information.

[1018] (Claim 3)

[1019] The system according to claim 1, comprising means for analyzing the aforementioned natural language input using natural language processing technology and providing information to the user in real time.

[1020] "Example 2 of combining an emotion engine"

[1021] (Claim 1)

[1022] A means of obtaining a user's payment history and storing it in a storage medium,

[1023] A method for analyzing payment history obtained using a learning model and extracting user purchasing behavior,

[1024] A means of creating product suggestions based on user-specific purchasing behavior,

[1025] A means of obtaining the latest promotional information and making the best possible proposals to users,

[1026] A means of acquiring user emotion data and analyzing it using an emotion recognition engine,

[1027] A means for integrating purchase behavior and promotional information based on emotion recognition results to generate optimal proposals,

[1028] A means of analyzing natural language input from users and providing appropriate information,

[1029] A system that includes this.

[1030] (Claim 2)

[1031] The system according to claim 1, comprising means for providing the user with optimal product suggestions and promotional information, taking into account the user's purchasing behavior, emotion recognition results, and the latest promotional information.

[1032] (Claim 3)

[1033] The system according to claim 1, comprising means for analyzing the aforementioned natural language input using natural language processing technology and providing information to the user in real time.

[1034] "Application example 2 when combining with an emotional engine"

[1035] (Claim 1)

[1036] A means for collecting the user's payment history and storing it in an information storage unit,

[1037] A method for analyzing payment history collected using machine learning techniques and extracting user transaction patterns,

[1038] A means for generating product recommendations based on user-specific purchasing patterns and emotional states,

[1039] A means of gathering the latest sales promotion information and making the best proposals to users,

[1040] A means of analyzing natural language input from users and providing appropriate information,

[1041] A system that includes this.

[1042] (Claim 2)

[1043] The system according to claim 1, comprising means for providing the user with optimal product recommendations and sales promotion information, taking into account the user's transaction patterns, emotional state, and the latest sales promotion information.

[1044] (Claim 3)

[1045] The system according to claim 1, comprising means for analyzing the aforementioned natural language input using natural language processing technology and providing information to the user in real time. [Explanation of Symbols]

[1046] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of collecting and storing users' payment history in a database, A method for analyzing payment history collected using machine learning algorithms and extracting user purchasing patterns, A means for generating product recommendations based on user-specific purchasing patterns, A means of collecting the latest market activity information and making the best possible proposals to users, A means of analyzing natural language input from users and providing appropriate information, A means of selecting and notifying market activity information when arriving at a specific facility using location information, A system that includes this.

2. The system according to claim 1, comprising means for providing the user with optimal product recommendations and market activity information, taking into account the user's purchasing patterns and the latest market activity information.

3. The system according to claim 1, comprising means for analyzing the aforementioned natural language input using natural language processing technology and providing information to the user in real time.

Images