system

Abstract

Provide a system. [Solution] A means for acquiring a receipt image; Optical character recognition means for extracting text information from the captured receipt image; means for classifying the extracted text information and assigning specific categories and tags; means for storing the classified information in a database; means for awarding points to a user account based on the stored information; an artificial intelligence means for analyzing the user's purchasing trends based on the stored information; means for generating personalized advertisements or coupons for the user based on the analysis results; A system including means for displaying advertisements or coupons on a user terminal.

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional electronic payment systems have limited means for effectively acquiring and analyzing user purchasing data, making it particularly difficult to acquire purchasing information from paper receipts. This has resulted in insufficient provision of personalized advertising based on users' purchasing habits and in generating insights that businesses can use to conduct effective sales promotion activities. Furthermore, for small stores, the increased effort and cost required for data acquisition makes it impractical. The present invention aims to solve these problems. [Means for solving the problem]

[0005] The present invention solves the above problems by providing a system including the following means.

[0006] A system including a means for acquiring receipt images, an optical character recognition means for extracting text information from the acquired receipt images, a means for classifying the extracted text information and assigning specific categories and tags, a means for storing the classified information in a database, a means for assigning points to a user account based on the stored information, an artificial intelligence means for analyzing users' purchasing trends based on the stored information, a means for generating personalized advertisements or coupons for users based on the analysis results, and a means for displaying the advertisements or coupons on user terminals makes it possible to efficiently acquire and analyze user purchasing data, thereby enabling users to earn points and enjoy personalized services, and companies to obtain insights for increasing sales based on the purchasing data.

[0007] Below are definitions of important terms contained in the claims.

[0008] A "receipt image" is image data of a paper or electronic receipt that a user receives at the time of purchase.

[0009] "Means for acquiring" refers to a function for taking a photo of or uploading a receipt on a user terminal and sending the image data to the system.

[0010] "Optical character recognition means" refers to OCR (Optical Character Recognition) technology used to mechanically read text information in receipt images.

[0011] "Text information" refers to information such as letters, numbers, and symbols printed on the receipt, and specifically includes the product name, price, purchase date and time, etc.

[0012] "Means of classification" refers to the function of automatically sorting extracted text information into specific categories or tags.

[0013] A "category" is a group divided by type of data, and indicates a classification such as food or daily necessities.

[0014] "Tags" are keywords or attribute information related to data that are added to make it easier to search and classify the data.

[0015] A "database" is an electronic recording medium for efficiently storing and managing large amounts of data.

[0016] The "means for awarding points" is a function that adds points to a user's account as an incentive based on receipt information provided by the user.

[0017] "Artificial intelligence means" refers to AI (Artificial Intelligence) technology and related algorithms for analyzing users' purchasing trends.

[0018] "Purchase trends" are predicted consumption patterns and preferences based on data about products and services a user has previously purchased.

[0019] "Means for generating personalized advertisements or coupons" refers to a function for creating individually optimized advertisements or coupons according to a user's purchasing habits.

[0020] "User terminal" means an electronic device used by a user to access the system, including a smartphone, tablet, computer, etc.

[0021] "Verification means" is a function for verifying the accuracy of the receipt information provided by the user.

[0022] A "dashboard" is an interface that allows companies to visually view data and understand analytical results.

[0023] "Insights" are useful findings and insights gained based on analytical data.

[0024] This clarifies the meaning of key words contained in the claims. [Brief explanation of the drawings]

[0025] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0026] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0027] First, the terms used in the following description will be explained.

[0028] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0029] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0030] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0031] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0032] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0033] [First embodiment]

[0034] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0035] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0036] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0037] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0038] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0039] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0040] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0041] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0042] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0043] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0044] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0045] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0046] The present invention provides an electronic payment system that acquires and analyzes receipt information and awards points to users, and a system that provides personalized advertisements and coupons based on the points. This system includes a user terminal, a server, a database, an artificial intelligence model, and related communication means.

[0047] Capture and send receipt images

[0048] Terminal (user terminal)

[0049] When a user launches the app, they are presented with the option to take a photo or upload a receipt.

[0050] The user either takes a photo of a paper receipt with the camera or selects an electronic receipt from within the device.

[0051] The terminal temporarily stores the photographed or uploaded receipt image and transmits the data to the server.

[0052] Examples:

[0053] The user presses the "take a photo" button to take a photo of the receipt with the smartphone camera, then presses the "send" button to send the image data to the server.

[0054] Receipt data processing and analysis

[0055] server

[0056] The server receives the receipt image data sent from the terminal.

[0057] The received image data is processed using OCR (Optical Character Recognition) to extract text information, including the product name, price, purchase date, etc.

[0058] The extracted text information is classified into categories and tags.

[0059] Examples:

[0060] The server processes the receipt image and extracts the text data "Product A - 200 yen," "Product B - 300 yen," and "October 1, 2023." This is then automatically classified into categories such as "food" and "daily necessities."

[0061] Data storage and point allocation

[0062] server

[0063] The classified text information is stored in a database, including the user ID, purchased item, price, purchase date and time, etc.

[0064] After confirming that the receipt has been successfully processed and the data has been saved, points are credited to the user's account.

[0065] The result of the points allocation is notified to the user terminal.

[0066] Terminal

[0067] The user's device receives the point allocation notification from the server and displays the notification within the app, allowing the user to confirm that they have earned points.

[0068] Examples:

[0069] The server awards 50 points to user ID "12345" and notifies the terminal of the result. The terminal displays the message "50 points have been awarded."

[0070] Data analytics and personalized advertising

[0071] server

[0072] The saved purchasing data is periodically input into the AI ​​model to analyze users' purchasing trends.

[0073] Based on the analysis results, optimal advertisements and coupons are generated for the user and sent to the terminal.

[0074] Terminal

[0075] The device receives advertisements and coupons sent from the server and displays them in a designated area within the app, allowing users to view offers based on their purchasing habits.

[0076] Examples:

[0077] The server analyzes the user's purchase history and finds that they frequently purchase items in the "food" category, generates new food-related product advertisements, sends them to the device, and displays "recommended foods" in the user's app.

[0078] Enterprise Dashboard

[0079] server

[0080] We provide a dashboard for businesses that displays aggregated data and analytical results.

[0081] The dashboard visually displays, for example, sales trends over a specific period or user purchasing trends, allowing companies to gain insights for formulating effective sales strategies.

[0082] Examples:

[0083] The server analyzes the information, such as "Sales in the food category have increased by 25% in the past month," and displays it on a dashboard. Based on this information, the company makes a plan to further strengthen its product lineup.

[0084] Verifying User Receipt Information

[0085] server

[0086] Implement functionality to validate the accuracy of receipt information submitted by users, including checking that the information contained in the receipt matches an existing database.

[0087] Only if the verification is successful, points are awarded and the user is notified.

[0088] Examples:

[0089] The server verifies that the purchase date and time, store name, product name, etc. on the receipt match the records in the database, and then awards points.

[0090] The above is a concrete example of how to implement the present invention. This system allows users to not only earn points but also enjoy personalized services based on their purchasing habits. It also enables companies to effectively utilize purchasing data to strengthen their sales promotion activities.

[0091] The processing flow will be explained below.

[0092] Step 1:

[0093] The user launches the app and selects the "Receipt Capture" or "Receipt Upload" function. The user either takes a photo of a paper receipt with the camera or selects an electronic receipt from within the device. When the user presses the "Send" button, the receipt image data is temporarily saved within the device.

[0094] Step 2:

[0095] The device prepares the captured or uploaded receipt image for transmission to the server, specifically by encoding the image data into the appropriate format and attaching metadata (user ID, timestamp, etc.).

[0096] Step 3:

[0097] The device sends the prepared data to the server, which usually uses the HTTPS protocol to ensure secure communication.

[0098] Step 4:

[0099] The server receives the receipt image data sent from the terminal, temporarily stores the received raw data, and prepares it for OCR processing.

[0100] Step 5:

[0101] The server performs OCR (optical character recognition) processing to extract text information (product name, price, purchase date and time, etc.) from the receipt image. For example, Tesseract or Google® Cloud Vision API is used as the OCR engine.

[0102] Step 6:

[0103] The server analyzes the extracted text information and assigns categories (e.g., food, daily necessities, etc.) and tags (e.g., sale, double points, etc.) using a pre-trained classification model.

[0104] Step 7:

[0105] The server stores the classified text information in a database, including the user ID, purchased item, price, purchase date and time, and category / tag.

[0106] Step 8:

[0107] The server verifies that the receipt has been processed successfully and then grants the points to the user's account by executing an SQL query using the user ID as the key to add the points.

[0108] Step 9:

[0109] The server generates a response to notify the user of the point allocation result. This is usually a JSON formatted message.

[0110] Step 10:

[0111] The device receives the response from the server and displays a notification to the user that points have been awarded. Specifically, the device uses the app's notification function to display a message such as "50 points have been awarded."

[0112] Step 11:

[0113] The server periodically supplies the stored purchase data to the AI ​​model to analyze users' purchasing trends. The analysis uses a combination of various data, such as user attributes and past purchase history.

[0114] Step 12:

[0115] The server generates personalized ads and coupons based on the results of the AI ​​model, which are tailored to the user's purchasing habits.

[0116] Step 13:

[0117] The server then sends the generated advertisements and coupons to the user's device, again using a secure communication protocol.

[0118] Step 14:

[0119] The device receives advertisements and coupon information sent from the server and displays them in a designated area within the app, allowing users to view offers based on their purchasing habits.

[0120] Step 15:

[0121] The server provides a dashboard for companies to display aggregated data and analysis results, visually displaying sales trends and user purchasing trends for a specific period.

[0122] Step 16:

[0123] The server verifies the accuracy of the receipt information provided by the user. This involves checking that the receipt contents match existing database records. Only if the verification is successful are points awarded and the user notified.

[0124] These are the processing steps and specific operations of the system program. This allows users to earn points through receipts and receive personalized advertisements and coupons. It also enables companies to optimize their sales promotion activities based on detailed purchase data.

[0125] Example 1

[0126] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0127] The present invention relates to a system that effectively acquires and analyzes receipt information and awards points to users. Conventional systems lack accurate extraction of receipt information and subsequent analysis, making it difficult to award points to users or provide personalized services. Furthermore, they lack effective data analysis and purchasing trend information for businesses. As a result, these systems fall short in improving user experience and promoting corporate sales.

[0128] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0129] In this invention, the server includes a means for acquiring receipt images, an optical character recognition means for extracting text information from the acquired receipt images, a means for classifying the extracted text information using natural language processing and assigning specific categories and tags, a means for saving the classified information in a database, a means for awarding points to a user account based on the saved information, a means for notifying the user terminal of the point awarding results, an artificial intelligence means for analyzing the user's purchasing habits based on the saved information, a means for generating personalized advertisements or coupons for the user based on the analysis results, and a means for displaying the advertisements or coupons on the user terminal. This allows users to easily acquire and analyze receipt information and earn points, as well as enjoy personalized services based on their purchasing habits. It also enables companies to effectively utilize purchasing data to strengthen their sales promotion activities.

[0130] A "user terminal" refers to an electronic device operated by a user, and includes smartphones, tablets, laptops, and the like.

[0131] "Receipt image" refers to image data of a receipt photographed or uploaded by a user.

[0132] A "server" refers to a centralized computer system that receives data sent from user terminals over a network and performs subsequent processing.

[0133] "Optical Character Recognition (OCR) Method" means a technology for extracting text information from receipt images, such as Tesseract, Google Cloud Vision, or similar tools.

[0134] "Natural language processing means" refers to processes that use algorithms or libraries, such as NLTK or spaCy, to classify extracted text information into specific categories or tags.

[0135] "Database" refers to a software system for systematically storing and managing classified information, and includes relational databases such as MySQL (registered trademark) and PostgreSQL.

[0136] "Artificial intelligence means" refers to machine learning models and algorithms used to analyze users' purchasing trends based on stored data, and uses frameworks such as TENSORFLOW (registered trademark) and PyTorch.

[0137] "Personalized ads or coupons" refers to ads or coupons that are individually generated based on a user's purchasing habits.

[0138] "Verification Procedure" refers to the process of verifying the accuracy of receipt information submitted by a user and ensuring it matches existing data in the database.

[0139] A "dashboard" is an interface for visually displaying aggregated data and analytical results for companies, providing data in the form of graphs, charts, etc.

[0140] The present invention provides an electronic payment system that acquires and analyzes receipt information and awards points to users, and a system that provides personalized advertisements and coupons based on the points. This system includes a user terminal, a server, a database, an artificial intelligence model, and related communication means.

[0141] Capture and send receipt images

[0142] When a user launches the app, a receipt capture or upload function is displayed. The user can capture a receipt image by using the camera to capture a paper receipt or by selecting an electronic receipt from within the device. The device temporarily stores the captured or uploaded receipt image and then sends the data to the server.

[0143] Example: A user presses the "take a photo" button to take a photo of a receipt with their smartphone camera, then presses the "send" button to send the image data to the server.

[0144] Receipt data processing and analysis

[0145] The server receives the receipt image data sent from the terminal and applies OCR (Optical Character Recognition) processing to the image. Specifically, it uses OCR tools such as Tesseract or Google Cloud Vision API to extract text information from the image. This extracted text information includes the product name, price, purchase date, etc.

[0146] Next, the extracted text information is classified using natural language processing (NLP) and assigned specific categories and tags, using NLP libraries such as Python's NLTK and spaCy.

[0147] Example: The server processes an image of a receipt and extracts the text data "Product A - 200 yen," "Product B - 300 yen," and "October 1, 2023." This data is then automatically classified into categories such as "food" and "daily necessities."

[0148] Data storage and point allocation

[0149] The server stores the classified text information in a database (e.g., MySQL or PostgreSQL), including the user ID, purchased item, price, purchase date and time, etc.

[0150] After confirming that the receipt has been successfully processed and the data has been saved, the server will award points to the user's account. The server will generate a notification message and send it to the user's terminal to notify them of the awarding result.

[0151] Example: The server awards 50 points to user ID "12345" and notifies the terminal of the result. The terminal displays the message "50 points have been awarded."

[0152] Data analytics and personalized advertising

[0153] The server periodically inputs the saved purchase data into an AI model to analyze users' purchasing trends. The AI ​​model uses TensorFlow and PyTorch. Based on the results of this analysis, optimal advertisements and coupons are generated for the user and sent to the device.

[0154] Example: The server analyzes the user's purchase history and finds that they frequently purchase items in the "food" category, generates new food-related product advertisements, sends them to the device, and displays "recommended foods" in the user's app.

[0155] Enterprise Dashboard

[0156] The server generates a dashboard for displaying aggregated data and analysis results for the company, visually displaying sales trends and user purchasing trends for a specific period.

[0157] Example: The server analyzes information such as "Sales in the food category have increased by 25% in the past month" and displays it on a dashboard. Based on this information, the company makes a plan to further strengthen its product lineup.

[0158] Verifying User Receipt Information

[0159] The server implements functionality to verify the accuracy of receipt information submitted by the user, including checking that the information contained in the receipt matches an existing database.

[0160] Example: The server verifies that the purchase date and time, store name, product name, etc. on the receipt match the records in the database, and then awards points.

[0161] Example prompt

[0162] "Please explain in detail how the server analyzes the receipt information and awards points to the user after the user takes a photo of the receipt and sends it to the server."

[0163] The above is a concrete example of how to implement the present invention. This system allows users to not only earn points but also enjoy personalized services based on their purchasing habits. It also enables companies to effectively utilize purchasing data to strengthen their sales promotion activities.

[0164] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0165] Step 1:

[0166] User

[0167] The user launches the smartphone app and selects the receipt capture or upload function. The user either takes a photo of a paper receipt using the device's camera or selects an electronic receipt from within the device. An image of the receipt is generated as input, which becomes the input data for the next step.

[0168] Specific operation:

[0169] Pressing the "Take a Receipt Photo" button will activate the camera, which will focus on the receipt and take a photo.

[0170] Press the "Upload" button and select the electronic receipt file on your device.

[0171] Step 2:

[0172] Terminal

[0173] The terminal temporarily stores the captured receipt image and then transmits the data to the server. The input is a photographed or uploaded receipt image. The output is image data that is sent to the server.

[0174] Specific operation:

[0175] The captured receipt image is previewed and the user presses the "Send" button.

[0176] The terminal transmits the image data to the server.

[0177] Step 3:

[0178] server

[0179] The server receives receipt image data sent from the terminal. The receipt image data is sent as input. The received image data is temporarily saved as output.

[0180] Specific operation:

[0181] The received image data is saved in a specific directory.

[0182] Step 4:

[0183] server

[0184] The server applies OCR (Optical Character Recognition) processing to the stored image data. Specifically, it uses Tesseract or Google Cloud Vision API to extract text information from the image. The input is the stored receipt image, and the output is the extracted text information.

[0185] Specific operation:

[0186] Start the OCR engine and input the image file.

[0187] Extract information such as product name, price, and purchase date and time.

[0188] Step 5:

[0189] server

[0190] The server uses natural language processing (NLP) to classify the extracted text information and assign specific categories and tags to it. It uses NLP libraries such as Python's NLTK or spaCy. The extracted text information is the input, and the classified text data is generated as the output.

[0191] Specific operation:

[0192] Analyzes text information and classifies product names and prices by category.

[0193] Add tags such as "food" and "daily necessities."

[0194] Step 6:

[0195] server

[0196] The server stores the classified text information in a database, using MySQL or PostgreSQL. The input is the classified text data, and the output is a new record stored in the database.

[0197] Specific operation:

[0198] Information such as user ID, product name, price, purchase date and time, etc. is inserted into the database via an SQL query.

[0199] Step 7:

[0200] server

[0201] The server assigns points to the user's account based on the information stored in the database. The input is the stored receipt information, and the output is the result of the points assignment.

[0202] Specific operation:

[0203] Run the point-granting algorithm and grant 50 points to user ID "12345".

[0204] A record of points awarded is stored in a database.

[0205] Step 8:

[0206] server

[0207] The server generates a notification message to notify the user of the point allocation result and sends it to the user terminal. The input is the point allocation result data, and the output is the generated notification message.

[0208] Specific operation:

[0209] The result data is sent to the terminal in JSON format.

[0210] Generates the message "50 points awarded."

[0211] Step 9:

[0212] Terminal

[0213] The device receives notification messages from the server and displays the notifications within the app. As input, it has the notification message sent by the server and as output, it generates the notification that is displayed to the user.

[0214] Specific operation:

[0215] The notification message is parsed and displayed in the user interface.

[0216] Step 10:

[0217] server

[0218] The server periodically inputs the saved purchase data into an AI model to analyze the user's purchasing trends. The AI ​​model uses TensorFlow and PyTorch. The input is the user's purchase history data, and the output is an analysis of purchasing trends.

[0219] Specific operation:

[0220] Purchase history data is fed into the AI ​​model, which then identifies purchasing patterns.

[0221] The analysis results include "User ID: 12345 frequently purchases food."

[0222] Step 11:

[0223] server

[0224] The server generates personalized advertisements and coupons based on the analysis results and sends them to the user's device. The input is the analysis results of purchasing trends, and the output is the generated advertisement and coupon data.

[0225] Specific operation:

[0226] Run ad generation algorithms to generate appropriate ads and coupons.

[0227] The generated advertisement is transmitted to the terminal.

[0228] Step 12:

[0229] Terminal

[0230] The device receives advertisements and coupon information sent from the server and displays them in a designated area within the app. The input is the advertisement data sent from the server, and the output is the advertisement displayed.

[0231] Specific operation:

[0232] The received advertising data is analyzed and displayed within the app.

[0233] The above are the processing steps of this system. The user, terminal, and server work together to carry out a series of processes to award points and provide personalized services.

[0234] (Application example 1)

[0235] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0236] While previous systems had the ability to use receipt information to award points and provide personalized advertisements and coupons, they lacked real-time support tools to help store associates improve their service. Furthermore, they lacked an immediate and intuitive interface for store associates to provide personalized service to customers. As a result, their effectiveness in improving customer satisfaction and maximizing store sales was limited.

[0237] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0238] In this invention, the server includes a means for acquiring a receipt image, an optical character recognition means for extracting text information from the acquired receipt image, and a means for classifying the extracted text information and assigning specific categories and tags. This enables a means for scanning receipt information and processing the data in real time. The server also includes a means installed in the smart glasses for providing display information when a store clerk provides customer service. This allows the store clerk to provide instant personalized service to customers.

[0239] definition statement

[0240] The "means for acquiring a receipt image" refers to a device such as a camera or scanner for electronically acquiring a receipt for a product purchased by a user.

[0241] An "optical character recognition means" is software or hardware for analyzing and extracting textual information from captured images.

[0242] The "means for assigning specific categories and tags" refers to a system that has the function of classifying extracted text information and automatically assigning appropriate categories and tags.

[0243] A "database storage means" is a data management device or system used to store classified information securely and efficiently.

[0244] The "means for adding points to a user account" is a system that automatically adds points to a user account based on the acquired and classified information.

[0245] "Artificial intelligence means" refers to machine learning models and algorithms used to analyze users' purchasing habits based on stored information.

[0246] The "means for generating personalized advertisements or coupons" is a system that generates advertisements and coupons optimized for each user based on analyzed purchasing data.

[0247] The "means for displaying on the user terminal" is an interface having a function for displaying the generated advertisements and coupons on the user terminal.

[0248] "Means installed on smart glasses to provide display information when store clerks provide customer service" refers to an application installed on smart glasses, which is a system that displays information necessary for store clerks to provide service to customers in real time.

[0249] "Means for scanning receipt information and processing data in real time" refers to hardware and software for quickly recognizing receipt information and immediately digitizing and processing that information.

[0250] MODE FOR CARRYING OUT THE INVENTION

[0251] The present invention provides an electronic payment system that acquires and analyzes receipt information, awards points to users, and provides personalized advertisements and coupons based on the points. The system includes smart glasses, a server, a database, an artificial intelligence model, and related communication means.

[0252] Capture and send receipt images

[0253] Terminal (smart glasses)

[0254] The store clerk puts on the smart glasses, launches the dedicated app, and scans the receipt for the item purchased by the user with the smart glasses' camera.

[0255] The scanned receipt image is temporarily stored in the smart glasses and the data is sent to the server.

[0256] Examples:

[0257] The store clerk presses the "scan" button and scans the receipt with the smart glasses' camera, after which the image data is automatically sent to the server.

[0258] Receipt data processing and analysis

[0259] server

[0260] The server receives the receipt image data sent from the terminal.

[0261] The received image data is processed using OCR (Optical Character Recognition) to extract text information, including the product name, price, purchase date, etc.

[0262] The extracted text information is classified into categories and tags.

[0263] Examples:

[0264] The server processes the receipt image and extracts the text data "Product A - 200 yen," "Product B - 300 yen," and "October 1, 2023." This is then automatically classified into categories such as "food" and "daily necessities."

[0265] Data storage and point allocation

[0266] server

[0267] The classified text information is stored in a database, including the user ID, purchased item, price, purchase date and time, etc.

[0268] After confirming that the receipt has been successfully processed and the data has been saved, points are credited to the user's account.

[0269] The results of point allocation are notified to the smart glasses.

[0270] Terminal (smart glasses)

[0271] The store clerk's smart glasses receive the point award notification from the server and display it as a message, allowing the store clerk to inform the customer that the user has earned points.

[0272] Examples:

[0273] The server awards 50 points to user ID "12345" and notifies the result to the smart glasses, which display the message "50 points have been awarded."

[0274] Data analytics and personalized advertising

[0275] server

[0276] The saved purchasing data is periodically input into the AI ​​model to analyze users' purchasing trends.

[0277] Based on the analysis results, optimal advertisements and coupons are generated for the user and sent to the smart glasses.

[0278] Terminal (smart glasses)

[0279] The smart glasses receive and display advertisements and coupon information sent from the server, and store clerks can use this information to suggest personalized products to customers.

[0280] Examples:

[0281] The server analyzes the user's purchase history and finds that they frequently purchase items in the "food" category, and generates advertisements for new food-related products. These advertisements are then sent to the smart glasses, and store clerks suggest "recommended foods" to the customer.

[0282] Enterprise Dashboard

[0283] server

[0284] We provide a dashboard for businesses that displays aggregated data and analytical results.

[0285] The dashboard visually displays, for example, sales trends over a specific period or user purchasing trends, allowing companies to gain insights for formulating effective sales strategies.

[0286] Examples:

[0287] The server analyzes the information, such as "Sales in the food category have increased by 25% in the past month," and displays it on a dashboard. Based on this information, the company makes a plan to further strengthen its product lineup.

[0288] Verifying User Receipt Information

[0289] server

[0290] Implement functionality to validate the accuracy of receipt information submitted by users, including checking that the information contained in the receipt matches an existing database.

[0291] Only if the verification is successful, points are awarded and the user is notified.

[0292] Examples:

[0293] The server verifies that the purchase date and time, store name, product name, etc. on the receipt match the records in the database, and then awards points.

[0294] Example prompt sentence:

[0295] Generate new product suggestions and coupons based on the purchase history of user ID "12345." Recent purchases include "Product A - 200 yen," "Product B - 300 yen," and "Purchase date - October 1, 2023."

[0296] The above is a concrete example of how to implement the present invention. This system allows users to not only earn points but also enjoy personalized services based on their purchasing habits. It also enables companies to effectively utilize purchasing data to strengthen their sales promotion activities.

[0297] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0298] Program processing steps

[0299] Step 1:

[0300] A store clerk wearing smart glasses launches a dedicated app and scans the receipt of the customer's purchase with the smart glasses' camera. The input is the scanned receipt image, and the output is image data temporarily stored in the smart glasses. Specifically, the store clerk presses the "scan" button and takes a photo of the receipt using the smart glasses' camera. The captured image is temporarily stored in the smart glasses' memory.

[0301] Step 2:

[0302] The smart glasses send the scanned receipt image to the server. The input is the temporarily stored receipt image data, and the output is the image data sent to the server. Specifically, the smart glasses' communication module is used to send the captured image to the server via a secure communication protocol.

[0303] Step 3:

[0304] The server applies OCR processing to the received receipt image data and extracts text information. The input is the receipt image data received by the server, and the output is the extracted text information. Specifically, the server uses OCR software to analyze the characters in the image and generate text data such as "Product A - 200 yen," "Product B - 300 yen," and "October 1, 2023."

[0305] Step 4:

[0306] The server classifies the extracted text information into categories and tags. The input is the extracted text information, and the output is data with categories and tags. Specifically, the server uses a rule-based or machine learning model to classify the information into categories such as "food" and "daily necessities" based on product name, price, purchase date, etc.

[0307] Step 5:

[0308] The server stores the classified text information in a database. The input is data with categories and tags, and the output is the information stored in the database. Specifically, the server stores information such as the user ID, purchased item, price, and purchase date and time in a relational database or cloud storage.

[0309] Step 6:

[0310] The server verifies the accuracy of the receipt information. The input is the information stored in the database, and the output is the result of the verification. Specifically, the server verifies that the receipt information is accurate by matching it with existing records in the database.

[0311] Step 7:

[0312] After the verification is successful, the server will grant points to the user account. The input is the successfully verified receipt information, and the output is the points granted to the user account. Specifically, the server calculates the points based on the user ID and updates the user's point balance.

[0313] Step 8:

[0314] The result of point allocation is notified to the smart glasses. The input is the point information allocated, and the output is a notification to the smart glasses. Specifically, the server sends the point allocation result to the smart glasses in real time, and the message "50 points have been allocated" is displayed on the store clerk's glasses.

[0315] Step 9:

[0316] The server inputs the stored purchasing data into an AI model to analyze the user's purchasing trends. The input is the stored purchasing data, and the output is the analysis results. Specifically, the server uses a machine learning model to analyze the user's purchasing patterns and predict future purchasing behavior.

[0317] Step 10:

[0318] Based on the analysis results, the server generates personalized advertisements and coupons for users. The input is the analysis results, and the output is the generated advertisements and coupons. Specifically, the server runs an algorithm to generate optimal advertisements and coupons for each user based on purchasing trends.

[0319] Step 11:

[0320] The generated advertisements and coupons are displayed on the smart glasses. The input is the generated advertisement or coupon information, and the output is the information displayed on the smart glasses. Specifically, the server sends the generated advertisement or coupon information to the smart glasses, and the store clerk provides services to the customer based on that information.

[0321] Example prompt sentence:

[0322] Generate new product suggestions and coupons based on the purchase history of user ID "12345." Recent purchases include "Product A - 200 yen," "Product B - 300 yen," and "Purchase date - October 1, 2023."

[0323] The above are the specific processing steps of the system program that realizes this application example. This system allows users to earn points and enjoy personalized services based on their purchasing habits. Store clerks can use the smart glasses to provide customers with personalized product suggestions and services in real time.

[0324] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0325] The present invention is a system that combines an electronic payment system that acquires and analyzes receipt information and awards points to users with an emotion engine that recognizes user emotions. This system includes a user terminal, a server, a database, an artificial intelligence model, the emotion engine, and related communication means.

[0326] Capture and send receipt images

[0327] Terminal (user terminal)

[0328] The user launches the app and selects the "Receipt Capture" or "Receipt Upload" function. The user either takes a photo of a paper receipt with the camera or selects an electronic receipt from within the device.

[0329] The terminal temporarily stores the photographed or uploaded receipt image and transmits the data to the server.

[0330] Examples:

[0331] The user presses the "take a photo" button to take a photo of the receipt with the smartphone camera, then presses the "send" button to send the image data to the server.

[0332] Receipt data processing and analysis

[0333] server

[0334] The server receives the receipt image data sent from the terminal.

[0335] The received image data is processed using OCR (Optical Character Recognition) to extract text information (product name, price, purchase date, etc.) using, for example, Tesseract or Google Cloud Vision API.

[0336] The extracted text information is classified into categories and tags.

[0337] Examples:

[0338] The server processes the receipt image and extracts the text data "Product A - 200 yen," "Product B - 300 yen," and "October 1, 2023." This is then automatically classified into categories such as "food" and "daily necessities."

[0339] Data storage and point allocation

[0340] server

[0341] The classified text information is stored in a database, including the user ID, purchased item, price, purchase date and time, etc.

[0342] After confirming that the receipt has been successfully processed and the data has been saved, points are credited to the user's account.

[0343] The result of the points allocation is notified to the user terminal.

[0344] Terminal

[0345] The user's device receives the point allocation notification from the server and displays the notification within the app, allowing the user to confirm that they have earned points.

[0346] Examples:

[0347] The server awards 50 points to user ID "12345" and notifies the terminal of the result. The terminal displays the message "50 points have been awarded."

[0348] Data analytics and personalized advertising

[0349] server

[0350] The saved purchase data is periodically input into the AI ​​model to analyze users' purchasing trends. The analysis uses a combination of various data such as user attributes and past purchase history.

[0351] Based on the analysis results, optimal advertisements and coupons are generated for the user and sent to the terminal.

[0352] Terminal

[0353] The device receives advertisements and coupon information sent from the server and displays them in a designated area within the app, allowing users to view offers based on their purchasing habits.

[0354] Examples:

[0355] The server analyzes the user's purchase history and finds that they frequently purchase items in the "food" category, generates new food-related product advertisements, sends them to the device, and displays "recommended foods" in the user's app.

[0356] Recognizing and utilizing user emotions

[0357] server

[0358] The emotion engine is incorporated to recognize user emotions. The emotion engine identifies emotions by analyzing, for example, feedback and product ratings entered by users within the app.

[0359] The recognized emotion data is integrated into the purchasing tendency analysis, specifically, to reflect whether the user is satisfied or dissatisfied with a particular product.

[0360] Terminal

[0361] When users enter feedback or ratings, the data is sent to the server, where the emotion engine analyzes it and generates emotion data.

[0362] Examples:

[0363] When a user enters feedback in the app such as "I am satisfied with this product," the emotion engine evaluates it as "satisfied" and adds it to the purchase data.

[0364] Personalized advertising based on emotional data

[0365] server

[0366] Generate more personalized ads and coupons based on user sentiment data, and improve the user experience by prioritizing ads related to products and services that generate high satisfaction.

[0367] Terminal

[0368] Receive personalized advertisements and coupon information based on emotional data and display them in designated areas within the app.

[0369] Examples:

[0370] The server generates related advertisements for products rated as "Satisfied" and displays them in the user's application. For example, a "10% off coupon for new products related to the product you were satisfied with" is displayed.

[0371] Enterprise Dashboard

[0372] server

[0373] It provides a dashboard for businesses to display aggregated data and analytical results, visually displaying insights based on sales trends, user purchasing habits, and sentiment data for a specific period.

[0374] By utilizing emotional data, companies can use it as a reference for more effective marketing strategies and product development.

[0375] Examples:

[0376] The dashboard displays information such as "Sales in the food category have increased by 25% in the past month" along with user satisfaction ratings for the relevant products, allowing companies to plan product improvements and new product development.

[0377] Verifying User Receipt Information

[0378] server

[0379] Validate the accuracy of the receipt information provided by the user. This involves checking whether the information contained in the receipt matches an existing database. Only if the validation is successful are points awarded and the user notified.

[0380] Examples:

[0381] The server verifies that the purchase date and time, store name, product name, etc. on the receipt match the records in the database, and then awards points.

[0382] The above is a specific embodiment for implementing the present invention. This system allows users to earn points through receipts and receive personalized advertisements and coupons, and further improves the user experience by utilizing emotional data. Furthermore, companies can optimize their sales promotion activities based on detailed purchasing data and emotional data.

[0383] The processing flow will be explained below.

[0384] Step 1:

[0385] The user launches the app and selects the "Receipt Capture" or "Receipt Upload" function. The user either takes a photo of a paper receipt with the camera or selects an electronic receipt from within the device. When the user presses the "Send" button, the receipt image data is temporarily saved within the device.

[0386] Step 2:

[0387] The device prepares the captured or uploaded receipt image for transmission to the server, specifically by encoding the image data into the appropriate format and attaching metadata (user ID, timestamp, etc.).

[0388] Step 3:

[0389] The device sends the prepared data to the server, which usually uses the HTTPS protocol to ensure secure communication.

[0390] Step 4:

[0391] The server receives the receipt image data sent from the terminal, temporarily stores the received raw data, and prepares it for OCR processing.

[0392] Step 5:

[0393] The server performs OCR (optical character recognition) processing to extract text information (product name, price, purchase date, etc.) from the receipt image. The OCR engine used can be, for example, Tesseract or Google Cloud Vision API.

[0394] Step 6:

[0395] The server analyzes the extracted text information and assigns categories (e.g., food, daily necessities, etc.) and tags (e.g., sale, double points, etc.) using a pre-trained classification model.

[0396] Step 7:

[0397] The server stores the classified text information in a database, including the user ID, purchased item, price, purchase date and time, and category / tag.

[0398] Step 8:

[0399] The server verifies that the receipt has been processed successfully and then grants the points to the user's account by executing an SQL query using the user ID as the key to add the points.

[0400] Step 9:

[0401] The server generates a response to notify the user of the point allocation result. This is usually a JSON formatted message.

[0402] Step 10:

[0403] The device receives the response from the server and displays a notification to the user that points have been awarded. Specifically, the device uses the app's notification function to display a message such as "50 points have been awarded."

[0404] Step 11:

[0405] The server periodically supplies the stored purchase data to the AI ​​model to analyze users' purchasing trends. The analysis uses a combination of various data, such as user attributes and past purchase history.

[0406] Step 12:

[0407] The server generates personalized ads and coupons based on the results of the AI ​​model, which are tailored to the user's purchasing habits.

[0408] Step 13:

[0409] The server then sends the generated advertisements and coupons to the user's device, again using a secure communication protocol.

[0410] Step 14:

[0411] The device receives advertisements and coupon information sent from the server and displays them in a designated area within the app, allowing users to view offers based on their purchasing habits.

[0412] Step 15:

[0413] The server runs an emotion engine to recognize the user's emotions and analyzes the feedback and product ratings that the user writes in the app, generating analyzed emotion data.

[0414] Step 16:

[0415] The device sends the feedback and evaluation data entered by the user to the server, where the emotion engine analyzes it and generates emotion data.

[0416] Step 17:

[0417] The server integrates the recognized emotion data into a purchasing tendency analysis, specifically, reflecting whether the user is satisfied or dissatisfied with a particular product.

[0418] Step 18:

[0419] The server generates more personalized advertisements and coupons based on the user's emotional data, improving the user experience by prioritizing advertisements related to products and services that generate high satisfaction.

[0420] Step 19:

[0421] The device receives personalized advertisements and coupons based on emotion data and displays them in a designated area within the app, allowing users to view emotion-based offers.

[0422] Step 20:

[0423] The server provides a dashboard for businesses to display aggregate data and sentiment data, visually displaying sales trends over a specific period, user purchasing habits, and insights based on sentiment data.

[0424] Step 21:

[0425] The server verifies the accuracy of the receipt information provided by the user. Validation involves checking whether the information contained in the receipt matches an existing database record. Only if the validation is successful are points awarded and the user notified.

[0426] Specific examples

[0427] For example, if a user enters feedback within the app such as "I was satisfied with this product," the emotion engine will rate it as "satisfied" and add it to the purchase data. The server will generate related advertisements for products rated as "satisfied" and display them in the user's app. A dashboard for businesses will display information such as "Sales in the food category have increased by 25% in the past month, and user satisfaction with the relevant products is high," allowing businesses to make plans for product improvements and new product development.

[0428] The above are the processing steps and specific operations of the system's program. This allows users to earn points through receipts and receive personalized advertisements and coupons, and also provides an improved user experience by utilizing emotional data. Furthermore, companies can optimize their sales promotion activities based on detailed purchasing data and emotional data.

[0429] Example 2

[0430] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0431] Conventional electronic payment systems provide functions for obtaining receipt information and awarding points, but they lack the ability to provide personalized advertising using user emotional data or to analyze detailed purchasing trends. Furthermore, they lack a means to provide businesses with analysis results that integrate purchasing data and emotional data. This has made it difficult to improve user experience and optimize corporate marketing strategies.

[0432] The identification process performed by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for acquiring a receipt image, an optical character recognition means for extracting text information from the acquired receipt image, a means for categorizing the extracted text information and assigning specific categories and tags, a means for saving the classified information in a database, a means for awarding points to a user account based on the saved information, an artificial intelligence means for analyzing a user's purchasing habits based on the saved information, a means for generating personalized advertisements or coupons for the user based on the analysis results, a means for displaying the advertisements or coupons on the user terminal, an emotion recognition means for recognizing the user's emotional data and further analyzing the purchasing habits based on the emotional data, a means for generating personalized advertisements or coupons based on the emotional data, and a means for providing a dashboard displaying aggregated data and analysis results to businesses. This allows users to earn points through receipts and receive personalized advertisements that utilize their emotional data. Businesses can also receive analysis results based on detailed purchasing data and emotional data via the dashboard, optimizing their sales promotion activities.

[0433] "Means for acquiring receipt images" refers to devices or software that have the function of electronically acquiring receipt images by a user taking a photo of a paper receipt with a camera or selecting an electronic receipt from within a terminal.

[0434] "Optical character recognition means" refers to technology or devices for extracting text information from an acquired receipt image, and specifically refers to devices that use OCR (optical character recognition) technology to read character information.

[0435] "Means for classifying extracted text information and assigning specific categories and tags" refers to software or algorithms for classifying text information obtained by OCR processing into specific categories (e.g., food, daily necessities, etc.) and tags (e.g., price, product name, etc.).

[0436] "Means for storing classified information in a database" refers to a device or system that stores the classified information extracted from receipts in a database for future reference and analysis.

[0437] "Means for awarding points to a user account based on stored information" means a system or algorithm for calculating and awarding points to a user's account based on stored information in a database.

[0438] "Artificial intelligence means" refers to machine learning algorithms and AI models that use stored receipt information and other data to analyze users' purchasing habits.

[0439] "Means for generating personalized advertisements or coupons" refers to systems or software for generating advertisements or coupons that are optimal for individual users based on the user's purchasing habits and emotional data.

[0440] "Means for displaying advertisements or coupons on a user terminal" refers to an interface or software for displaying the generated advertisements or coupons on a user's smartphone or other terminal.

[0441] "Emotion recognition means" refers to a device or software that analyzes user feedback and evaluation data to identify user emotions and reflect the results in purchasing trend analysis.

[0442] "A means of providing dashboards that display aggregated data and analytical results for businesses" refers to visual interfaces and software that allow businesses to grasp their sales trends, user purchasing habits, and sentiment data at a glance.

[0443] The present invention is a system that combines an electronic payment system that acquires and analyzes receipt information and awards points to users with an emotion engine that recognizes user emotions. This system includes a user terminal, a server, a database, an artificial intelligence model, the emotion engine, and related communication means.

[0444] Capture and send receipt images

[0445] Terminal

[0446] The user launches the smartphone app and selects the "Receipt Capture" or "Receipt Upload" function. The user either takes a photo of a paper receipt with the camera or selects an electronic receipt from within the device. The device temporarily stores the captured or uploaded receipt image and sends the data to the server.

[0447] Examples:

[0448] The user presses the "take a photo" button, takes a photo of the receipt with the smartphone camera, and then presses the "send" button to send the image data to the server.

[0449] Example prompt for generative AI model:

[0450] "Please explain the steps to take a picture of the receipt and send it to the server."

[0451] Receipt data processing and analysis

[0452] server

[0453] The server receives the receipt image data sent from the device. It performs OCR processing on the received image data using Tesseract or Google Cloud Vision API to extract text information (product name, price, purchase date and time, etc.) and classifies the extracted text information into categories and tags.

[0454] Examples:

[0455] The server processes the receipt image and extracts the text data "Product A - 200 yen," "Product B - 300 yen," and "October 1, 2023," and then classifies this into categories such as "food" and "daily necessities."

[0456] Example prompt for generative AI model:

[0457] "Please explain the steps to extract text information from receipt images and classify it into categories."

[0458] Data storage and point allocation

[0459] server

[0460] The classified text information is stored in a database. The stored information includes the user ID, purchased item, price, purchase date and time, etc. After confirming that the receipt has been processed correctly and the data has been saved, points are awarded to the user's account. The result of point awarding is notified to the user's device.

[0461] Terminal

[0462] The user device receives the point allocation notification from the server and displays the notification within the app.

[0463] Examples:

[0464] The server awards 50 points to user ID "12345" and notifies the terminal of the result. The terminal displays the message "50 points have been awarded."

[0465] Example prompt for generative AI model:

[0466] Please explain the process from when the user submits the receipt until points are awarded.

[0467] Data analytics and personalized advertising

[0468] server

[0469] The stored purchase data is periodically input into an AI model to analyze the user's purchasing trends. This analysis uses user attributes and past purchase history. Based on the analysis results, optimal advertisements and coupons are generated for the user and sent to the device.

[0470] Terminal

[0471] The device receives advertisements and coupon information sent from the server and displays it within the app.

[0472] Examples:

[0473] The server uses an AI model to analyze past purchase data, generate advertisements for new food-related products for users who frequently purchase products in the "food" category, and send them to the user's device. The user's device then displays "recommended food products" within the app.

[0474] Example prompt for generative AI model:

[0475] "Please explain the steps to analyze user purchasing data and generate optimal advertisements."

[0476] Recognizing and utilizing user emotions

[0477] server

[0478] Incorporate an emotion engine to recognize user emotions. Analyze feedback and product ratings written by users within the app to identify emotions. Integrate the recognized emotion data into purchasing tendency analysis.

[0479] Terminal

[0480] When users enter feedback or ratings, the data is sent to the server and analyzed by the emotion engine.

[0481] Examples:

[0482] The user enters feedback in the app, such as "I'm satisfied with this product," and the data is sent to the server. The emotion engine evaluates the feedback as "satisfied," and the result is reflected in the purchase data.

[0483] Example prompt for generative AI model:

[0484] "Please explain the steps to recognize user sentiment data and reflect it in purchasing data."

[0485] Personalized advertising based on emotional data

[0486] server

[0487] Generate personalized ads and coupons based on user sentiment data, and prioritize ads related to products and services that generate high satisfaction.

[0488] Terminal

[0489] Receive personalized ads and coupon information based on emotional data and display it within the app.

[0490] Examples:

[0491] The server will prioritize generating related advertisements for products that have been rated as "satisfactory." For example, a "10% off coupon for a new product related to the product you were satisfied with" will be sent to the device and displayed within the app.

[0492] Example prompt for generative AI model:

[0493] "Explain the steps to generate and display personalized ads based on emotional data."

[0494] Enterprise Dashboard

[0495] server

[0496] It provides a dashboard for businesses to display aggregated data and analytical results, visually displaying insights based on sales trends, user purchasing habits, and sentiment data.

[0497] Examples:

[0498] The dashboard displays information such as "Sales in the food category have increased by 25% in the past month" along with user satisfaction ratings for the relevant products. Companies can use this information to consider marketing strategies and product development policies.

[0499] Example prompt for generative AI model:

[0500] "Describe the data you want to display in your enterprise dashboard."

[0501] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0502] Step 1: Capture and send a receipt image

[0503] 1-1. The user launches the app on their smartphone and selects the "take a photo of a receipt" or "upload a receipt" function.

[0504] 1-2. The user takes a photo of a paper receipt with the camera or selects an electronic receipt from within the device. The input data is an image file of the paper or electronic receipt.

[0505] 1-3. The device temporarily saves the captured or uploaded receipt image and sends the data to the server. The output data is the receipt image file sent to the server.

[0506] Specific behavior:

[0507] The user presses the "take a photo" button to take a photo of the receipt with the smartphone camera, then presses the "send" button, and the device transfers the image data to the server.

[0508] Step 2: Process and parse receipt data

[0509] 2-1. The server receives the receipt image data sent from the terminal. The input data is the receipt image file sent from the terminal.

[0510] 2-2. The server uses Tesseract or Google Cloud Vision API to perform OCR processing on the image data received and extracts text information (product name, price, purchase date, etc.). OCR processing is performed as data processing, and the output data is the extracted text information.

[0511] 2-3. Classify the extracted text information into categories and tags. Data classification is performed, and the output data is text information with categories or tags assigned.

[0512] Specific behavior:

[0513] After the server obtains the receipt image data, it calls the Tesseract OCR engine to perform character recognition, generating text data such as "Product A - 200 yen," "Product B - 300 yen," and "October 1, 2023," which is then automatically classified into categories such as "food" and "daily necessities."

[0514] Step 3: Save your data and earn points

[0515] 3-1. The server saves the classified text information in a database. The input data is text information with categories. The data is saved, and the output data is the information stored in the database.

[0516] 3-2. After confirming that the receipt has been successfully processed and the data has been saved, the server will award points to the user account. The input data is the saved data. The awarding process is executed, and the output data is the awarded points.

[0517] 3-3. The server notifies the user terminal of the point allocation result. The input data is the point allocation result. The notification is made, and the output data is the notification information sent to the user terminal.

[0518] Specific behavior:

[0519] The server saves the information "User ID: 12345, Product: Product A, Price: 200 yen, Purchase date: October 1, 2023" in the database, triggers the point allocation process, adds 50 points to user ID "12345", and sends the result to the terminal. The user terminal displays the message "50 points have been allocated."

[0520] Step 4: Data analysis and personalized advertising

[0521] 4-1. The server inputs the stored purchasing data into the AI ​​model to analyze the user's purchasing trends. The input data is the stored purchasing data. Data analysis is performed, and the output data is the analysis results.

[0522] 4-2. Based on the analysis results, the server generates the optimal advertisement or coupon for the user and sends it to the terminal. The input data is the analysis results. The generation process is carried out, and the output data is the generated advertisement or coupon.

[0523] 4-3. The device receives the advertisement or coupon information sent from the server and displays it within the app. The input data is the advertisement or coupon information sent from the server. The display process is performed, and the output data is the advertisement or coupon displayed within the app.

[0524] Specific behavior:

[0525] The server uses an AI model to analyze past purchase data, generate advertisements for new food-related products for users who frequently purchase products in the "food" category, and send them to the user's device. The user's device then displays "recommended food products" within the app.

[0526] Step 5: Recognize and leverage user emotions

[0527] 5-1. The server incorporates an emotion engine to recognize the user's emotions. The input data is the feedback and product ratings entered by the user within the app. An analysis process is performed, and the output data is the recognized emotion data.

[0528] 5-2. The server integrates the recognized emotion data into the purchasing tendency analysis. The input data is emotion data. The integration process is carried out, and the output data is the integrated analysis data.

[0529] 5-3. When a user inputs feedback or evaluation, the data is sent to the server and analyzed by the emotion engine. The input data is the feedback or evaluation data. The analysis process is performed, and the output data is the analyzed emotion data.

[0530] Specific behavior:

[0531] The user enters feedback in the app, such as "I'm satisfied with this product," and the data is sent to the server. The emotion engine evaluates the feedback as "satisfied" and integrates the evaluation into the purchase data.

[0532] Step 6: Personalized advertising based on emotional data

[0533] 6-1. The server generates personalized advertisements and coupons based on the user's emotional data. The input data is the emotional data. The generation process is performed, and the output data is personalized advertisements and coupons.

[0534] 6-2. The server sends advertisements and coupons to the user terminal. The input data is the personalized advertisements and coupons. The transmission process is carried out, and the output data is the advertisement and coupon data sent to the user terminal.

[0535] 6-3. The device receives advertisements or coupon information based on the emotion data and displays it within the app. The input data is advertisements or coupon information based on the emotion data. The display process is performed, and the output data is the advertisement or coupon displayed within the app.

[0536] Specific behavior:

[0537] The server generates related advertisements for products rated as "satisfactory" and sends them to the user's device. For example, a "10% off coupon for a new product related to the product you were satisfied with" is displayed in the app.

[0538] Step 7: Enterprise dashboard

[0539] 7-1. The server provides a dashboard for companies to display aggregated data and analysis results. The input data is the aggregated data and analysis results. The display process is carried out, and the output data is the information displayed on the dashboard.

[0540] Specific behavior:

[0541] The dashboard displays information such as "Sales in the food category have increased by 25% in the past month" along with user satisfaction ratings for the relevant products. Companies can use this information to consider the direction of their marketing strategies and product development.

[0542] (Application example 2)

[0543] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0544] Current electronic payment systems are limited to collecting receipt information and awarding points, and lack a deep understanding of users' purchasing behavior and emotions. Furthermore, current systems lack the functionality to provide personalized advertising or coupons that take user emotions into account, preventing improvements to the user experience. Furthermore, the data provided to businesses is limited and insufficient for use in marketing and product development.

[0545] The identification process performed by the identification processing unit 290 of the data processing device 12 in Application Example 2 is implemented by the following means. In this invention, the server includes: means for acquiring receipt images; optical character recognition means for extracting text information from the acquired receipt images; means for categorizing the extracted text information and assigning specific categories and tags; means for saving the classified information in a database; means for awarding points to a user account based on the saved information; artificial intelligence means for analyzing users' purchasing trends based on the saved information; means including an emotion engine for recognizing and integrating user emotions into the analysis; means for generating personalized advertisements or coupons for users based on the analysis results; and means for displaying the advertisements or coupons on the user terminal. This enables analysis of user purchasing behavior and emotions based on receipt information, thereby improving the user experience by providing more accurate personalized advertisements and coupons. Furthermore, providing a wide range of data analysis results through a corporate dashboard can improve the quality of marketing strategies and product development.

[0546] The "receipt image acquisition means" is a means for a user to take or upload a receipt image.

[0547] The "optical character recognition means" is a means for extracting text information from the captured receipt image.

[0548] The "text information classification means" is a means for classifying extracted text information and assigning specific categories and tags to it.

[0549] "Database storage means" refers to a means for safely storing classified information in a database.

[0550] The "points granting means" is a means for granting points to a user account based on the stored information.

[0551] "Artificial intelligence means" refers to means for analyzing users' purchasing trends based on stored information.

[0552] An "emotion engine" is a means for recognizing and integrating user emotions into analysis.

[0553] The "personalized advertisement generating means" is a means for generating advertisements or coupons individually tailored for users based on the analysis results.

[0554] The "advertisement display means" is a means for displaying the generated advertisement or coupon on the user terminal.

[0555] This invention combines an emotion engine with an electronic payment system that acquires and analyzes receipt information and awards points to users. This system is realized by a program consisting of the following elements:

[0556] First, the user terminal provides a means for acquiring receipt images. The user launches the application and sends the receipt image to the system by taking a photo or uploading it. This process uses the smartphone's camera or image upload function.

[0557] The server then extracts text information from the captured receipt image using optical character recognition (OCR) software such as OpenCV or Tesseract. The extracted text information includes data points such as product name, price, and purchase date and time.

[0558] The extracted text information is then classified and assigned categories and tags. The server stores the classified information in a database, including the user ID, purchased item, price, purchase date and time, etc.

[0559] Based on the saved information, the server will assign points to the user's account. The result of the points assignment will be notified to the user's device, and a message stating "Points have been assigned" will be displayed within the app.

[0560] Furthermore, the server uses artificial intelligence to analyze the stored purchase data and understand the user's purchasing trends. This analysis integrates data such as past purchase history and user attributes. This process uses a generative AI model.

[0561] The emotion engine recognizes user emotions and integrates them into purchasing trend analysis. For example, when a user enters product ratings or feedback within the app, that data is sent to the emotion engine, and the system generates emotion data such as "satisfied" or "dissatisfied."

[0562] Based on this emotional data, a means is provided for generating more personalized advertisements and coupons. The server creates optimal advertisements and coupons based on the user's purchasing tendencies and emotional data, and sends them to the user's device. The user's device then displays these advertisements and coupons in a designated area within the app.

[0563] As a specific example, if a user purchases "milk" and rates it as "very satisfied," the system will generate a 10% off coupon for the next purchase related to "milk" and provide it to the user.

[0564] Additionally, a dashboard for businesses is provided to visually display sales trends and purchasing habits, including insights based on sentiment data, which can be used by businesses to develop more effective marketing strategies and product development.

[0565] Finally, a means is provided for verifying the accuracy of the receipt information provided by the user. The server compares the receipt information with an existing database, confirming accuracy, and then awards points. In this way, the system improves the user experience and provides valuable data for businesses.

[0566] (Example of a prompt)

[0567] Please enter your thoughts about the product you purchased, "Milk." Example: "I am very satisfied with this milk."

[0568] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0569] Step 1: Get the receipt image

[0570] The user launches the smartphone app and selects the "take a photo of a receipt" or "upload a receipt" function. The user either takes a photo of a paper receipt with the camera or selects an electronic receipt from within the device. The device temporarily saves the captured or uploaded receipt image. The input is the receipt image selected by the user, and the temporarily saved image data is generated as the output.

[0571] Step 2: Send a receipt image

[0572] The user device sends the temporarily saved receipt image to the server. Here, the receipt image data is sent as input to the server and a success response is received. Specifically, the image data is sent to a specified endpoint on the server using the HTTP protocol.

[0573] Step 3: Optical Character Recognition (OCR)

[0574] The server applies OCR processing to the receipt image data it receives. The input is the receipt image data, and the output is the extracted text information. Specifically, it uses Tesseract and the Google Cloud Vision API to extract the text information.

[0575] Step 4: Classifying text information

[0576] The server classifies the extracted text information into categories and tags. The input is the text information resulting from the OCR process, and the output is the categorized and tagged text information. Here, classification is performed using a predefined rule-based or machine learning algorithm.

[0577] Step 5: Saving to the Database

[0578] The server stores the categorized text information in a database. The input is the categorized and tagged text information, and the output is a record stored in the database. Specifically, the database operation involves information such as the user ID, purchased item, price, and purchase date and time.

[0579] Step 6: Points awarded

[0580] The server will then assign points to the user's account based on the stored information. The input is the purchase data in the database, and the output is the updated user's points information. Specifically, points are calculated according to existing business rules, and the user's points balance is updated.

[0581] Step 7: Notification of points awarded

[0582] The server notifies the user device of the point awarding results. The input is the updated user point information, and the output is a notification message. Specifically, a push notification or in-app notification is used to send a message saying "Points have been awarded."

[0583] Step 8: Analyze purchasing data

[0584] The server periodically inputs the stored purchasing data into the AI ​​model to analyze the user's purchasing trends. The input is the purchasing data in the database, and the output is the analysis results of purchasing trends. The generative AI model combines past purchasing history and user attributes to analyze purchasing trends.

[0585] Step 9: Recognizing Emotional Data

[0586] The server uses an emotion engine to recognize the user's emotions. The input is the user's feedback and evaluation data, and the output is the recognized emotion data. Specifically, the emotion engine analyzes the user's input data and extracts emotions such as "satisfaction" or "dissatisfaction."

[0587] Step 10: Generate personalized ads

[0588] The server generates personalized advertisements and coupons based on purchasing trend analysis and emotional data. The input is the purchasing trend analysis results and emotional data, and the output is individually tailored advertisements and coupon information. Here, optimal advertisements are generated based on the user's interests and satisfaction.

[0589] Step 11: Displaying Ads

[0590] The user device receives the advertisements and coupon information sent from the server and displays them in a designated area within the app. The input is personalized advertisements and coupon information, and the output is the advertisements and coupons displayed on the user's device screen.

[0591] Finally, as a concrete example of a prompt:

[0592] "Please enter your thoughts about the product you purchased, 'Milk'. Example: 'I am very satisfied with this milk.'"

[0593] is used.

[0594] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0595] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0596] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0597] [Second embodiment]

[0598] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0599] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0600] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0601] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0602] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0603] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0604] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0605] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0606] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0607] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0608] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0609] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0610] The present invention provides an electronic payment system that acquires and analyzes receipt information and awards points to users, and a system that provides personalized advertisements and coupons based on the points. This system includes a user terminal, a server, a database, an artificial intelligence model, and related communication means.

[0611] Capture and send receipt images

[0612] Terminal (user terminal)

[0613] When a user launches the app, they are presented with the option to take a photo or upload a receipt.

[0614] The user either takes a photo of a paper receipt with the camera or selects an electronic receipt from within the device.

[0615] The terminal temporarily stores the photographed or uploaded receipt image and transmits the data to the server.

[0616] Examples:

[0617] The user presses the "take a photo" button to take a photo of the receipt with the smartphone camera, then presses the "send" button to send the image data to the server.

[0618] Receipt data processing and analysis

[0619] server

[0620] The server receives the receipt image data sent from the terminal.

[0621] The received image data is processed using OCR (Optical Character Recognition) to extract text information, including the product name, price, purchase date, etc.

[0622] The extracted text information is classified into categories and tags.

[0623] Examples:

[0624] The server processes the receipt image and extracts the text data "Product A - 200 yen," "Product B - 300 yen," and "October 1, 2023." This is then automatically classified into categories such as "food" and "daily necessities."

[0625] Data storage and point allocation

[0626] server

[0627] The classified text information is stored in a database, including the user ID, purchased item, price, purchase date and time, etc.

[0628] After confirming that the receipt has been successfully processed and the data has been saved, points are credited to the user's account.

[0629] The result of the points allocation is notified to the user terminal.

[0630] Terminal

[0631] The user's device receives the point allocation notification from the server and displays the notification within the app, allowing the user to confirm that they have earned points.

[0632] Examples:

[0633] The server awards 50 points to user ID "12345" and notifies the terminal of the result. The terminal displays the message "50 points have been awarded."

[0634] Data analytics and personalized advertising

[0635] server

[0636] The saved purchasing data is periodically input into the AI ​​model to analyze users' purchasing trends.

[0637] Based on the analysis results, optimal advertisements and coupons are generated for the user and sent to the terminal.

[0638] Terminal

[0639] The device receives advertisements and coupons sent from the server and displays them in a designated area within the app, allowing users to view offers based on their purchasing habits.

[0640] Examples:

[0641] The server analyzes the user's purchase history and finds that they frequently purchase items in the "food" category, generates new food-related product advertisements, sends them to the device, and displays "recommended foods" in the user's app.

[0642] Enterprise Dashboard

[0643] server

[0644] We provide a dashboard for businesses that displays aggregated data and analytical results.

[0645] The dashboard visually displays, for example, sales trends over a specific period or user purchasing trends, allowing companies to gain insights for formulating effective sales strategies.

[0646] Examples:

[0647] The server analyzes the information, such as "Sales in the food category have increased by 25% in the past month," and displays it on a dashboard. Based on this information, the company makes a plan to further strengthen its product lineup.

[0648] Verifying User Receipt Information

[0649] server

[0650] Implement functionality to validate the accuracy of receipt information submitted by users, including checking that the information contained in the receipt matches an existing database.

[0651] Only if the verification is successful, points are awarded and the user is notified.

[0652] Examples:

[0653] The server verifies that the purchase date and time, store name, product name, etc. on the receipt match the records in the database, and then awards points.

[0654] The above is a concrete example of how to implement the present invention. This system allows users to not only earn points but also enjoy personalized services based on their purchasing habits. It also enables companies to effectively utilize purchasing data to strengthen their sales promotion activities.

[0655] The processing flow will be explained below.

[0656] Step 1:

[0657] The user launches the app and selects the "Receipt Capture" or "Receipt Upload" function. The user either takes a photo of a paper receipt with the camera or selects an electronic receipt from within the device. When the user presses the "Send" button, the receipt image data is temporarily saved within the device.

[0658] Step 2:

[0659] The device prepares the captured or uploaded receipt image for transmission to the server, specifically by encoding the image data into the appropriate format and attaching metadata (user ID, timestamp, etc.).

[0660] Step 3:

[0661] The device sends the prepared data to the server, which usually uses the HTTPS protocol to ensure secure communication.

[0662] Step 4:

[0663] The server receives the receipt image data sent from the terminal, temporarily stores the received raw data, and prepares it for OCR processing.

[0664] Step 5:

[0665] The server performs OCR (optical character recognition) processing to extract text information (product name, price, purchase date, etc.) from the receipt image. The OCR engine used can be, for example, Tesseract or Google Cloud Vision API.

[0666] Step 6:

[0667] The server analyzes the extracted text information and assigns categories (e.g., food, daily necessities, etc.) and tags (e.g., sale, double points, etc.) using a pre-trained classification model.

[0668] Step 7:

[0669] The server stores the classified text information in a database, including the user ID, purchased item, price, purchase date and time, and category / tag.

[0670] Step 8:

[0671] The server verifies that the receipt has been processed successfully and then grants the points to the user's account by executing an SQL query using the user ID as the key to add the points.

[0672] Step 9:

[0673] The server generates a response to notify the user of the point allocation result. This is usually a JSON formatted message.

[0674] Step 10:

[0675] The device receives the response from the server and displays a notification to the user that points have been awarded. Specifically, the device uses the app's notification function to display a message such as "50 points have been awarded."

[0676] Step 11:

[0677] The server periodically supplies the stored purchase data to the AI ​​model to analyze users' purchasing trends. The analysis uses a combination of various data, such as user attributes and past purchase history.

[0678] Step 12:

[0679] The server generates personalized ads and coupons based on the results of the AI ​​model, which are tailored to the user's purchasing habits.

[0680] Step 13:

[0681] The server then sends the generated advertisements and coupons to the user's device, again using a secure communication protocol.

[0682] Step 14:

[0683] The device receives advertisements and coupon information sent from the server and displays them in a designated area within the app, allowing users to view offers based on their purchasing habits.

[0684] Step 15:

[0685] The server provides a dashboard for companies to display aggregated data and analysis results, visually displaying sales trends and user purchasing trends for a specific period.

[0686] Step 16:

[0687] The server verifies the accuracy of the receipt information provided by the user. This involves checking that the receipt contents match existing database records. Only if the verification is successful are points awarded and the user notified.

[0688] These are the processing steps and specific operations of the system program. This allows users to earn points through receipts and receive personalized advertisements and coupons. It also enables companies to optimize their sales promotion activities based on detailed purchase data.

[0689] Example 1

[0690] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0691] The present invention relates to a system that effectively acquires and analyzes receipt information and awards points to users. Conventional systems lack accurate extraction of receipt information and subsequent analysis, making it difficult to award points to users or provide personalized services. Furthermore, they lack effective data analysis and purchasing trend information for businesses. As a result, these systems fall short in improving user experience and promoting corporate sales.

[0692] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0693] In this invention, the server includes a means for acquiring receipt images, an optical character recognition means for extracting text information from the acquired receipt images, a means for classifying the extracted text information using natural language processing and assigning specific categories and tags, a means for saving the classified information in a database, a means for awarding points to a user account based on the saved information, a means for notifying the user terminal of the point awarding results, an artificial intelligence means for analyzing the user's purchasing habits based on the saved information, a means for generating personalized advertisements or coupons for the user based on the analysis results, and a means for displaying the advertisements or coupons on the user terminal. This allows users to easily acquire and analyze receipt information and earn points, as well as enjoy personalized services based on their purchasing habits. It also enables companies to effectively utilize purchasing data to strengthen their sales promotion activities.

[0694] A "user terminal" refers to an electronic device operated by a user, and includes smartphones, tablets, laptops, and the like.

[0695] "Receipt image" refers to image data of a receipt photographed or uploaded by a user.

[0696] A "server" refers to a centralized computer system that receives data sent from user terminals over a network and performs subsequent processing.

[0697] "Optical Character Recognition (OCR) Method" means a technology for extracting text information from receipt images, such as Tesseract, Google Cloud Vision, or similar tools.

[0698] "Natural language processing means" refers to processes that use algorithms or libraries, such as NLTK or spaCy, to classify extracted text information into specific categories or tags.

[0699] "Database" refers to a software system for systematically storing and managing classified information, including relational databases such as MySQL and PostgreSQL.

[0700] "Artificial intelligence means" refers to machine learning models and algorithms used to analyze user purchasing trends based on stored data, and uses frameworks such as TensorFlow and PyTorch.

[0701] "Personalized ads or coupons" refers to ads or coupons that are individually generated based on a user's purchasing habits.

[0702] "Verification Procedure" refers to the process of verifying the accuracy of receipt information submitted by a user and ensuring it matches existing data in the database.

[0703] A "dashboard" is an interface for visually displaying aggregated data and analytical results for companies, providing data in the form of graphs, charts, etc.

[0704] The present invention provides an electronic payment system that acquires and analyzes receipt information and awards points to users, and a system that provides personalized advertisements and coupons based on the points. This system includes a user terminal, a server, a database, an artificial intelligence model, and related communication means.

[0705] Capture and send receipt images

[0706] When a user launches the app, a receipt capture or upload function is displayed. The user can capture a receipt image by using the camera to capture a paper receipt or by selecting an electronic receipt from within the device. The device temporarily stores the captured or uploaded receipt image and then sends the data to the server.

[0707] Example: A user presses the "take a photo" button to take a photo of a receipt with their smartphone camera, then presses the "send" button to send the image data to the server.

[0708] Receipt data processing and analysis

[0709] The server receives the receipt image data sent from the terminal and applies OCR (Optical Character Recognition) processing to the image. Specifically, it uses OCR tools such as Tesseract or Google Cloud Vision API to extract text information from the image. This extracted text information includes the product name, price, purchase date, etc.

[0710] Next, the extracted text information is classified using natural language processing (NLP) and assigned specific categories and tags, using NLP libraries such as Python's NLTK and spaCy.

[0711] Example: The server processes an image of a receipt and extracts the text data "Product A - 200 yen," "Product B - 300 yen," and "October 1, 2023." This data is then automatically classified into categories such as "food" and "daily necessities."

[0712] Data storage and point allocation

[0713] The server stores the classified text information in a database (e.g., MySQL or PostgreSQL), including the user ID, purchased item, price, purchase date and time, etc.

[0714] After confirming that the receipt has been successfully processed and the data has been saved, the server will award points to the user's account. The server will generate a notification message and send it to the user's terminal to notify them of the awarding result.

[0715] Example: The server awards 50 points to user ID "12345" and notifies the terminal of the result. The terminal displays the message "50 points have been awarded."

[0716] Data analytics and personalized advertising

[0717] The server periodically inputs the saved purchase data into an AI model to analyze users' purchasing trends. The AI ​​model uses TensorFlow and PyTorch. Based on the results of this analysis, optimal advertisements and coupons are generated for the user and sent to the device.

[0718] Example: The server analyzes the user's purchase history and finds that they frequently purchase items in the "food" category, generates new food-related product advertisements, sends them to the device, and displays "recommended foods" in the user's app.

[0719] Enterprise Dashboard

[0720] The server generates a dashboard for displaying aggregated data and analysis results for the company, visually displaying sales trends and user purchasing trends for a specific period.

[0721] Example: The server analyzes information such as "Sales in the food category have increased by 25% in the past month" and displays it on a dashboard. Based on this information, the company makes a plan to further strengthen its product lineup.

[0722] Verifying User Receipt Information

[0723] The server implements functionality to verify the accuracy of receipt information submitted by the user, including checking that the information contained in the receipt matches an existing database.

[0724] Example: The server verifies that the purchase date and time, store name, product name, etc. on the receipt match the records in the database, and then awards points.

[0725] Example prompt

[0726] "Please explain in detail how the server analyzes the receipt information and awards points to the user after the user takes a photo of the receipt and sends it to the server."

[0727] The above is a concrete example of how to implement the present invention. This system allows users to not only earn points but also enjoy personalized services based on their purchasing habits. It also enables companies to effectively utilize purchasing data to strengthen their sales promotion activities.

[0728] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0729] Step 1:

[0730] User

[0731] The user launches the smartphone app and selects the receipt capture or upload function. The user either takes a photo of a paper receipt using the device's camera or selects an electronic receipt from within the device. An image of the receipt is generated as input, which becomes the input data for the next step.

[0732] Specific operation:

[0733] Pressing the "Take a Receipt Photo" button will activate the camera, which will focus on the receipt and take a photo.

[0734] Press the "Upload" button and select the electronic receipt file on your device.

[0735] Step 2:

[0736] Terminal

[0737] The terminal temporarily stores the captured receipt image and then transmits the data to the server. The input is a photographed or uploaded receipt image. The output is image data that is sent to the server.

[0738] Specific operation:

[0739] The captured receipt image is previewed and the user presses the "Send" button.

[0740] The terminal transmits the image data to the server.

[0741] Step 3:

[0742] server

[0743] The server receives receipt image data sent from the terminal. The receipt image data is sent as input. The received image data is temporarily saved as output.

[0744] Specific operation:

[0745] The received image data is saved in a specific directory.

[0746] Step 4:

[0747] server

[0748] The server applies OCR (Optical Character Recognition) processing to the stored image data. Specifically, it uses Tesseract or Google Cloud Vision API to extract text information from the image. The input is the stored receipt image, and the output is the extracted text information.

[0749] Specific operation:

[0750] Start the OCR engine and input the image file.

[0751] Extract information such as product name, price, and purchase date and time.

[0752] Step 5:

[0753] server

[0754] The server uses natural language processing (NLP) to classify the extracted text information and assign specific categories and tags to it. It uses NLP libraries such as Python's NLTK or spaCy. The extracted text information is the input, and the classified text data is generated as the output.

[0755] Specific operation:

[0756] Analyzes text information and classifies product names and prices by category.

[0757] Add tags such as "food" and "daily necessities."

[0758] Step 6:

[0759] server

[0760] The server stores the classified text information in a database, using MySQL or PostgreSQL. The input is the classified text data, and the output is a new record stored in the database.

[0761] Specific operation:

[0762] Information such as user ID, product name, price, purchase date and time, etc. is inserted into the database via an SQL query.

[0763] Step 7:

[0764] server

[0765] The server assigns points to the user's account based on the information stored in the database. The input is the stored receipt information, and the output is the result of the points assignment.

[0766] Specific operation:

[0767] Run the point-granting algorithm and grant 50 points to user ID "12345".

[0768] A record of points awarded is stored in a database.

[0769] Step 8:

[0770] server

[0771] The server generates a notification message to notify the user of the point allocation result and sends it to the user terminal. The input is the point allocation result data, and the output is the generated notification message.

[0772] Specific operation:

[0773] The result data is sent to the terminal in JSON format.

[0774] Generates the message "50 points awarded."

[0775] Step 9:

[0776] Terminal

[0777] The device receives notification messages from the server and displays the notifications within the app. As input, it has the notification message sent by the server and as output, it generates the notification that is displayed to the user.

[0778] Specific operation:

[0779] The notification message is parsed and displayed in the user interface.

[0780] Step 10:

[0781] server

[0782] The server periodically inputs the saved purchase data into an AI model to analyze the user's purchasing trends. The AI ​​model uses TensorFlow and PyTorch. The input is the user's purchase history data, and the output is an analysis of purchasing trends.

[0783] Specific operation:

[0784] Purchase history data is fed into the AI ​​model, which then identifies purchasing patterns.

[0785] The analysis results include "User ID: 12345 frequently purchases food."

[0786] Step 11:

[0787] server

[0788] The server generates personalized advertisements and coupons based on the analysis results and sends them to the user's device. The input is the analysis results of purchasing trends, and the output is the generated advertisement and coupon data.

[0789] Specific operation:

[0790] Run ad generation algorithms to generate appropriate ads and coupons.

[0791] The generated advertisement is transmitted to the terminal.

[0792] Step 12:

[0793] Terminal

[0794] The device receives advertisements and coupon information sent from the server and displays them in a designated area within the app. The input is the advertisement data sent from the server, and the output is the advertisement displayed.

[0795] Specific operation:

[0796] The received advertising data is analyzed and displayed within the app.

[0797] The above are the processing steps of this system. The user, terminal, and server work together to carry out a series of processes to award points and provide personalized services.

[0798] (Application example 1)

[0799] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0800] While previous systems had the ability to use receipt information to award points and provide personalized advertisements and coupons, they lacked real-time support tools to help store associates improve their service. Furthermore, they lacked an immediate and intuitive interface for store associates to provide personalized service to customers. As a result, their effectiveness in improving customer satisfaction and maximizing store sales was limited.

[0801] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0802] In this invention, the server includes a means for acquiring a receipt image, an optical character recognition means for extracting text information from the acquired receipt image, and a means for classifying the extracted text information and assigning specific categories and tags. This enables a means for scanning receipt information and processing the data in real time. The server also includes a means installed in the smart glasses for providing display information when a store clerk provides customer service. This allows the store clerk to provide instant personalized service to customers.

[0803] definition statement

[0804] The "means for acquiring a receipt image" refers to a device such as a camera or scanner for electronically acquiring a receipt for a product purchased by a user.

[0805] An "optical character recognition means" is software or hardware for analyzing and extracting textual information from captured images.

[0806] The "means for assigning specific categories and tags" refers to a system that has the function of classifying extracted text information and automatically assigning appropriate categories and tags.

[0807] A "database storage means" is a data management device or system used to store classified information securely and efficiently.

[0808] The "means for adding points to a user account" is a system that automatically adds points to a user account based on the acquired and classified information.

[0809] "Artificial intelligence means" refers to machine learning models and algorithms used to analyze users' purchasing habits based on stored information.

[0810] The "means for generating personalized advertisements or coupons" is a system that generates advertisements and coupons optimized for each user based on analyzed purchasing data.

[0811] The "means for displaying on the user terminal" is an interface having a function for displaying the generated advertisements and coupons on the user terminal.

[0812] "Means installed on smart glasses to provide display information when store clerks provide customer service" refers to an application installed on smart glasses, which is a system that displays information necessary for store clerks to provide service to customers in real time.

[0813] "Means for scanning receipt information and processing data in real time" refers to hardware and software for quickly recognizing receipt information and immediately digitizing and processing that information.

[0814] MODE FOR CARRYING OUT THE INVENTION

[0815] The present invention provides an electronic payment system that acquires and analyzes receipt information, awards points to users, and provides personalized advertisements and coupons based on the points. The system includes smart glasses, a server, a database, an artificial intelligence model, and related communication means.

[0816] Capture and send receipt images

[0817] Terminal (smart glasses)

[0818] The store clerk puts on the smart glasses, launches the dedicated app, and scans the receipt for the item purchased by the user with the smart glasses' camera.

[0819] The scanned receipt image is temporarily stored in the smart glasses and the data is sent to the server.

[0820] Examples:

[0821] The store clerk presses the "scan" button and scans the receipt with the smart glasses' camera, after which the image data is automatically sent to the server.

[0822] Receipt data processing and analysis

[0823] server

[0824] The server receives the receipt image data sent from the terminal.

[0825] The received image data is processed using OCR (Optical Character Recognition) to extract text information, including the product name, price, purchase date, etc.

[0826] The extracted text information is classified into categories and tags.

[0827] Examples:

[0828] The server processes the receipt image and extracts the text data "Product A - 200 yen," "Product B - 300 yen," and "October 1, 2023." This is then automatically classified into categories such as "food" and "daily necessities."

[0829] Data storage and point allocation

[0830] server

[0831] The classified text information is stored in a database, including the user ID, purchased item, price, purchase date and time, etc.

[0832] After confirming that the receipt has been successfully processed and the data has been saved, points are credited to the user's account.

[0833] The results of point allocation are notified to the smart glasses.

[0834] Terminal (smart glasses)

[0835] The store clerk's smart glasses receive the point award notification from the server and display it as a message, allowing the store clerk to inform the customer that the user has earned points.

[0836] Examples:

[0837] The server awards 50 points to user ID "12345" and notifies the result to the smart glasses, which display the message "50 points have been awarded."

[0838] Data analytics and personalized advertising

[0839] server

[0840] The saved purchasing data is periodically input into the AI ​​model to analyze users' purchasing trends.

[0841] Based on the analysis results, optimal advertisements and coupons are generated for the user and sent to the smart glasses.

[0842] Terminal (smart glasses)

[0843] The smart glasses receive and display advertisements and coupon information sent from the server, and store clerks can use this information to suggest personalized products to customers.

[0844] Examples:

[0845] The server analyzes the user's purchase history and finds that they frequently purchase items in the "food" category, and generates advertisements for new food-related products. These advertisements are then sent to the smart glasses, and store clerks suggest "recommended foods" to the customer.

[0846] Enterprise Dashboard

[0847] server

[0848] We provide a dashboard for businesses that displays aggregated data and analytical results.

[0849] The dashboard visually displays, for example, sales trends over a specific period or user purchasing trends, allowing companies to gain insights for formulating effective sales strategies.

[0850] Examples:

[0851] The server analyzes the information, such as "Sales in the food category have increased by 25% in the past month," and displays it on a dashboard. Based on this information, the company makes a plan to further strengthen its product lineup.

[0852] Verifying User Receipt Information

[0853] server

[0854] Implement functionality to validate the accuracy of receipt information submitted by users, including checking that the information contained in the receipt matches an existing database.

[0855] Only if the verification is successful, points are awarded and the user is notified.

[0856] Examples:

[0857] The server verifies that the purchase date and time, store name, product name, etc. on the receipt match the records in the database, and then awards points.

[0858] Example prompt sentence:

[0859] Generate new product suggestions and coupons based on the purchase history of user ID "12345." Recent purchases include "Product A - 200 yen," "Product B - 300 yen," and "Purchase date - October 1, 2023."

[0860] The above is a concrete example of how to implement the present invention. This system allows users to not only earn points but also enjoy personalized services based on their purchasing habits. It also enables companies to effectively utilize purchasing data to strengthen their sales promotion activities.

[0861] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0862] Program processing steps

[0863] Step 1:

[0864] A store clerk wearing smart glasses launches a dedicated app and scans the receipt of the customer's purchase with the smart glasses' camera. The input is the scanned receipt image, and the output is image data temporarily stored in the smart glasses. Specifically, the store clerk presses the "scan" button and takes a photo of the receipt using the smart glasses' camera. The captured image is temporarily stored in the smart glasses' memory.

[0865] Step 2:

[0866] The smart glasses send the scanned receipt image to the server. The input is the temporarily stored receipt image data, and the output is the image data sent to the server. Specifically, the smart glasses' communication module is used to send the captured image to the server via a secure communication protocol.

[0867] Step 3:

[0868] The server applies OCR processing to the received receipt image data and extracts text information. The input is the receipt image data received by the server, and the output is the extracted text information. Specifically, the server uses OCR software to analyze the characters in the image and generate text data such as "Product A - 200 yen," "Product B - 300 yen," and "October 1, 2023."

[0869] Step 4:

[0870] The server classifies the extracted text information into categories and tags. The input is the extracted text information, and the output is data with categories and tags. Specifically, the server uses a rule-based or machine learning model to classify the information into categories such as "food" and "daily necessities" based on product name, price, purchase date, etc.

[0871] Step 5:

[0872] The server stores the classified text information in a database. The input is data with categories and tags, and the output is the information stored in the database. Specifically, the server stores information such as the user ID, purchased item, price, and purchase date and time in a relational database or cloud storage.

[0873] Step 6:

[0874] The server verifies the accuracy of the receipt information. The input is the information stored in the database, and the output is the result of the verification. Specifically, the server verifies that the receipt information is accurate by matching it with existing records in the database.

[0875] Step 7:

[0876] After the verification is successful, the server will grant points to the user account. The input is the successfully verified receipt information, and the output is the points granted to the user account. Specifically, the server calculates the points based on the user ID and updates the user's point balance.

[0877] Step 8:

[0878] The result of point allocation is notified to the smart glasses. The input is the point information allocated, and the output is a notification to the smart glasses. Specifically, the server sends the point allocation result to the smart glasses in real time, and the message "50 points have been allocated" is displayed on the store clerk's glasses.

[0879] Step 9:

[0880] The server inputs the stored purchasing data into an AI model to analyze the user's purchasing trends. The input is the stored purchasing data, and the output is the analysis results. Specifically, the server uses a machine learning model to analyze the user's purchasing patterns and predict future purchasing behavior.

[0881] Step 10:

[0882] Based on the analysis results, the server generates personalized advertisements and coupons for users. The input is the analysis results, and the output is the generated advertisements and coupons. Specifically, the server runs an algorithm to generate optimal advertisements and coupons for each user based on purchasing trends.

[0883] Step 11:

[0884] The generated advertisements and coupons are displayed on the smart glasses. The input is the generated advertisement or coupon information, and the output is the information displayed on the smart glasses. Specifically, the server sends the generated advertisement or coupon information to the smart glasses, and the store clerk provides services to the customer based on that information.

[0885] Example prompt sentence:

[0886] Generate new product suggestions and coupons based on the purchase history of user ID "12345." Recent purchases include "Product A - 200 yen," "Product B - 300 yen," and "Purchase date - October 1, 2023."

[0887] The above are the specific processing steps of the system program that realizes this application example. This system allows users to earn points and enjoy personalized services based on their purchasing habits. Store clerks can use the smart glasses to provide customers with personalized product suggestions and services in real time.

[0888] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0889] The present invention is a system that combines an electronic payment system that acquires and analyzes receipt information and awards points to users with an emotion engine that recognizes user emotions. This system includes a user terminal, a server, a database, an artificial intelligence model, the emotion engine, and related communication means.

[0890] Capture and send receipt images

[0891] Terminal (user terminal)

[0892] The user launches the app and selects the "Receipt Capture" or "Receipt Upload" function. The user either takes a photo of a paper receipt with the camera or selects an electronic receipt from within the device.

[0893] The terminal temporarily stores the photographed or uploaded receipt image and transmits the data to the server.

[0894] Examples:

[0895] The user presses the "take a photo" button to take a photo of the receipt with the smartphone camera, then presses the "send" button to send the image data to the server.

[0896] Receipt data processing and analysis

[0897] server

[0898] The server receives the receipt image data sent from the terminal.

[0899] The received image data is processed using OCR (Optical Character Recognition) to extract text information (product name, price, purchase date, etc.) using, for example, Tesseract or Google Cloud Vision API.

[0900] The extracted text information is classified into categories and tags.

[0901] Examples:

[0902] The server processes the receipt image and extracts the text data "Product A - 200 yen," "Product B - 300 yen," and "October 1, 2023." This is then automatically classified into categories such as "food" and "daily necessities."

[0903] Data storage and point allocation

[0904] server

[0905] The classified text information is stored in a database, including the user ID, purchased item, price, purchase date and time, etc.

[0906] After confirming that the receipt has been successfully processed and the data has been saved, points are credited to the user's account.

[0907] The result of the points allocation is notified to the user terminal.

[0908] Terminal

[0909] The user's device receives the point allocation notification from the server and displays the notification within the app, allowing the user to confirm that they have earned points.

[0910] Examples:

[0911] The server awards 50 points to user ID "12345" and notifies the terminal of the result. The terminal displays the message "50 points have been awarded."

[0912] Data analytics and personalized advertising

[0913] server

[0914] The saved purchase data is periodically input into the AI ​​model to analyze users' purchasing trends. The analysis uses a combination of various data such as user attributes and past purchase history.

[0915] Based on the analysis results, optimal advertisements and coupons are generated for the user and sent to the terminal.

[0916] Terminal

[0917] The device receives advertisements and coupon information sent from the server and displays them in a designated area within the app, allowing users to view offers based on their purchasing habits.

[0918] Examples:

[0919] The server analyzes the user's purchase history and finds that they frequently purchase items in the "food" category, generates new food-related product advertisements, sends them to the device, and displays "recommended foods" in the user's app.

[0920] Recognizing and utilizing user emotions

[0921] server

[0922] The emotion engine is incorporated to recognize user emotions. The emotion engine identifies emotions by analyzing, for example, feedback and product ratings entered by users within the app.

[0923] The recognized emotion data is integrated into the purchasing tendency analysis, specifically, to reflect whether the user is satisfied or dissatisfied with a particular product.

[0924] Terminal

[0925] When users enter feedback or ratings, the data is sent to the server, where the emotion engine analyzes it and generates emotion data.

[0926] Examples:

[0927] When a user enters feedback in the app such as "I am satisfied with this product," the emotion engine evaluates it as "satisfied" and adds it to the purchase data.

[0928] Personalized advertising based on emotional data

[0929] server

[0930] Generate more personalized ads and coupons based on user sentiment data, and improve the user experience by prioritizing ads related to products and services that generate high satisfaction.

[0931] Terminal

[0932] Receive personalized advertisements and coupon information based on emotional data and display them in designated areas within the app.

[0933] Examples:

[0934] The server generates related advertisements for products rated as "Satisfied" and displays them in the user's application. For example, a "10% off coupon for new products related to the product you were satisfied with" is displayed.

[0935] Enterprise Dashboard

[0936] server

[0937] It provides a dashboard for businesses to display aggregated data and analytical results, visually displaying insights based on sales trends, user purchasing habits, and sentiment data for a specific period.

[0938] By utilizing emotional data, companies can use it as a reference for more effective marketing strategies and product development.

[0939] Examples:

[0940] The dashboard displays information such as "Sales in the food category have increased by 25% in the past month" along with user satisfaction ratings for the relevant products, allowing companies to plan product improvements and new product development.

[0941] Verifying User Receipt Information

[0942] server

[0943] Validate the accuracy of the receipt information provided by the user. This involves checking whether the information contained in the receipt matches an existing database. Only if the validation is successful are points awarded and the user notified.

[0944] Examples:

[0945] The server verifies that the purchase date and time, store name, product name, etc. on the receipt match the records in the database, and then awards points.

[0946] The above is a specific embodiment for implementing the present invention. This system allows users to earn points through receipts and receive personalized advertisements and coupons, and further improves the user experience by utilizing emotional data. Furthermore, companies can optimize their sales promotion activities based on detailed purchasing data and emotional data.

[0947] The processing flow will be explained below.

[0948] Step 1:

[0949] The user launches the app and selects the "Receipt Capture" or "Receipt Upload" function. The user either takes a photo of a paper receipt with the camera or selects an electronic receipt from within the device. When the user presses the "Send" button, the receipt image data is temporarily saved within the device.

[0950] Step 2:

[0951] The device prepares the captured or uploaded receipt image for transmission to the server, specifically by encoding the image data into the appropriate format and attaching metadata (user ID, timestamp, etc.).

[0952] Step 3:

[0953] The device sends the prepared data to the server, which usually uses the HTTPS protocol to ensure secure communication.

[0954] Step 4:

[0955] The server receives the receipt image data sent from the terminal, temporarily stores the received raw data, and prepares it for OCR processing.

[0956] Step 5:

[0957] The server performs OCR (optical character recognition) processing to extract text information (product name, price, purchase date, etc.) from the receipt image. The OCR engine used can be, for example, Tesseract or Google Cloud Vision API.

[0958] Step 6:

[0959] The server analyzes the extracted text information and assigns categories (e.g., food, daily necessities, etc.) and tags (e.g., sale, double points, etc.) using a pre-trained classification model.

[0960] Step 7:

[0961] The server stores the classified text information in a database, including the user ID, purchased item, price, purchase date and time, and category / tag.

[0962] Step 8:

[0963] The server verifies that the receipt has been processed successfully and then grants the points to the user's account by executing an SQL query using the user ID as the key to add the points.

[0964] Step 9:

[0965] The server generates a response to notify the user of the point allocation result. This is usually a JSON formatted message.

[0966] Step 10:

[0967] The device receives the response from the server and displays a notification to the user that points have been awarded. Specifically, the device uses the app's notification function to display a message such as "50 points have been awarded."

[0968] Step 11:

[0969] The server periodically supplies the stored purchase data to the AI ​​model to analyze users' purchasing trends. The analysis uses a combination of various data, such as user attributes and past purchase history.

[0970] Step 12:

[0971] The server generates personalized ads and coupons based on the results of the AI ​​model, which are tailored to the user's purchasing habits.

[0972] Step 13:

[0973] The server then sends the generated advertisements and coupons to the user's device, again using a secure communication protocol.

[0974] Step 14:

[0975] The device receives advertisements and coupon information sent from the server and displays them in a designated area within the app, allowing users to view offers based on their purchasing habits.

[0976] Step 15:

[0977] The server runs an emotion engine to recognize the user's emotions and analyzes the feedback and product ratings that the user writes in the app, generating analyzed emotion data.

[0978] Step 16:

[0979] The device sends the feedback and evaluation data entered by the user to the server, where the emotion engine analyzes it and generates emotion data.

[0980] Step 17:

[0981] The server integrates the recognized emotion data into a purchasing tendency analysis, specifically, reflecting whether the user is satisfied or dissatisfied with a particular product.

[0982] Step 18:

[0983] The server generates more personalized advertisements and coupons based on the user's emotional data, improving the user experience by prioritizing advertisements related to products and services that generate high satisfaction.

[0984] Step 19:

[0985] The device receives personalized advertisements and coupons based on emotion data and displays them in a designated area within the app, allowing users to view emotion-based offers.

[0986] Step 20:

[0987] The server provides a dashboard for businesses to display aggregate data and sentiment data, visually displaying sales trends over a specific period, user purchasing habits, and insights based on sentiment data.

[0988] Step 21:

[0989] The server verifies the accuracy of the receipt information provided by the user. Validation involves checking whether the information contained in the receipt matches an existing database record. Only if the validation is successful are points awarded and the user notified.

[0990] Specific examples

[0991] For example, if a user enters feedback within the app such as "I was satisfied with this product," the emotion engine will rate it as "satisfied" and add it to the purchase data. The server will generate related advertisements for products rated as "satisfied" and display them in the user's app. A dashboard for businesses will display information such as "Sales in the food category have increased by 25% in the past month, and user satisfaction with the relevant products is high," allowing businesses to make plans for product improvements and new product development.

[0992] The above are the processing steps and specific operations of the system's program. This allows users to earn points through receipts and receive personalized advertisements and coupons, and also provides an improved user experience by utilizing emotional data. Furthermore, companies can optimize their sales promotion activities based on detailed purchasing data and emotional data.

[0993] Example 2

[0994] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0995] Conventional electronic payment systems provide functions for obtaining receipt information and awarding points, but they lack the ability to provide personalized advertising using user emotional data or to analyze detailed purchasing trends. Furthermore, they lack a means to provide businesses with analysis results that integrate purchasing data and emotional data. This has made it difficult to improve user experience and optimize corporate marketing strategies.

[0996] The identification process performed by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for acquiring a receipt image, an optical character recognition means for extracting text information from the acquired receipt image, a means for categorizing the extracted text information and assigning specific categories and tags, a means for saving the classified information in a database, a means for awarding points to a user account based on the saved information, an artificial intelligence means for analyzing a user's purchasing habits based on the saved information, a means for generating personalized advertisements or coupons for the user based on the analysis results, a means for displaying the advertisements or coupons on the user terminal, an emotion recognition means for recognizing the user's emotional data and further analyzing the purchasing habits based on the emotional data, a means for generating personalized advertisements or coupons based on the emotional data, and a means for providing a dashboard displaying aggregated data and analysis results to businesses. This allows users to earn points through receipts and receive personalized advertisements that utilize their emotional data. Businesses can also receive analysis results based on detailed purchasing data and emotional data via the dashboard, optimizing their sales promotion activities.

[0997] "Means for acquiring receipt images" refers to devices or software that have the function of electronically acquiring receipt images by a user taking a photo of a paper receipt with a camera or selecting an electronic receipt from within a terminal.

[0998] "Optical character recognition means" refers to technology or devices for extracting text information from an acquired receipt image, and specifically refers to devices that use OCR (optical character recognition) technology to read character information.

[0999] "Means for classifying extracted text information and assigning specific categories and tags" refers to software or algorithms for classifying text information obtained by OCR processing into specific categories (e.g., food, daily necessities, etc.) and tags (e.g., price, product name, etc.).

[1000] "Means for storing classified information in a database" refers to a device or system that stores the classified information extracted from receipts in a database for future reference and analysis.

[1001] "Means for awarding points to a user account based on stored information" means a system or algorithm for calculating and awarding points to a user's account based on stored information in a database.

[1002] "Artificial intelligence means" refers to machine learning algorithms and AI models that use stored receipt information and other data to analyze users' purchasing habits.

[1003] "Means for generating personalized advertisements or coupons" refers to systems or software for generating advertisements or coupons that are optimal for individual users based on the user's purchasing habits and emotional data.

[1004] "Means for displaying advertisements or coupons on a user terminal" refers to an interface or software for displaying the generated advertisements or coupons on a user's smartphone or other terminal.

[1005] "Emotion recognition means" refers to a device or software that analyzes user feedback and evaluation data to identify user emotions and reflect the results in purchasing trend analysis.

[1006] "A means of providing dashboards that display aggregated data and analytical results for businesses" refers to visual interfaces and software that allow businesses to grasp their sales trends, user purchasing habits, and sentiment data at a glance.

[1007] The present invention is a system that combines an electronic payment system that acquires and analyzes receipt information and awards points to users with an emotion engine that recognizes user emotions. This system includes a user terminal, a server, a database, an artificial intelligence model, the emotion engine, and related communication means.

[1008] Capture and send receipt images

[1009] Terminal

[1010] The user launches the smartphone app and selects the "Receipt Capture" or "Receipt Upload" function. The user either takes a photo of a paper receipt with the camera or selects an electronic receipt from within the device. The device temporarily stores the captured or uploaded receipt image and sends the data to the server.

[1011] Examples:

[1012] The user presses the "take a photo" button, takes a photo of the receipt with the smartphone camera, and then presses the "send" button to send the image data to the server.

[1013] Example prompt for generative AI model:

[1014] "Please explain the steps to take a picture of the receipt and send it to the server."

[1015] Receipt data processing and analysis

[1016] server

[1017] The server receives the receipt image data sent from the device. It performs OCR processing on the received image data using Tesseract or Google Cloud Vision API to extract text information (product name, price, purchase date and time, etc.) and classifies the extracted text information into categories and tags.

[1018] Examples:

[1019] The server processes the receipt image and extracts the text data "Product A - 200 yen," "Product B - 300 yen," and "October 1, 2023," and then classifies this into categories such as "food" and "daily necessities."

[1020] Example prompt for generative AI model:

[1021] "Please explain the steps to extract text information from receipt images and classify it into categories."

[1022] Data storage and point allocation

[1023] server

[1024] The classified text information is stored in a database. The stored information includes the user ID, purchased item, price, purchase date and time, etc. After confirming that the receipt has been processed correctly and the data has been saved, points are awarded to the user's account. The result of point awarding is notified to the user's device.

[1025] Terminal

[1026] The user device receives the point allocation notification from the server and displays the notification within the app.

[1027] Examples:

[1028] The server awards 50 points to user ID "12345" and notifies the terminal of the result. The terminal displays the message "50 points have been awarded."

[1029] Example prompt for generative AI model:

[1030] Please explain the process from when the user submits the receipt until points are awarded.

[1031] Data analytics and personalized advertising

[1032] server

[1033] The stored purchase data is periodically input into an AI model to analyze the user's purchasing trends. This analysis uses user attributes and past purchase history. Based on the analysis results, optimal advertisements and coupons are generated for the user and sent to the device.

[1034] Terminal

[1035] The device receives advertisements and coupon information sent from the server and displays it within the app.

[1036] Examples:

[1037] The server uses an AI model to analyze past purchase data, generate advertisements for new food-related products for users who frequently purchase products in the "food" category, and send them to the user's device. The user's device then displays "recommended food products" within the app.

[1038] Example prompt for generative AI model:

[1039] "Please explain the steps to analyze user purchasing data and generate optimal advertisements."

[1040] Recognizing and utilizing user emotions

[1041] server

[1042] Incorporate an emotion engine to recognize user emotions. Analyze feedback and product ratings written by users within the app to identify emotions. Integrate the recognized emotion data into purchasing tendency analysis.

[1043] Terminal

[1044] When users enter feedback or ratings, the data is sent to the server and analyzed by the emotion engine.

[1045] Examples:

[1046] The user enters feedback in the app, such as "I'm satisfied with this product," and the data is sent to the server. The emotion engine evaluates the feedback as "satisfied," and the result is reflected in the purchase data.

[1047] Example prompt for generative AI model:

[1048] "Please explain the steps to recognize user sentiment data and reflect it in purchasing data."

[1049] Personalized advertising based on emotional data

[1050] server

[1051] Generate personalized ads and coupons based on user sentiment data, and prioritize ads related to products and services that generate high satisfaction.

[1052] Terminal

[1053] Receive personalized ads and coupon information based on emotional data and display it within the app.

[1054] Examples:

[1055] The server will prioritize generating related advertisements for products that have been rated as "satisfactory." For example, a "10% off coupon for a new product related to the product you were satisfied with" will be sent to the device and displayed within the app.

[1056] Example prompt for generative AI model:

[1057] "Explain the steps to generate and display personalized ads based on emotional data."

[1058] Enterprise Dashboard

[1059] server

[1060] It provides a dashboard for businesses to display aggregated data and analytical results, visually displaying insights based on sales trends, user purchasing habits, and sentiment data.

[1061] Examples:

[1062] The dashboard displays information such as "Sales in the food category have increased by 25% in the past month" along with user satisfaction ratings for the relevant products. Companies can use this information to consider marketing strategies and product development policies.

[1063] Example prompt for generative AI model:

[1064] "Describe the data you want to display in your enterprise dashboard."

[1065] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1066] Step 1: Capture and send a receipt image

[1067] 1-1. The user launches the app on their smartphone and selects the "take a photo of a receipt" or "upload a receipt" function.

[1068] 1-2. The user takes a photo of a paper receipt with the camera or selects an electronic receipt from within the device. The input data is an image file of the paper or electronic receipt.

[1069] 1-3. The device temporarily saves the captured or uploaded receipt image and sends the data to the server. The output data is the receipt image file sent to the server.

[1070] Specific behavior:

[1071] The user presses the "take a photo" button to take a photo of the receipt with the smartphone camera, then presses the "send" button, and the device transfers the image data to the server.

[1072] Step 2: Process and parse receipt data

[1073] 2-1. The server receives the receipt image data sent from the terminal. The input data is the receipt image file sent from the terminal.

[1074] 2-2. The server uses Tesseract or Google Cloud Vision API to perform OCR processing on the image data received and extracts text information (product name, price, purchase date, etc.). OCR processing is performed as data processing, and the output data is the extracted text information.

[1075] 2-3. Classify the extracted text information into categories and tags. Data classification is performed, and the output data is text information with categories or tags assigned.

[1076] Specific behavior:

[1077] After the server obtains the receipt image data, it calls the Tesseract OCR engine to perform character recognition, generating text data such as "Product A - 200 yen," "Product B - 300 yen," and "October 1, 2023," which is then automatically classified into categories such as "food" and "daily necessities."

[1078] Step 3: Save your data and earn points

[1079] 3-1. The server saves the classified text information in a database. The input data is text information with categories. The data is saved, and the output data is the information stored in the database.

[1080] 3-2. After confirming that the receipt has been successfully processed and the data has been saved, the server will award points to the user account. The input data is the saved data. The awarding process is executed, and the output data is the awarded points.

[1081] 3-3. The server notifies the user terminal of the point allocation result. The input data is the point allocation result. The notification is made, and the output data is the notification information sent to the user terminal.

[1082] Specific behavior:

[1083] The server saves the information "User ID: 12345, Product: Product A, Price: 200 yen, Purchase date: October 1, 2023" in the database, triggers the point allocation process, adds 50 points to user ID "12345", and sends the result to the terminal. The user terminal displays the message "50 points have been allocated."

[1084] Step 4: Data analysis and personalized advertising

[1085] 4-1. The server inputs the stored purchasing data into the AI ​​model to analyze the user's purchasing trends. The input data is the stored purchasing data. Data analysis is performed, and the output data is the analysis results.

[1086] 4-2. Based on the analysis results, the server generates the optimal advertisement or coupon for the user and sends it to the terminal. The input data is the analysis results. The generation process is carried out, and the output data is the generated advertisement or coupon.

[1087] 4-3. The device receives the advertisement or coupon information sent from the server and displays it within the app. The input data is the advertisement or coupon information sent from the server. The display process is performed, and the output data is the advertisement or coupon displayed within the app.

[1088] Specific behavior:

[1089] The server uses an AI model to analyze past purchase data, generate advertisements for new food-related products for users who frequently purchase products in the "food" category, and send them to the user's device. The user's device then displays "recommended food products" within the app.

[1090] Step 5: Recognize and leverage user emotions

[1091] 5-1. The server incorporates an emotion engine to recognize the user's emotions. The input data is the feedback and product ratings entered by the user within the app. An analysis process is performed, and the output data is the recognized emotion data.

[1092] 5-2. The server integrates the recognized emotion data into the purchasing tendency analysis. The input data is emotion data. The integration process is carried out, and the output data is the integrated analysis data.

[1093] 5-3. When a user inputs feedback or evaluation, the data is sent to the server and analyzed by the emotion engine. The input data is the feedback or evaluation data. The analysis process is performed, and the output data is the analyzed emotion data.

[1094] Specific behavior:

[1095] The user enters feedback in the app, such as "I'm satisfied with this product," and the data is sent to the server. The emotion engine evaluates the feedback as "satisfied" and integrates the evaluation into the purchase data.

[1096] Step 6: Personalized advertising based on emotional data

[1097] 6-1. The server generates personalized advertisements and coupons based on the user's emotional data. The input data is the emotional data. The generation process is performed, and the output data is personalized advertisements and coupons.

[1098] 6-2. The server sends advertisements and coupons to the user terminal. The input data is the personalized advertisements and coupons. The transmission process is carried out, and the output data is the advertisement and coupon data sent to the user terminal.

[1099] 6-3. The device receives advertisements or coupon information based on the emotion data and displays it within the app. The input data is advertisements or coupon information based on the emotion data. The display process is performed, and the output data is the advertisement or coupon displayed within the app.

[1100] Specific behavior:

[1101] The server generates related advertisements for products rated as "satisfactory" and sends them to the user's device. For example, a "10% off coupon for a new product related to the product you were satisfied with" is displayed in the app.

[1102] Step 7: Enterprise dashboard

[1103] 7-1. The server provides a dashboard for companies to display aggregated data and analysis results. The input data is the aggregated data and analysis results. The display process is carried out, and the output data is the information displayed on the dashboard.

[1104] Specific behavior:

[1105] The dashboard displays information such as "Sales in the food category have increased by 25% in the past month" along with user satisfaction ratings for the relevant products. Companies can use this information to consider the direction of their marketing strategies and product development.

[1106] (Application example 2)

[1107] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1108] Current electronic payment systems are limited to collecting receipt information and awarding points, and lack a deep understanding of users' purchasing behavior and emotions. Furthermore, current systems lack the functionality to provide personalized advertising or coupons that take user emotions into account, preventing improvements to the user experience. Furthermore, the data provided to businesses is limited and insufficient for use in marketing and product development.

[1109] The identification process performed by the identification processing unit 290 of the data processing device 12 in Application Example 2 is implemented by the following means. In this invention, the server includes: means for acquiring receipt images; optical character recognition means for extracting text information from the acquired receipt images; means for categorizing the extracted text information and assigning specific categories and tags; means for saving the classified information in a database; means for awarding points to a user account based on the saved information; artificial intelligence means for analyzing users' purchasing trends based on the saved information; means including an emotion engine for recognizing and integrating user emotions into the analysis; means for generating personalized advertisements or coupons for users based on the analysis results; and means for displaying the advertisements or coupons on the user terminal. This enables analysis of user purchasing behavior and emotions based on receipt information, thereby improving the user experience by providing more accurate personalized advertisements and coupons. Furthermore, providing a wide range of data analysis results through a corporate dashboard can improve the quality of marketing strategies and product development.

[1110] The "receipt image acquisition means" is a means for a user to take or upload a receipt image.

[1111] The "optical character recognition means" is a means for extracting text information from the captured receipt image.

[1112] The "text information classification means" is a means for classifying extracted text information and assigning specific categories and tags to it.

[1113] "Database storage means" refers to a means for safely storing classified information in a database.

[1114] The "points granting means" is a means for granting points to a user account based on the stored information.

[1115] "Artificial intelligence means" refers to means for analyzing users' purchasing trends based on stored information.

[1116] An "emotion engine" is a means for recognizing and integrating user emotions into analysis.

[1117] The "personalized advertisement generating means" is a means for generating advertisements or coupons individually tailored for users based on the analysis results.

[1118] The "advertisement display means" is a means for displaying the generated advertisement or coupon on the user terminal.

[1119] This invention combines an emotion engine with an electronic payment system that acquires and analyzes receipt information and awards points to users. This system is realized by a program consisting of the following elements:

[1120] First, the user terminal provides a means for acquiring receipt images. The user launches the application and sends the receipt image to the system by taking a photo or uploading it. This process uses the smartphone's camera or image upload function.

[1121] The server then extracts text information from the captured receipt image using optical character recognition (OCR) software such as OpenCV or Tesseract. The extracted text information includes data points such as product name, price, and purchase date and time.

[1122] The extracted text information is then classified and assigned categories and tags. The server stores the classified information in a database, including the user ID, purchased item, price, purchase date and time, etc.

[1123] Based on the saved information, the server will assign points to the user's account. The result of the points assignment will be notified to the user's device, and a message stating "Points have been assigned" will be displayed within the app.

[1124] Furthermore, the server uses artificial intelligence to analyze the stored purchase data and understand the user's purchasing trends. This analysis integrates data such as past purchase history and user attributes. This process uses a generative AI model.

[1125] The emotion engine recognizes user emotions and integrates them into purchasing trend analysis. For example, when a user enters product ratings or feedback within the app, that data is sent to the emotion engine, and the system generates emotion data such as "satisfied" or "dissatisfied."

[1126] Based on this emotional data, a means is provided for generating more personalized advertisements and coupons. The server creates optimal advertisements and coupons based on the user's purchasing tendencies and emotional data, and sends them to the user's device. The user's device then displays these advertisements and coupons in a designated area within the app.

[1127] As a specific example, if a user purchases "milk" and rates it as "very satisfied," the system will generate a 10% off coupon for the next purchase related to "milk" and provide it to the user.

[1128] Additionally, a dashboard for businesses is provided to visually display sales trends and purchasing habits, including insights based on sentiment data, which can be used by businesses to develop more effective marketing strategies and product development.

[1129] Finally, a means is provided for verifying the accuracy of the receipt information provided by the user. The server compares the receipt information with an existing database, confirming accuracy, and then awards points. In this way, the system improves the user experience and provides valuable data for businesses.

[1130] (Example of a prompt)

[1131] Please enter your thoughts about the product you purchased, "Milk." Example: "I am very satisfied with this milk."

[1132] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1133] Step 1: Get the receipt image

[1134] The user launches the smartphone app and selects the "take a photo of a receipt" or "upload a receipt" function. The user either takes a photo of a paper receipt with the camera or selects an electronic receipt from within the device. The device temporarily saves the captured or uploaded receipt image. The input is the receipt image selected by the user, and the temporarily saved image data is generated as the output.

[1135] Step 2: Send a receipt image

[1136] The user device sends the temporarily saved receipt image to the server. Here, the receipt image data is sent as input to the server and a success response is received. Specifically, the image data is sent to a specified endpoint on the server using the HTTP protocol.

[1137] Step 3: Optical Character Recognition (OCR)

[1138] The server applies OCR processing to the receipt image data it receives. The input is the receipt image data, and the output is the extracted text information. Specifically, it uses Tesseract and the Google Cloud Vision API to extract the text information.

[1139] Step 4: Classifying text information

[1140] The server classifies the extracted text information into categories and tags. The input is the text information resulting from the OCR process, and the output is the categorized and tagged text information. Here, classification is performed using a predefined rule-based or machine learning algorithm.

[1141] Step 5: Saving to the Database

[1142] The server stores the categorized text information in a database. The input is the categorized and tagged text information, and the output is a record stored in the database. Specifically, the database operation involves information such as the user ID, purchased item, price, and purchase date and time.

[1143] Step 6: Points awarded

[1144] The server will then assign points to the user's account based on the stored information. The input is the purchase data in the database, and the output is the updated user's points information. Specifically, points are calculated according to existing business rules, and the user's points balance is updated.

[1145] Step 7: Notification of points awarded

[1146] The server notifies the user device of the point awarding results. The input is the updated user point information, and the output is a notification message. Specifically, a push notification or in-app notification is used to send a message saying "Points have been awarded."

[1147] Step 8: Analyze purchasing data

[1148] The server periodically inputs the stored purchasing data into the AI ​​model to analyze the user's purchasing trends. The input is the purchasing data in the database, and the output is the analysis results of purchasing trends. The generative AI model combines past purchasing history and user attributes to analyze purchasing trends.

[1149] Step 9: Recognizing Emotional Data

[1150] The server uses an emotion engine to recognize the user's emotions. The input is the user's feedback and evaluation data, and the output is the recognized emotion data. Specifically, the emotion engine analyzes the user's input data and extracts emotions such as "satisfaction" or "dissatisfaction."

[1151] Step 10: Generate personalized ads

[1152] The server generates personalized advertisements and coupons based on purchasing trend analysis and emotional data. The input is the purchasing trend analysis results and emotional data, and the output is individually tailored advertisements and coupon information. Here, optimal advertisements are generated based on the user's interests and satisfaction.

[1153] Step 11: Displaying Ads

[1154] The user device receives the advertisements and coupon information sent from the server and displays them in a designated area within the app. The input is personalized advertisements and coupon information, and the output is the advertisements and coupons displayed on the user's device screen.

[1155] Finally, as a concrete example of a prompt:

[1156] "Please enter your thoughts about the product you purchased, 'Milk'. Example: 'I am very satisfied with this milk.'"

[1157] is used.

[1158] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1159] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1160] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1161] [Third embodiment]

[1162] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1163] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1164] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1165] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1166] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1167] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1168] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1169] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1170] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1171] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1172] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1173] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1174] The present invention provides an electronic payment system that acquires and analyzes receipt information and awards points to users, and a system that provides personalized advertisements and coupons based on the points. This system includes a user terminal, a server, a database, an artificial intelligence model, and related communication means.

[1175] Capture and send receipt images

[1176] Terminal (user terminal)

[1177] When a user launches the app, they are presented with the option to take a photo or upload a receipt.

[1178] The user either takes a photo of a paper receipt with the camera or selects an electronic receipt from within the device.

[1179] The terminal temporarily stores the photographed or uploaded receipt image and transmits the data to the server.

[1180] Examples:

[1181] The user presses the "take a photo" button to take a photo of the receipt with the smartphone camera, then presses the "send" button to send the image data to the server.

[1182] Receipt data processing and analysis

[1183] server

[1184] The server receives the receipt image data sent from the terminal.

[1185] The received image data is processed using OCR (Optical Character Recognition) to extract text information, including the product name, price, purchase date, etc.

[1186] The extracted text information is classified into categories and tags.

[1187] Examples:

[1188] The server processes the receipt image and extracts the text data "Product A - 200 yen," "Product B - 300 yen," and "October 1, 2023." This is then automatically classified into categories such as "food" and "daily necessities."

[1189] Data storage and point allocation

[1190] server

[1191] The classified text information is stored in a database, including the user ID, purchased item, price, purchase date and time, etc.

[1192] After confirming that the receipt has been successfully processed and the data has been saved, points are credited to the user's account.

[1193] The result of the points allocation is notified to the user terminal.

[1194] Terminal

[1195] The user's device receives the point allocation notification from the server and displays the notification within the app, allowing the user to confirm that they have earned points.

[1196] Examples:

[1197] The server awards 50 points to user ID "12345" and notifies the terminal of the result. The terminal displays the message "50 points have been awarded."

[1198] Data analytics and personalized advertising

[1199] server

[1200] The saved purchasing data is periodically input into the AI ​​model to analyze users' purchasing trends.

[1201] Based on the analysis results, optimal advertisements and coupons are generated for the user and sent to the terminal.

[1202] Terminal

[1203] The device receives advertisements and coupons sent from the server and displays them in a designated area within the app, allowing users to view offers based on their purchasing habits.

[1204] Examples:

[1205] The server analyzes the user's purchase history and finds that they frequently purchase items in the "food" category, generates new food-related product advertisements, sends them to the device, and displays "recommended foods" in the user's app.

[1206] Enterprise Dashboard

[1207] server

[1208] We provide a dashboard for businesses that displays aggregated data and analytical results.

[1209] The dashboard visually displays, for example, sales trends over a specific period or user purchasing trends, allowing companies to gain insights for formulating effective sales strategies.

[1210] Examples:

[1211] The server analyzes the information, such as "Sales in the food category have increased by 25% in the past month," and displays it on a dashboard. Based on this information, the company makes a plan to further strengthen its product lineup.

[1212] Verifying User Receipt Information

[1213] server

[1214] Implement functionality to validate the accuracy of receipt information submitted by users, including checking that the information contained in the receipt matches an existing database.

[1215] Only if the verification is successful, points are awarded and the user is notified.

[1216] Examples:

[1217] The server verifies that the purchase date and time, store name, product name, etc. on the receipt match the records in the database, and then awards points.

[1218] The above is a concrete example of how to implement the present invention. This system allows users to not only earn points but also enjoy personalized services based on their purchasing habits. It also enables companies to effectively utilize purchasing data to strengthen their sales promotion activities.

[1219] The processing flow will be explained below.

[1220] Step 1:

[1221] The user launches the app and selects the "Receipt Capture" or "Receipt Upload" function. The user either takes a photo of a paper receipt with the camera or selects an electronic receipt from within the device. When the user presses the "Send" button, the receipt image data is temporarily saved within the device.

[1222] Step 2:

[1223] The device prepares the captured or uploaded receipt image for transmission to the server, specifically by encoding the image data into the appropriate format and attaching metadata (user ID, timestamp, etc.).

[1224] Step 3:

[1225] The device then sends the prepared data to the server, which typically uses the HTTPS protocol to ensure secure communication.

[1226] Step 4:

[1227] The server receives the receipt image data sent from the terminal, temporarily stores the received raw data, and prepares it for OCR processing.

[1228] Step 5:

[1229] The server performs OCR (optical character recognition) processing to extract text information (product name, price, purchase date, etc.) from the receipt image. The OCR engine used can be, for example, Tesseract or Google Cloud Vision API.

[1230] Step 6:

[1231] The server analyzes the extracted text information and assigns categories (e.g., food, daily necessities, etc.) and tags (e.g., sale, double points, etc.) using a pre-trained classification model.

[1232] Step 7:

[1233] The server stores the classified text information in a database, including the user ID, purchased item, price, purchase date and time, and category / tag.

[1234] Step 8:

[1235] The server verifies that the receipt has been processed successfully and then grants the points to the user's account by executing an SQL query using the user ID as the key to add the points.

[1236] Step 9:

[1237] The server generates a response to notify the user of the point allocation result. This is usually a JSON formatted message.

[1238] Step 10:

[1239] The device receives the response from the server and displays a notification to the user that points have been awarded. Specifically, the device uses the app's notification function to display a message such as "50 points have been awarded."

[1240] Step 11:

[1241] The server periodically supplies the stored purchase data to the AI ​​model to analyze users' purchasing trends. The analysis uses a combination of various data, such as user attributes and past purchase history.

[1242] Step 12:

[1243] The server generates personalized ads and coupons based on the results of the AI ​​model, which are tailored to the user's purchasing habits.

[1244] Step 13:

[1245] The server then sends the generated advertisements and coupons to the user's device, again using a secure communication protocol.

[1246] Step 14:

[1247] The device receives advertisements and coupon information sent from the server and displays them in a designated area within the app, allowing users to view offers based on their purchasing habits.

[1248] Step 15:

[1249] The server provides a dashboard for companies to display aggregated data and analysis results, visually displaying sales trends and user purchasing trends for a specific period.

[1250] Step 16:

[1251] The server verifies the accuracy of the receipt information provided by the user. This involves checking that the receipt contents match existing database records. Only if the verification is successful are points awarded and the user notified.

[1252] These are the processing steps and specific operations of the system program. This allows users to earn points through receipts and receive personalized advertisements and coupons. It also enables companies to optimize their sales promotion activities based on detailed purchase data.

[1253] Example 1

[1254] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1255] The present invention relates to a system that effectively acquires and analyzes receipt information and awards points to users. Conventional systems lack accurate extraction of receipt information and subsequent analysis, making it difficult to award points to users or provide personalized services. Furthermore, they lack effective data analysis and purchasing trend information for businesses. As a result, these systems fall short in improving user experience and promoting corporate sales.

[1256] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1257] In this invention, the server includes a means for acquiring receipt images, an optical character recognition means for extracting text information from the acquired receipt images, a means for classifying the extracted text information using natural language processing and assigning specific categories and tags, a means for saving the classified information in a database, a means for awarding points to a user account based on the saved information, a means for notifying the user terminal of the point awarding results, an artificial intelligence means for analyzing the user's purchasing habits based on the saved information, a means for generating personalized advertisements or coupons for the user based on the analysis results, and a means for displaying the advertisements or coupons on the user terminal. This allows users to easily acquire and analyze receipt information and earn points, as well as enjoy personalized services based on their purchasing habits. It also enables companies to effectively utilize purchasing data to strengthen their sales promotion activities.

[1258] A "user terminal" refers to an electronic device operated by a user, and includes smartphones, tablets, laptops, and the like.

[1259] "Receipt image" refers to image data of a receipt photographed or uploaded by a user.

[1260] A "server" refers to a centralized computer system that receives data sent from user terminals over a network and performs subsequent processing.

[1261] "Optical Character Recognition (OCR) Method" means a technology for extracting text information from receipt images, such as Tesseract, Google Cloud Vision, or similar tools.

[1262] "Natural language processing means" refers to processes that use algorithms or libraries, such as NLTK or spaCy, to classify extracted text information into specific categories or tags.

[1263] "Database" refers to a software system for systematically storing and managing classified information, including relational databases such as MySQL and PostgreSQL.

[1264] "Artificial intelligence means" refers to machine learning models and algorithms used to analyze user purchasing trends based on stored data, and uses frameworks such as TensorFlow and PyTorch.

[1265] "Personalized ads or coupons" refers to ads or coupons that are individually generated based on a user's purchasing habits.

[1266] "Verification Procedure" refers to the process of verifying the accuracy of receipt information submitted by a user and ensuring it matches existing data in the database.

[1267] A "dashboard" is an interface for visually displaying aggregated data and analytical results for companies, providing data in the form of graphs, charts, etc.

[1268] The present invention provides an electronic payment system that acquires and analyzes receipt information and awards points to users, and a system that provides personalized advertisements and coupons based on the points. This system includes a user terminal, a server, a database, an artificial intelligence model, and related communication means.

[1269] Capture and send receipt images

[1270] When a user launches the app, a receipt capture or upload function is displayed. The user can capture a receipt image by using the camera to capture a paper receipt or by selecting an electronic receipt from within the device. The device temporarily stores the captured or uploaded receipt image and then sends the data to the server.

[1271] Example: A user presses the "take a photo" button to take a photo of a receipt with their smartphone camera, then presses the "send" button to send the image data to the server.

[1272] Receipt data processing and analysis

[1273] The server receives the receipt image data sent from the terminal and applies OCR (Optical Character Recognition) processing to the image. Specifically, it uses OCR tools such as Tesseract or Google Cloud Vision API to extract text information from the image. This extracted text information includes the product name, price, purchase date, etc.

[1274] Next, the extracted text information is classified using natural language processing (NLP) and assigned specific categories and tags, using NLP libraries such as Python's NLTK and spaCy.

[1275] Example: The server processes an image of a receipt and extracts the text data "Product A - 200 yen," "Product B - 300 yen," and "October 1, 2023." This data is then automatically classified into categories such as "food" and "daily necessities."

[1276] Data storage and point allocation

[1277] The server stores the classified text information in a database (e.g., MySQL or PostgreSQL), including the user ID, purchased item, price, purchase date and time, etc.

[1278] After confirming that the receipt has been successfully processed and the data has been saved, the server will award points to the user's account. The server will generate a notification message and send it to the user's terminal to notify them of the awarding result.

[1279] Example: The server awards 50 points to user ID "12345" and notifies the terminal of the result. The terminal displays the message "50 points have been awarded."

[1280] Data analytics and personalized advertising

[1281] The server periodically inputs the saved purchase data into an AI model to analyze users' purchasing trends. The AI ​​model uses TensorFlow and PyTorch. Based on the results of this analysis, optimal advertisements and coupons are generated for the user and sent to the device.

[1282] Example: The server analyzes the user's purchase history and finds that they frequently purchase items in the "food" category, generates new food-related product advertisements, sends them to the device, and displays "recommended foods" in the user's app.

[1283] Enterprise Dashboard

[1284] The server generates a dashboard for displaying aggregated data and analysis results for the company, visually displaying sales trends and user purchasing trends for a specific period.

[1285] Example: The server analyzes information such as "Sales in the food category have increased by 25% in the past month" and displays it on a dashboard. Based on this information, the company makes a plan to further strengthen its product lineup.

[1286] Verifying User Receipt Information

[1287] The server implements functionality to verify the accuracy of receipt information submitted by the user, including checking that the information contained in the receipt matches an existing database.

[1288] Example: The server verifies that the purchase date and time, store name, product name, etc. on the receipt match the records in the database, and then awards points.

[1289] Example prompt

[1290] "Please explain in detail how the server analyzes the receipt information and awards points to the user after the user takes a photo of the receipt and sends it to the server."

[1291] The above is a concrete example of how to implement the present invention. This system allows users to not only earn points but also enjoy personalized services based on their purchasing habits. It also enables companies to effectively utilize purchasing data to strengthen their sales promotion activities.

[1292] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1293] Step 1:

[1294] User

[1295] The user launches the smartphone app and selects the receipt capture or upload function. The user either takes a photo of a paper receipt using the device's camera or selects an electronic receipt from within the device. An image of the receipt is generated as input, which becomes the input data for the next step.

[1296] Specific operation:

[1297] Pressing the "Take a Receipt Photo" button will activate the camera, which will focus on the receipt and take a photo.

[1298] Press the "Upload" button and select the electronic receipt file on your device.

[1299] Step 2:

[1300] Terminal

[1301] The terminal temporarily stores the captured receipt image and then transmits the data to the server. The input is a photographed or uploaded receipt image. The output is image data that is sent to the server.

[1302] Specific operation:

[1303] The captured receipt image is previewed and the user presses the "Send" button.

[1304] The terminal transmits the image data to the server.

[1305] Step 3:

[1306] server

[1307] The server receives receipt image data sent from the terminal. The receipt image data is sent as input. The received image data is temporarily saved as output.

[1308] Specific operation:

[1309] The received image data is saved in a specific directory.

[1310] Step 4:

[1311] server

[1312] The server applies OCR (Optical Character Recognition) processing to the stored image data. Specifically, it uses Tesseract or Google Cloud Vision API to extract text information from the image. The input is the stored receipt image, and the output is the extracted text information.

[1313] Specific operation:

[1314] Start the OCR engine and input the image file.

[1315] Extract information such as product name, price, and purchase date and time.

[1316] Step 5:

[1317] server

[1318] The server uses natural language processing (NLP) to classify the extracted text information and assign specific categories and tags to it. It uses NLP libraries such as Python's NLTK or spaCy. The extracted text information is the input, and the classified text data is generated as the output.

[1319] Specific operation:

[1320] Analyzes text information and classifies product names and prices by category.

[1321] Add tags such as "food" and "daily necessities."

[1322] Step 6:

[1323] server

[1324] The server stores the classified text information in a database, using MySQL or PostgreSQL. The input is the classified text data, and the output is a new record stored in the database.

[1325] Specific operation:

[1326] Information such as user ID, product name, price, purchase date and time, etc. is inserted into the database via an SQL query.

[1327] Step 7:

[1328] server

[1329] The server assigns points to the user's account based on the information stored in the database. The input is the stored receipt information, and the output is the result of the points assignment.

[1330] Specific operation:

[1331] Run the point-granting algorithm and grant 50 points to user ID "12345".

[1332] A record of points awarded is stored in a database.

[1333] Step 8:

[1334] server

[1335] The server generates a notification message to notify the user of the point allocation result and sends it to the user terminal. The input is the point allocation result data, and the output is the generated notification message.

[1336] Specific operation:

[1337] The result data is sent to the terminal in JSON format.

[1338] Generates the message "50 points awarded."

[1339] Step 9:

[1340] Terminal

[1341] The device receives notification messages from the server and displays the notifications within the app. As input, it has the notification message sent by the server and as output, it generates the notification that is displayed to the user.

[1342] Specific operation:

[1343] The notification message is parsed and displayed in the user interface.

[1344] Step 10:

[1345] server

[1346] The server periodically inputs the saved purchase data into an AI model to analyze the user's purchasing trends. The AI ​​model uses TensorFlow and PyTorch. The input is the user's purchase history data, and the output is an analysis of purchasing trends.

[1347] Specific operation:

[1348] Purchase history data is fed into the AI ​​model, which then identifies purchasing patterns.

[1349] The analysis results include "User ID: 12345 frequently purchases food."

[1350] Step 11:

[1351] server

[1352] The server generates personalized advertisements and coupons based on the analysis results and sends them to the user's device. The input is the analysis results of purchasing trends, and the output is the generated advertisement and coupon data.

[1353] Specific operation:

[1354] Run ad generation algorithms to generate appropriate ads and coupons.

[1355] The generated advertisement is transmitted to the terminal.

[1356] Step 12:

[1357] Terminal

[1358] The device receives advertisements and coupon information sent from the server and displays them in a designated area within the app. The input is the advertisement data sent from the server, and the output is the advertisement displayed.

[1359] Specific operation:

[1360] The received advertising data is analyzed and displayed within the app.

[1361] The above are the processing steps of this system. The user, terminal, and server work together to carry out a series of processes to award points and provide personalized services.

[1362] (Application example 1)

[1363] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1364] While previous systems had the ability to use receipt information to award points and provide personalized advertisements and coupons, they lacked real-time support tools to help store associates improve their service. Furthermore, they lacked an immediate and intuitive interface for store associates to provide personalized service to customers. As a result, their effectiveness in improving customer satisfaction and maximizing store sales was limited.

[1365] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1366] In this invention, the server includes a means for acquiring a receipt image, an optical character recognition means for extracting text information from the acquired receipt image, and a means for classifying the extracted text information and assigning specific categories and tags. This enables a means for scanning receipt information and processing the data in real time. The server also includes a means installed in the smart glasses for providing display information when a store clerk provides customer service. This allows the store clerk to provide instant personalized service to customers.

[1367] definition statement

[1368] The "means for acquiring a receipt image" refers to a device such as a camera or scanner for electronically acquiring a receipt for a product purchased by a user.

[1369] An "optical character recognition means" is software or hardware for analyzing and extracting textual information from captured images.

[1370] The "means for assigning specific categories and tags" refers to a system that has the function of classifying extracted text information and automatically assigning appropriate categories and tags.

[1371] A "database storage means" is a data management device or system used to store classified information securely and efficiently.

[1372] The "means for adding points to a user account" is a system that automatically adds points to a user account based on the acquired and classified information.

[1373] "Artificial intelligence means" refers to machine learning models and algorithms used to analyze users' purchasing habits based on stored information.

[1374] The "means for generating personalized advertisements or coupons" is a system that generates advertisements and coupons optimized for each user based on analyzed purchasing data.

[1375] The "means for displaying on the user terminal" is an interface having a function for displaying the generated advertisements and coupons on the user terminal.

[1376] "Means installed on smart glasses to provide display information when store clerks provide customer service" refers to an application installed on smart glasses, which is a system that displays information necessary for store clerks to provide service to customers in real time.

[1377] "Means for scanning receipt information and processing data in real time" refers to hardware and software for quickly recognizing receipt information and immediately digitizing and processing that information.

[1378] MODE FOR CARRYING OUT THE INVENTION

[1379] The present invention provides an electronic payment system that acquires and analyzes receipt information, awards points to users, and provides personalized advertisements and coupons based on the points. The system includes smart glasses, a server, a database, an artificial intelligence model, and related communication means.

[1380] Capture and send receipt images

[1381] Terminal (smart glasses)

[1382] The store clerk puts on the smart glasses, launches the dedicated app, and scans the receipt for the item purchased by the user with the smart glasses' camera.

[1383] The scanned receipt image is temporarily stored in the smart glasses and the data is sent to the server.

[1384] Examples:

[1385] The store clerk presses the "scan" button and scans the receipt with the smart glasses' camera, after which the image data is automatically sent to the server.

[1386] Receipt data processing and analysis

[1387] server

[1388] The server receives the receipt image data sent from the terminal.

[1389] The received image data is processed using OCR (Optical Character Recognition) to extract text information, including the product name, price, purchase date, etc.

[1390] The extracted text information is classified into categories and tags.

[1391] Examples:

[1392] The server processes the receipt image and extracts the text data "Product A - 200 yen," "Product B - 300 yen," and "October 1, 2023." This is then automatically classified into categories such as "food" and "daily necessities."

[1393] Data storage and point allocation

[1394] server

[1395] The classified text information is stored in a database, including the user ID, purchased item, price, purchase date and time, etc.

[1396] After confirming that the receipt has been successfully processed and the data has been saved, points are credited to the user's account.

[1397] The results of point allocation are notified to the smart glasses.

[1398] Terminal (smart glasses)

[1399] The store clerk's smart glasses receive the point award notification from the server and display it as a message, allowing the store clerk to inform the customer that the user has earned points.

[1400] Examples:

[1401] The server awards 50 points to user ID "12345" and notifies the result to the smart glasses, which display the message "50 points have been awarded."

[1402] Data analytics and personalized advertising

[1403] server

[1404] The saved purchasing data is periodically input into the AI ​​model to analyze users' purchasing trends.

[1405] Based on the analysis results, optimal advertisements and coupons are generated for the user and sent to the smart glasses.

[1406] Terminal (smart glasses)

[1407] The smart glasses receive and display advertisements and coupon information sent from the server, and store clerks can use this information to suggest personalized products to customers.

[1408] Examples:

[1409] The server analyzes the user's purchase history and finds that they frequently purchase items in the "food" category, and generates advertisements for new food-related products. These advertisements are then sent to the smart glasses, and store clerks suggest "recommended foods" to the customer.

[1410] Enterprise Dashboard

[1411] server

[1412] We provide a dashboard for businesses that displays aggregated data and analytical results.

[1413] The dashboard visually displays, for example, sales trends over a specific period or user purchasing trends, allowing companies to gain insights for formulating effective sales strategies.

[1414] Examples:

[1415] The server analyzes the information, such as "Sales in the food category have increased by 25% in the past month," and displays it on a dashboard. Based on this information, the company makes a plan to further strengthen its product lineup.

[1416] Verifying User Receipt Information

[1417] server

[1418] Implement functionality to validate the accuracy of receipt information submitted by users, including checking that the information contained in the receipt matches an existing database.

[1419] Only if the verification is successful, points are awarded and the user is notified.

[1420] Examples:

[1421] The server verifies that the purchase date and time, store name, product name, etc. on the receipt match the records in the database, and then awards points.

[1422] Example prompt sentence:

[1423] Generate new product suggestions and coupons based on the purchase history of user ID "12345." Recent purchases include "Product A - 200 yen," "Product B - 300 yen," and "Purchase date - October 1, 2023."

[1424] The above is a concrete example of how to implement the present invention. This system allows users to not only earn points but also enjoy personalized services based on their purchasing habits. It also enables companies to effectively utilize purchasing data to strengthen their sales promotion activities.

[1425] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1426] Program processing steps

[1427] Step 1:

[1428] A store clerk wearing smart glasses launches a dedicated app and scans the receipt of the customer's purchase with the smart glasses' camera. The input is the scanned receipt image, and the output is image data temporarily stored in the smart glasses. Specifically, the store clerk presses the "scan" button and takes a photo of the receipt using the smart glasses' camera. The captured image is temporarily stored in the smart glasses' memory.

[1429] Step 2:

[1430] The smart glasses send the scanned receipt image to the server. The input is the temporarily stored receipt image data, and the output is the image data sent to the server. Specifically, the smart glasses' communication module is used to send the captured image to the server via a secure communication protocol.

[1431] Step 3:

[1432] The server applies OCR processing to the received receipt image data and extracts text information. The input is the receipt image data received by the server, and the output is the extracted text information. Specifically, the server uses OCR software to analyze the characters in the image and generate text data such as "Product A - 200 yen," "Product B - 300 yen," and "October 1, 2023."

[1433] Step 4:

[1434] The server classifies the extracted text information into categories and tags. The input is the extracted text information, and the output is data with categories and tags. Specifically, the server uses a rule-based or machine learning model to classify the information into categories such as "food" and "daily necessities" based on product name, price, purchase date, etc.

[1435] Step 5:

[1436] The server stores the classified text information in a database. The input is data with categories and tags, and the output is the information stored in the database. Specifically, the server stores information such as the user ID, purchased item, price, and purchase date and time in a relational database or cloud storage.

[1437] Step 6:

[1438] The server verifies the accuracy of the receipt information. The input is the information stored in the database, and the output is the result of the verification. Specifically, the server verifies that the receipt information is accurate by matching it with existing records in the database.

[1439] Step 7:

[1440] After the verification is successful, the server will grant points to the user account. The input is the successfully verified receipt information, and the output is the points granted to the user account. Specifically, the server calculates the points based on the user ID and updates the user's point balance.

[1441] Step 8:

[1442] The result of point allocation is notified to the smart glasses. The input is the point information allocated, and the output is a notification to the smart glasses. Specifically, the server sends the point allocation result to the smart glasses in real time, and the message "50 points have been allocated" is displayed on the store clerk's glasses.

[1443] Step 9:

[1444] The server inputs the stored purchasing data into an AI model to analyze the user's purchasing trends. The input is the stored purchasing data, and the output is the analysis results. Specifically, the server uses a machine learning model to analyze the user's purchasing patterns and predict future purchasing behavior.

[1445] Step 10:

[1446] Based on the analysis results, the server generates personalized advertisements and coupons for users. The input is the analysis results, and the output is the generated advertisements and coupons. Specifically, the server runs an algorithm to generate optimal advertisements and coupons for each user based on purchasing trends.

[1447] Step 11:

[1448] The generated advertisements and coupons are displayed on the smart glasses. The input is the generated advertisement or coupon information, and the output is the information displayed on the smart glasses. Specifically, the server sends the generated advertisement or coupon information to the smart glasses, and the store clerk provides services to the customer based on that information.

[1449] Example prompt sentence:

[1450] Generate new product suggestions and coupons based on the purchase history of user ID "12345." Recent purchases include "Product A - 200 yen," "Product B - 300 yen," and "Purchase date - October 1, 2023."

[1451] The above are the specific processing steps of the system program that realizes this application example. This system allows users to earn points and enjoy personalized services based on their purchasing habits. Store clerks can use the smart glasses to provide customers with personalized product suggestions and services in real time.

[1452] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1453] The present invention is a system that combines an electronic payment system that acquires and analyzes receipt information and awards points to users with an emotion engine that recognizes user emotions. This system includes a user terminal, a server, a database, an artificial intelligence model, the emotion engine, and related communication means.

[1454] Capture and send receipt images

[1455] Terminal (user terminal)

[1456] The user launches the app and selects the "Receipt Capture" or "Receipt Upload" function. The user either takes a photo of a paper receipt with the camera or selects an electronic receipt from within the device.

[1457] The terminal temporarily stores the photographed or uploaded receipt image and transmits the data to the server.

[1458] Examples:

[1459] The user presses the "take a photo" button to take a photo of the receipt with the smartphone camera, then presses the "send" button to send the image data to the server.

[1460] Receipt data processing and analysis

[1461] server

[1462] The server receives the receipt image data sent from the terminal.

[1463] The received image data is processed using OCR (Optical Character Recognition) to extract text information (product name, price, purchase date, etc.) using, for example, Tesseract or Google Cloud Vision API.

[1464] The extracted text information is classified into categories and tags.

[1465] Examples:

[1466] The server processes the receipt image and extracts the text data "Product A - 200 yen," "Product B - 300 yen," and "October 1, 2023." This is then automatically classified into categories such as "food" and "daily necessities."

[1467] Data storage and point allocation

[1468] server

[1469] The classified text information is stored in a database, including the user ID, purchased item, price, purchase date and time, etc.

[1470] After confirming that the receipt has been successfully processed and the data has been saved, points are credited to the user's account.

[1471] The result of the points allocation is notified to the user terminal.

[1472] Terminal

[1473] The user's device receives the point allocation notification from the server and displays the notification within the app, allowing the user to confirm that they have earned points.

[1474] Examples:

[1475] The server awards 50 points to user ID "12345" and notifies the terminal of the result. The terminal displays the message "50 points have been awarded."

[1476] Data analytics and personalized advertising

[1477] server

[1478] The saved purchase data is periodically input into the AI ​​model to analyze users' purchasing trends. The analysis uses a combination of various data such as user attributes and past purchase history.

[1479] Based on the analysis results, optimal advertisements and coupons are generated for the user and sent to the terminal.

[1480] Terminal

[1481] The device receives advertisements and coupon information sent from the server and displays them in a designated area within the app, allowing users to view offers based on their purchasing habits.

[1482] Examples:

[1483] The server analyzes the user's purchase history and finds that they frequently purchase items in the "food" category, generates new food-related product advertisements, sends them to the device, and displays "recommended foods" in the user's app.

[1484] Recognizing and utilizing user emotions

[1485] server

[1486] The emotion engine is incorporated to recognize user emotions. The emotion engine identifies emotions by analyzing, for example, feedback and product ratings entered by users within the app.

[1487] The recognized emotion data is integrated into the purchasing tendency analysis, specifically, to reflect whether the user is satisfied or dissatisfied with a particular product.

[1488] Terminal

[1489] When users enter feedback or ratings, the data is sent to the server, where the emotion engine analyzes it and generates emotion data.

[1490] Examples:

[1491] When a user enters feedback in the app such as "I am satisfied with this product," the emotion engine evaluates it as "satisfied" and adds it to the purchase data.

[1492] Personalized advertising based on emotional data

[1493] server

[1494] Generate more personalized ads and coupons based on user sentiment data, and improve the user experience by prioritizing ads related to products and services that generate high satisfaction.

[1495] Terminal

[1496] Receive personalized advertisements and coupon information based on emotional data and display them in designated areas within the app.

[1497] Examples:

[1498] The server generates related advertisements for products rated as "Satisfied" and displays them in the user's application. For example, a "10% off coupon for new products related to the product you were satisfied with" is displayed.

[1499] Enterprise Dashboard

[1500] server

[1501] It provides a dashboard for businesses to display aggregated data and analytical results, visually displaying insights based on sales trends, user purchasing habits, and sentiment data for a specific period.

[1502] By utilizing emotional data, companies can use it as a reference for more effective marketing strategies and product development.

[1503] Examples:

[1504] The dashboard displays information such as "Sales in the food category have increased by 25% in the past month" along with user satisfaction ratings for the relevant products, allowing companies to plan product improvements and new product development.

[1505] Verifying User Receipt Information

[1506] server

[1507] Validate the accuracy of the receipt information provided by the user. This involves checking whether the information contained in the receipt matches an existing database. Only if the validation is successful are points awarded and the user notified.

[1508] Examples:

[1509] The server verifies that the purchase date and time, store name, product name, etc. on the receipt match the records in the database, and then awards points.

[1510] The above is a specific embodiment for implementing the present invention. This system allows users to earn points through receipts and receive personalized advertisements and coupons, and further improves the user experience by utilizing emotional data. Furthermore, companies can optimize their sales promotion activities based on detailed purchasing data and emotional data.

[1511] The processing flow will be explained below.

[1512] Step 1:

[1513] The user launches the app and selects the "Receipt Capture" or "Receipt Upload" function. The user either takes a photo of a paper receipt with the camera or selects an electronic receipt from within the device. When the user presses the "Send" button, the receipt image data is temporarily saved within the device.

[1514] Step 2:

[1515] The device prepares the captured or uploaded receipt image for transmission to the server, specifically by encoding the image data into the appropriate format and attaching metadata (user ID, timestamp, etc.).

[1516] Step 3:

[1517] The device then sends the prepared data to the server, which typically uses the HTTPS protocol to ensure secure communication.

[1518] Step 4:

[1519] The server receives the receipt image data sent from the terminal, temporarily stores the received raw data, and prepares it for OCR processing.

[1520] Step 5:

[1521] The server performs OCR (optical character recognition) processing to extract text information (product name, price, purchase date, etc.) from the receipt image. The OCR engine used can be, for example, Tesseract or Google Cloud Vision API.

[1522] Step 6:

[1523] The server analyzes the extracted text information and assigns categories (e.g., food, daily necessities, etc.) and tags (e.g., sale, double points, etc.) using a pre-trained classification model.

[1524] Step 7:

[1525] The server stores the classified text information in a database, including the user ID, purchased item, price, purchase date and time, and category / tag.

[1526] Step 8:

[1527] The server verifies that the receipt has been processed successfully and then grants the points to the user's account by executing an SQL query using the user ID as the key to add the points.

[1528] Step 9:

[1529] The server generates a response to notify the user of the point allocation result. This is usually a JSON formatted message.

[1530] Step 10:

[1531] The device receives the response from the server and displays a notification to the user that points have been awarded. Specifically, the device uses the app's notification function to display a message such as "50 points have been awarded."

[1532] Step 11:

[1533] The server periodically supplies the stored purchase data to the AI ​​model to analyze users' purchasing trends. The analysis uses a combination of various data, such as user attributes and past purchase history.

[1534] Step 12:

[1535] The server generates personalized ads and coupons based on the results of the AI ​​model, which are tailored to the user's purchasing habits.

[1536] Step 13:

[1537] The server then sends the generated advertisements and coupons to the user's device, again using a secure communication protocol.

[1538] Step 14:

[1539] The device receives advertisements and coupon information sent from the server and displays them in a designated area within the app, allowing users to view offers based on their purchasing habits.

[1540] Step 15:

[1541] The server runs an emotion engine to recognize the user's emotions and analyzes the feedback and product ratings that the user writes in the app, generating analyzed emotion data.

[1542] Step 16:

[1543] The device sends the feedback and evaluation data entered by the user to the server, where the emotion engine analyzes it and generates emotion data.

[1544] Step 17:

[1545] The server integrates the recognized emotion data into a purchasing tendency analysis, specifically, reflecting whether the user is satisfied or dissatisfied with a particular product.

[1546] Step 18:

[1547] The server generates more personalized advertisements and coupons based on the user's emotional data, improving the user experience by prioritizing advertisements related to products and services that generate high satisfaction.

[1548] Step 19:

[1549] The device receives personalized advertisements and coupons based on emotion data and displays them in a designated area within the app, allowing users to view emotion-based offers.

[1550] Step 20:

[1551] The server provides a dashboard for businesses to display aggregate data and sentiment data, visually displaying sales trends over a specific period, user purchasing habits, and insights based on sentiment data.

[1552] Step 21:

[1553] The server verifies the accuracy of the receipt information provided by the user. Validation involves checking whether the information contained in the receipt matches an existing database record. Only if the validation is successful are points awarded and the user notified.

[1554] Specific examples

[1555] For example, if a user enters feedback within the app such as "I was satisfied with this product," the emotion engine will rate it as "satisfied" and add it to the purchase data. The server will generate related advertisements for products rated as "satisfied" and display them in the user's app. A dashboard for businesses will display information such as "Sales in the food category have increased by 25% in the past month, and user satisfaction with the relevant products is high," allowing businesses to make plans for product improvements and new product development.

[1556] The above are the processing steps and specific operations of the system's program. This allows users to earn points through receipts and receive personalized advertisements and coupons, and also provides an improved user experience by utilizing emotional data. Furthermore, companies can optimize their sales promotion activities based on detailed purchasing data and emotional data.

[1557] Example 2

[1558] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1559] Conventional electronic payment systems provide functions for obtaining receipt information and awarding points, but they lack the ability to provide personalized advertising using user emotional data or to analyze detailed purchasing trends. Furthermore, they lack a means to provide businesses with analysis results that integrate purchasing data and emotional data. This has made it difficult to improve user experience and optimize corporate marketing strategies.

[1560] The identification process performed by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for acquiring a receipt image, an optical character recognition means for extracting text information from the acquired receipt image, a means for categorizing the extracted text information and assigning specific categories and tags, a means for saving the classified information in a database, a means for awarding points to a user account based on the saved information, an artificial intelligence means for analyzing a user's purchasing habits based on the saved information, a means for generating personalized advertisements or coupons for the user based on the analysis results, a means for displaying the advertisements or coupons on the user terminal, an emotion recognition means for recognizing the user's emotional data and further analyzing the purchasing habits based on the emotional data, a means for generating personalized advertisements or coupons based on the emotional data, and a means for providing a dashboard displaying aggregated data and analysis results to businesses. This allows users to earn points through receipts and receive personalized advertisements that utilize their emotional data. Businesses can also receive analysis results based on detailed purchasing data and emotional data via the dashboard, optimizing their sales promotion activities.

[1561] "Means for acquiring receipt images" refers to devices or software that have the function of electronically acquiring receipt images by a user taking a photo of a paper receipt with a camera or selecting an electronic receipt from within a terminal.

[1562] "Optical character recognition means" refers to technology or devices for extracting text information from an acquired receipt image, and specifically refers to devices that use OCR (optical character recognition) technology to read character information.

[1563] "Means for classifying extracted text information and assigning specific categories and tags" refers to software or algorithms for classifying text information obtained by OCR processing into specific categories (e.g., food, daily necessities, etc.) and tags (e.g., price, product name, etc.).

[1564] "Means for storing classified information in a database" refers to a device or system that stores the classified information extracted from receipts in a database for future reference and analysis.

[1565] "Means for awarding points to a user account based on stored information" means a system or algorithm for calculating and awarding points to a user's account based on stored information in a database.

[1566] "Artificial intelligence means" refers to machine learning algorithms and AI models that use stored receipt information and other data to analyze users' purchasing habits.

[1567] "Means for generating personalized advertisements or coupons" refers to systems or software for generating advertisements or coupons that are optimal for individual users based on the user's purchasing habits and emotional data.

[1568] "Means for displaying advertisements or coupons on a user terminal" refers to an interface or software for displaying the generated advertisements or coupons on a user's smartphone or other terminal.

[1569] "Emotion recognition means" refers to a device or software that analyzes user feedback and evaluation data to identify user emotions and reflect the results in purchasing trend analysis.

[1570] "A means of providing dashboards that display aggregated data and analytical results for businesses" refers to visual interfaces and software that allow businesses to grasp their sales trends, user purchasing habits, and sentiment data at a glance.

[1571] The present invention is a system that combines an electronic payment system that acquires and analyzes receipt information and awards points to users with an emotion engine that recognizes user emotions. This system includes a user terminal, a server, a database, an artificial intelligence model, the emotion engine, and related communication means.

[1572] Capture and send receipt images

[1573] Terminal

[1574] The user launches the smartphone app and selects the "Receipt Capture" or "Receipt Upload" function. The user either takes a photo of a paper receipt with the camera or selects an electronic receipt from within the device. The device temporarily stores the captured or uploaded receipt image and sends the data to the server.

[1575] Examples:

[1576] The user presses the "take a photo" button, takes a photo of the receipt with the smartphone camera, and then presses the "send" button to send the image data to the server.

[1577] Example prompt for generative AI model:

[1578] "Please explain the steps to take a picture of the receipt and send it to the server."

[1579] Receipt data processing and analysis

[1580] server

[1581] The server receives the receipt image data sent from the device. It performs OCR processing on the received image data using Tesseract or Google Cloud Vision API to extract text information (product name, price, purchase date and time, etc.) and classifies the extracted text information into categories and tags.

[1582] Examples:

[1583] The server processes the receipt image and extracts the text data "Product A - 200 yen," "Product B - 300 yen," and "October 1, 2023," and then classifies this into categories such as "food" and "daily necessities."

[1584] Example prompt for generative AI model:

[1585] "Please explain the steps to extract text information from receipt images and classify it into categories."

[1586] Data storage and point allocation

[1587] server

[1588] The classified text information is stored in a database. The stored information includes the user ID, purchased item, price, purchase date and time, etc. After confirming that the receipt has been processed correctly and the data has been saved, points are awarded to the user's account. The result of point awarding is notified to the user's device.

[1589] Terminal

[1590] The user device receives the point allocation notification from the server and displays the notification within the app.

[1591] Examples:

[1592] The server awards 50 points to user ID "12345" and notifies the terminal of the result. The terminal displays the message "50 points have been awarded."

[1593] Example prompt for generative AI model:

[1594] Please explain the process from when the user submits the receipt until points are awarded.

[1595] Data analytics and personalized advertising

[1596] server

[1597] The stored purchase data is periodically input into an AI model to analyze the user's purchasing trends. This analysis uses user attributes and past purchase history. Based on the analysis results, optimal advertisements and coupons are generated for the user and sent to the device.

[1598] Terminal

[1599] The device receives advertisements and coupon information sent from the server and displays it within the app.

[1600] Examples:

[1601] The server uses an AI model to analyze past purchase data, generate advertisements for new food-related products for users who frequently purchase products in the "food" category, and send them to the user's device. The user's device then displays "recommended food products" within the app.

[1602] Example prompt for generative AI model:

[1603] "Please explain the steps to analyze user purchasing data and generate optimal advertisements."

[1604] Recognizing and utilizing user emotions

[1605] server

[1606] Incorporate an emotion engine to recognize user emotions. Analyze feedback and product ratings written by users within the app to identify emotions. Integrate the recognized emotion data into purchasing tendency analysis.

[1607] Terminal

[1608] When users enter feedback or ratings, the data is sent to the server and analyzed by the emotion engine.

[1609] Examples:

[1610] The user enters feedback in the app, such as "I'm satisfied with this product," and the data is sent to the server. The emotion engine evaluates the feedback as "satisfied," and the result is reflected in the purchase data.

[1611] Example prompt for generative AI model:

[1612] "Please explain the steps to recognize user sentiment data and reflect it in purchasing data."

[1613] Personalized advertising based on emotional data

[1614] server

[1615] Generate personalized ads and coupons based on user sentiment data, and prioritize ads related to products and services that generate high satisfaction.

[1616] Terminal

[1617] Receive personalized ads and coupon information based on emotional data and display it within the app.

[1618] Examples:

[1619] The server will prioritize generating related advertisements for products that have been rated as "satisfactory." For example, a "10% off coupon for a new product related to the product you were satisfied with" will be sent to the device and displayed within the app.

[1620] Example prompt for generative AI model:

[1621] "Explain the steps to generate and display personalized ads based on emotional data."

[1622] Enterprise Dashboard

[1623] server

[1624] It provides a dashboard for businesses to display aggregated data and analytical results, visually displaying insights based on sales trends, user purchasing habits, and sentiment data.

[1625] Examples:

[1626] The dashboard displays information such as "Sales in the food category have increased by 25% in the past month" along with user satisfaction ratings for the relevant products. Companies can use this information to consider marketing strategies and product development policies.

[1627] Example prompt for generative AI model:

[1628] "Describe the data you want to display in your enterprise dashboard."

[1629] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1630] Step 1: Capture and send a receipt image

[1631] 1-1. The user launches the app on their smartphone and selects the "take a photo of a receipt" or "upload a receipt" function.

[1632] 1-2. The user takes a photo of a paper receipt with the camera or selects an electronic receipt from within the device. The input data is an image file of the paper or electronic receipt.

[1633] 1-3. The device temporarily saves the captured or uploaded receipt image and sends the data to the server. The output data is the receipt image file sent to the server.

[1634] Specific behavior:

[1635] The user presses the "take a photo" button to take a photo of the receipt with the smartphone camera, then presses the "send" button, and the device transfers the image data to the server.

[1636] Step 2: Process and parse receipt data

[1637] 2-1. The server receives the receipt image data sent from the terminal. The input data is the receipt image file sent from the terminal.

[1638] 2-2. The server uses Tesseract or Google Cloud Vision API to perform OCR processing on the image data received and extracts text information (product name, price, purchase date, etc.). OCR processing is performed as data processing, and the output data is the extracted text information.

[1639] 2-3. Classify the extracted text information into categories and tags. Data classification is performed, and the output data is text information with categories or tags assigned.

[1640] Specific behavior:

[1641] After the server obtains the receipt image data, it calls the Tesseract OCR engine to perform character recognition, generating text data such as "Product A - 200 yen," "Product B - 300 yen," and "October 1, 2023," which is then automatically classified into categories such as "food" and "daily necessities."

[1642] Step 3: Save your data and earn points

[1643] 3-1. The server saves the classified text information in a database. The input data is text information with categories. The data is saved, and the output data is the information stored in the database.

[1644] 3-2. After confirming that the receipt has been successfully processed and the data has been saved, the server will award points to the user account. The input data is the saved data. The awarding process is executed, and the output data is the awarded points.

[1645] 3-3. The server notifies the user terminal of the point allocation result. The input data is the point allocation result. The notification is made, and the output data is the notification information sent to the user terminal.

[1646] Specific behavior:

[1647] The server saves the information "User ID: 12345, Product: Product A, Price: 200 yen, Purchase date: October 1, 2023" in the database, triggers the point allocation process, adds 50 points to user ID "12345", and sends the result to the terminal. The user terminal displays the message "50 points have been allocated."

[1648] Step 4: Data analysis and personalized advertising

[1649] 4-1. The server inputs the stored purchasing data into the AI ​​model to analyze the user's purchasing trends. The input data is the stored purchasing data. Data analysis is performed, and the output data is the analysis results.

[1650] 4-2. Based on the analysis results, the server generates the optimal advertisement or coupon for the user and sends it to the terminal. The input data is the analysis results. The generation process is carried out, and the output data is the generated advertisement or coupon.

[1651] 4-3. The device receives the advertisement or coupon information sent from the server and displays it within the app. The input data is the advertisement or coupon information sent from the server. The display process is performed, and the output data is the advertisement or coupon displayed within the app.

[1652] Specific behavior:

[1653] The server uses an AI model to analyze past purchase data, generate advertisements for new food-related products for users who frequently purchase products in the "food" category, and send them to the user's device. The user's device then displays "recommended food products" within the app.

[1654] Step 5: Recognize and leverage user emotions

[1655] 5-1. The server incorporates an emotion engine to recognize the user's emotions. The input data is the feedback and product ratings entered by the user within the app. An analysis process is performed, and the output data is the recognized emotion data.

[1656] 5-2. The server integrates the recognized emotion data into the purchasing tendency analysis. The input data is emotion data. The integration process is carried out, and the output data is the integrated analysis data.

[1657] 5-3. When a user inputs feedback or evaluation, the data is sent to the server and analyzed by the emotion engine. The input data is the feedback or evaluation data. The analysis process is performed, and the output data is the analyzed emotion data.

[1658] Specific behavior:

[1659] The user enters feedback in the app, such as "I'm satisfied with this product," and the data is sent to the server. The emotion engine evaluates the feedback as "satisfied" and integrates the evaluation into the purchase data.

[1660] Step 6: Personalized advertising based on emotional data

[1661] 6-1. The server generates personalized advertisements and coupons based on the user's emotional data. The input data is the emotional data. The generation process is performed, and the output data is personalized advertisements and coupons.

[1662] 6-2. The server sends advertisements and coupons to the user terminal. The input data is the personalized advertisements and coupons. The transmission process is carried out, and the output data is the advertisement and coupon data sent to the user terminal.

[1663] 6-3. The device receives advertisements or coupon information based on the emotion data and displays it within the app. The input data is advertisements or coupon information based on the emotion data. The display process is performed, and the output data is the advertisement or coupon displayed within the app.

[1664] Specific behavior:

[1665] The server generates related advertisements for products rated as "satisfactory" and sends them to the user's device. For example, a "10% off coupon for a new product related to the product you were satisfied with" is displayed in the app.

[1666] Step 7: Enterprise dashboard

[1667] 7-1. The server provides a dashboard for companies to display aggregated data and analysis results. The input data is the aggregated data and analysis results. The display process is carried out, and the output data is the information displayed on the dashboard.

[1668] Specific behavior:

[1669] The dashboard displays information such as "Sales in the food category have increased by 25% in the past month" along with user satisfaction ratings for the relevant products. Companies can use this information to consider the direction of their marketing strategies and product development.

[1670] (Application example 2)

[1671] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1672] Current electronic payment systems are limited to collecting receipt information and awarding points, and lack a deep understanding of users' purchasing behavior and emotions. Furthermore, current systems lack the functionality to provide personalized advertising or coupons that take user emotions into account, preventing improvements to the user experience. Furthermore, the data provided to businesses is limited and insufficient for use in marketing and product development.

[1673] The identification process performed by the identification processing unit 290 of the data processing device 12 in Application Example 2 is implemented by the following means. In this invention, the server includes: means for acquiring receipt images; optical character recognition means for extracting text information from the acquired receipt images; means for categorizing the extracted text information and assigning specific categories and tags; means for saving the classified information in a database; means for awarding points to a user account based on the saved information; artificial intelligence means for analyzing users' purchasing trends based on the saved information; means including an emotion engine for recognizing and integrating user emotions into the analysis; means for generating personalized advertisements or coupons for users based on the analysis results; and means for displaying the advertisements or coupons on the user terminal. This enables analysis of user purchasing behavior and emotions based on receipt information, thereby improving the user experience by providing more accurate personalized advertisements and coupons. Furthermore, providing a wide range of data analysis results through a corporate dashboard can improve the quality of marketing strategies and product development.

[1674] The "receipt image acquisition means" is a means for a user to take or upload a receipt image.

[1675] The "optical character recognition means" is a means for extracting text information from the captured receipt image.

[1676] The "text information classification means" is a means for classifying extracted text information and assigning specific categories and tags to it.

[1677] "Database storage means" refers to a means for safely storing classified information in a database.

[1678] The "points granting means" is a means for granting points to a user account based on the stored information.

[1679] "Artificial intelligence means" refers to means for analyzing users' purchasing trends based on stored information.

[1680] An "emotion engine" is a means for recognizing and integrating user emotions into analysis.

[1681] The "personalized advertisement generating means" is a means for generating advertisements or coupons individually tailored for users based on the analysis results.

[1682] The "advertisement display means" is a means for displaying the generated advertisement or coupon on the user terminal.

[1683] This invention combines an emotion engine with an electronic payment system that acquires and analyzes receipt information and awards points to users. This system is realized by a program consisting of the following elements:

[1684] First, the user terminal provides a means for acquiring receipt images. The user launches the application and sends the receipt image to the system by taking a photo or uploading it. This process uses the smartphone's camera or image upload function.

[1685] The server then extracts text information from the captured receipt image using optical character recognition (OCR) software such as OpenCV or Tesseract. The extracted text information includes data points such as product name, price, and purchase date and time.

[1686] The extracted text information is then classified and assigned categories and tags. The server stores the classified information in a database, including the user ID, purchased item, price, purchase date and time, etc.

[1687] Based on the saved information, the server will assign points to the user's account. The result of the points assignment will be notified to the user's device, and a message stating "Points have been assigned" will be displayed within the app.

[1688] Furthermore, the server uses artificial intelligence to analyze the stored purchase data and understand the user's purchasing trends. This analysis integrates data such as past purchase history and user attributes. This process uses a generative AI model.

[1689] The emotion engine recognizes user emotions and integrates them into purchasing trend analysis. For example, when a user enters product ratings or feedback within the app, that data is sent to the emotion engine, and the system generates emotion data such as "satisfied" or "dissatisfied."

[1690] Based on this emotional data, a means is provided for generating more personalized advertisements and coupons. The server creates optimal advertisements and coupons based on the user's purchasing tendencies and emotional data, and sends them to the user's device. The user's device then displays these advertisements and coupons in a designated area within the app.

[1691] As a specific example, if a user purchases "milk" and rates it as "very satisfied," the system will generate a 10% off coupon for the next purchase related to "milk" and provide it to the user.

[1692] Additionally, a dashboard for businesses is provided to visually display sales trends and purchasing habits, including insights based on sentiment data, which can be used by businesses to develop more effective marketing strategies and product development.

[1693] Finally, a means is provided for verifying the accuracy of the receipt information provided by the user. The server compares the receipt information with an existing database, confirming accuracy, and then awards points. In this way, the system improves the user experience and provides valuable data for businesses.

[1694] (Example of a prompt)

[1695] Please enter your thoughts about the product you purchased, "Milk." Example: "I am very satisfied with this milk."

[1696] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1697] Step 1: Get the receipt image

[1698] The user launches the smartphone app and selects the "take a photo of a receipt" or "upload a receipt" function. The user either takes a photo of a paper receipt with the camera or selects an electronic receipt from within the device. The device temporarily saves the captured or uploaded receipt image. The input is the receipt image selected by the user, and the temporarily saved image data is generated as the output.

[1699] Step 2: Send a receipt image

[1700] The user device sends the temporarily saved receipt image to the server. Here, the receipt image data is sent as input to the server and a success response is received. Specifically, the image data is sent to a specified endpoint on the server using the HTTP protocol.

[1701] Step 3: Optical Character Recognition (OCR)

[1702] The server applies OCR processing to the receipt image data it receives. The input is the receipt image data, and the output is the extracted text information. Specifically, it uses Tesseract and the Google Cloud Vision API to extract the text information.

[1703] Step 4: Classifying text information

[1704] The server classifies the extracted text information into categories and tags. The input is the text information resulting from the OCR process, and the output is the categorized and tagged text information. Here, classification is performed using a predefined rule-based or machine learning algorithm.

[1705] Step 5: Saving to the Database

[1706] The server stores the categorized text information in a database. The input is the categorized and tagged text information, and the output is a record stored in the database. Specifically, the database operation involves information such as the user ID, purchased item, price, and purchase date and time.

[1707] Step 6: Points awarded

[1708] The server will then assign points to the user's account based on the stored information. The input is the purchase data in the database, and the output is the updated user's points information. Specifically, points are calculated according to existing business rules, and the user's points balance is updated.

[1709] Step 7: Notification of points awarded

[1710] The server notifies the user device of the point awarding results. The input is the updated user point information, and the output is a notification message. Specifically, a push notification or in-app notification is used to send a message saying "Points have been awarded."

[1711] Step 8: Analyze purchasing data

[1712] The server periodically inputs the stored purchasing data into the AI ​​model to analyze the user's purchasing trends. The input is the purchasing data in the database, and the output is the analysis results of purchasing trends. The generative AI model combines past purchasing history and user attributes to analyze purchasing trends.

[1713] Step 9: Recognizing Emotional Data

[1714] The server uses an emotion engine to recognize the user's emotions. The input is the user's feedback and evaluation data, and the output is the recognized emotion data. Specifically, the emotion engine analyzes the user's input data and extracts emotions such as "satisfaction" or "dissatisfaction."

[1715] Step 10: Generate personalized ads

[1716] The server generates personalized advertisements and coupons based on purchasing trend analysis and emotional data. The input is the purchasing trend analysis results and emotional data, and the output is individually tailored advertisements and coupon information. Here, optimal advertisements are generated based on the user's interests and satisfaction.

[1717] Step 11: Displaying Ads

[1718] The user device receives the advertisements and coupon information sent from the server and displays them in a designated area within the app. The input is personalized advertisements and coupon information, and the output is the advertisements and coupons displayed on the user's device screen.

[1719] Finally, as a concrete example of a prompt:

[1720] "Please enter your thoughts about the product you purchased, 'Milk'. Example: 'I am very satisfied with this milk.'"

[1721] is used.

[1722] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1723] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1724] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1725] [Fourth embodiment]

[1726] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1727] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1728] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1729] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1730] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1731] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1732] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1733] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1734] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1735] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1736] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1737] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1738] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1739] The present invention provides an electronic payment system that acquires and analyzes receipt information and awards points to users, and a system that provides personalized advertisements and coupons based on the points. This system includes a user terminal, a server, a database, an artificial intelligence model, and related communication means.

[1740] Capture and send receipt images

[1741] Terminal (user terminal)

[1742] When a user launches the app, they are presented with the option to take a photo or upload a receipt.

[1743] The user either takes a photo of a paper receipt with the camera or selects an electronic receipt from within the device.

[1744] The terminal temporarily stores the photographed or uploaded receipt image and transmits the data to the server.

[1745] Examples:

[1746] The user presses the "take a photo" button to take a photo of the receipt with the smartphone camera, then presses the "send" button to send the image data to the server.

[1747] Receipt data processing and analysis

[1748] server

[1749] The server receives the receipt image data sent from the terminal.

[1750] The received image data is processed using OCR (Optical Character Recognition) to extract text information, including the product name, price, purchase date, etc.

[1751] The extracted text information is classified into categories and tags.

[1752] Examples:

[1753] The server processes the receipt image and extracts the text data "Product A - 200 yen," "Product B - 300 yen," and "October 1, 2023." This is then automatically classified into categories such as "food" and "daily necessities."

[1754] Data storage and point allocation

[1755] server

[1756] The classified text information is stored in a database, including the user ID, purchased item, price, purchase date and time, etc.

[1757] After confirming that the receipt has been successfully processed and the data has been saved, points are credited to the user's account.

[1758] The result of the points allocation is notified to the user terminal.

[1759] Terminal

[1760] The user's device receives the point allocation notification from the server and displays the notification within the app, allowing the user to confirm that they have earned points.

[1761] Examples:

[1762] The server awards 50 points to user ID "12345" and notifies the terminal of the result. The terminal displays the message "50 points have been awarded."

[1763] Data analytics and personalized advertising

[1764] server

[1765] The saved purchasing data is periodically input into the AI ​​model to analyze users' purchasing trends.

[1766] Based on the analysis results, optimal advertisements and coupons are generated for the user and sent to the terminal.

[1767] Terminal

[1768] The device receives advertisements and coupons sent from the server and displays them in a designated area within the app, allowing users to view offers based on their purchasing habits.

[1769] Examples:

[1770] The server analyzes the user's purchase history and finds that they frequently purchase items in the "food" category, generates new food-related product advertisements, sends them to the device, and displays "recommended foods" in the user's app.

[1771] Enterprise Dashboard

[1772] server

[1773] We provide a dashboard for businesses that displays aggregated data and analytical results.

[1774] The dashboard visually displays, for example, sales trends over a specific period or user purchasing trends, allowing companies to gain insights for formulating effective sales strategies.

[1775] Examples:

[1776] The server analyzes the information, such as "Sales in the food category have increased by 25% in the past month," and displays it on a dashboard. Based on this information, the company makes a plan to further strengthen its product lineup.

[1777] Verifying User Receipt Information

[1778] server

[1779] Implement functionality to validate the accuracy of receipt information submitted by users, including checking that the information contained in the receipt matches an existing database.

[1780] Only if the verification is successful, points are awarded and the user is notified.

[1781] Examples:

[1782] The server verifies that the purchase date and time, store name, product name, etc. on the receipt match the records in the database, and then awards points.

[1783] The above is a concrete example of how to implement the present invention. This system allows users to not only earn points but also enjoy personalized services based on their purchasing habits. It also enables companies to effectively utilize purchasing data to strengthen their sales promotion activities.

[1784] The processing flow will be explained below.

[1785] Step 1:

[1786] The user launches the app and selects the "Receipt Capture" or "Receipt Upload" function. The user either takes a photo of a paper receipt with the camera or selects an electronic receipt from within the device. When the user presses the "Send" button, the receipt image data is temporarily saved within the device.

[1787] Step 2:

[1788] The device prepares the captured or uploaded receipt image for transmission to the server, specifically by encoding the image data into the appropriate format and attaching metadata (user ID, timestamp, etc.).

[1789] Step 3:

[1790] The device then sends the prepared data to the server, which typically uses the HTTPS protocol to ensure secure communication.

[1791] Step 4:

[1792] The server receives the receipt image data sent from the terminal, temporarily stores the received raw data, and prepares it for OCR processing.

[1793] Step 5:

[1794] The server performs OCR (optical character recognition) processing to extract text information (product name, price, purchase date, etc.) from the receipt image. The OCR engine used can be, for example, Tesseract or Google Cloud Vision API.

[1795] Step 6:

[1796] The server analyzes the extracted text information and assigns categories (e.g., food, daily necessities, etc.) and tags (e.g., sale, double points, etc.) using a pre-trained classification model.

[1797] Step 7:

[1798] The server stores the classified text information in a database, including the user ID, purchased item, price, purchase date and time, and category / tag.

[1799] Step 8:

[1800] The server verifies that the receipt has been processed successfully and then grants the points to the user's account by executing an SQL query using the user ID as the key to add the points.

[1801] Step 9:

[1802] The server generates a response to notify the user of the point allocation result. This is usually a JSON formatted message.

[1803] Step 10:

[1804] The device receives the response from the server and displays a notification to the user that points have been awarded. Specifically, the device uses the app's notification function to display a message such as "50 points have been awarded."

[1805] Step 11:

[1806] The server periodically supplies the stored purchase data to the AI ​​model to analyze users' purchasing trends. The analysis uses a combination of various data, such as user attributes and past purchase history.

[1807] Step 12:

[1808] The server generates personalized ads and coupons based on the results of the AI ​​model, which are tailored to the user's purchasing habits.

[1809] Step 13:

[1810] The server then sends the generated advertisements and coupons to the user's device, again using a secure communication protocol.

[1811] Step 14:

[1812] The device receives advertisements and coupon information sent from the server and displays them in a designated area within the app, allowing users to view offers based on their purchasing habits.

[1813] Step 15:

[1814] The server provides a dashboard for companies to display aggregated data and analysis results, visually displaying sales trends and user purchasing trends for a specific period.

[1815] Step 16:

[1816] The server verifies the accuracy of the receipt information provided by the user. This involves checking that the receipt contents match existing database records. Only if the verification is successful are points awarded and the user notified.

[1817] These are the processing steps and specific operations of the system program. This allows users to earn points through receipts and receive personalized advertisements and coupons. It also enables companies to optimize their sales promotion activities based on detailed purchase data.

[1818] Example 1

[1819] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1820] The present invention relates to a system that effectively acquires and analyzes receipt information and awards points to users. Conventional systems lack accurate extraction of receipt information and subsequent analysis, making it difficult to award points to users or provide personalized services. Furthermore, they lack effective data analysis and purchasing trend information for businesses. As a result, these systems fall short in improving user experience and promoting corporate sales.

[1821] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1822] In this invention, the server includes a means for acquiring receipt images, an optical character recognition means for extracting text information from the acquired receipt images, a means for classifying the extracted text information using natural language processing and assigning specific categories and tags, a means for saving the classified information in a database, a means for awarding points to a user account based on the saved information, a means for notifying the user terminal of the point awarding results, an artificial intelligence means for analyzing the user's purchasing habits based on the saved information, a means for generating personalized advertisements or coupons for the user based on the analysis results, and a means for displaying the advertisements or coupons on the user terminal. This allows users to easily acquire and analyze receipt information and earn points, as well as enjoy personalized services based on their purchasing habits. It also enables companies to effectively utilize purchasing data to strengthen their sales promotion activities.

[1823] A "user terminal" refers to an electronic device operated by a user, and includes smartphones, tablets, laptops, and the like.

[1824] "Receipt image" refers to image data of a receipt photographed or uploaded by a user.

[1825] A "server" refers to a centralized computer system that receives data sent from user terminals over a network and performs subsequent processing.

[1826] "Optical Character Recognition (OCR) Method" means a technology for extracting text information from receipt images, such as Tesseract, Google Cloud Vision, or similar tools.

[1827] "Natural language processing means" refers to processes that use algorithms or libraries, such as NLTK or spaCy, to classify extracted text information into specific categories or tags.

[1828] "Database" refers to a software system for systematically storing and managing classified information, including relational databases such as MySQL and PostgreSQL.

[1829] "Artificial intelligence means" refers to machine learning models and algorithms used to analyze user purchasing trends based on stored data, and uses frameworks such as TensorFlow and PyTorch.

[1830] "Personalized ads or coupons" refers to ads or coupons that are individually generated based on a user's purchasing habits.

[1831] "Verification Procedure" refers to the process of verifying the accuracy of receipt information submitted by a user and ensuring it matches existing data in the database.

[1832] A "dashboard" is an interface for visually displaying aggregated data and analytical results for companies, providing data in the form of graphs, charts, etc.

[1833] The present invention provides an electronic payment system that acquires and analyzes receipt information and awards points to users, and a system that provides personalized advertisements and coupons based on the points. This system includes a user terminal, a server, a database, an artificial intelligence model, and related communication means.

[1834] Capture and send receipt images

[1835] When a user launches the app, a receipt capture or upload function is displayed. The user can capture a receipt image by using the camera to capture a paper receipt or by selecting an electronic receipt from within the device. The device temporarily stores the captured or uploaded receipt image and then sends the data to the server.

[1836] Example: A user presses the "take a photo" button to take a photo of a receipt with their smartphone camera, then presses the "send" button to send the image data to the server.

[1837] Receipt data processing and analysis

[1838] The server receives the receipt image data sent from the terminal and applies OCR (Optical Character Recognition) processing to the image. Specifically, it uses OCR tools such as Tesseract or Google Cloud Vision API to extract text information from the image. This extracted text information includes the product name, price, purchase date, etc.

[1839] Next, the extracted text information is classified using natural language processing (NLP) and assigned specific categories and tags, using NLP libraries such as Python's NLTK and spaCy.

[1840] Example: The server processes an image of a receipt and extracts the text data "Product A - 200 yen," "Product B - 300 yen," and "October 1, 2023." This data is then automatically classified into categories such as "food" and "daily necessities."

[1841] Data storage and point allocation

[1842] The server stores the classified text information in a database (e.g., MySQL or PostgreSQL), including the user ID, purchased item, price, purchase date and time, etc.

[1843] After confirming that the receipt has been successfully processed and the data has been saved, the server will award points to the user's account. The server will generate a notification message and send it to the user's terminal to notify them of the awarding result.

[1844] Example: The server awards 50 points to user ID "12345" and notifies the terminal of the result. The terminal displays the message "50 points have been awarded."

[1845] Data analytics and personalized advertising

[1846] The server periodically inputs the saved purchase data into an AI model to analyze users' purchasing trends. The AI ​​model uses TensorFlow and PyTorch. Based on the results of this analysis, optimal advertisements and coupons are generated for the user and sent to the device.

[1847] Example: The server analyzes the user's purchase history and finds that they frequently purchase items in the "food" category, generates new food-related product advertisements, sends them to the device, and displays "recommended foods" in the user's app.

[1848] Enterprise Dashboard

[1849] The server generates a dashboard for displaying aggregated data and analysis results for the company, visually displaying sales trends and user purchasing trends for a specific period.

[1850] Example: The server analyzes information such as "Sales in the food category have increased by 25% in the past month" and displays it on a dashboard. Based on this information, the company makes a plan to further strengthen its product lineup.

[1851] Verifying User Receipt Information

[1852] The server implements functionality to verify the accuracy of receipt information submitted by the user, including checking that the information contained in the receipt matches an existing database.

[1853] Example: The server verifies that the purchase date and time, store name, product name, etc. on the receipt match the records in the database, and then awards points.

[1854] Example prompt

[1855] "Please explain in detail how the server analyzes the receipt information and awards points to the user after the user takes a photo of the receipt and sends it to the server."

[1856] The above is a concrete example of how to implement the present invention. This system allows users to not only earn points but also enjoy personalized services based on their purchasing habits. It also enables companies to effectively utilize purchasing data to strengthen their sales promotion activities.

[1857] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1858] Step 1:

[1859] User

[1860] The user launches the smartphone app and selects the receipt capture or upload function. The user either takes a photo of a paper receipt using the device's camera or selects an electronic receipt from within the device. An image of the receipt is generated as input, which becomes the input data for the next step.

[1861] Specific operation:

[1862] Pressing the "Take a Receipt Photo" button will activate the camera, which will focus on the receipt and take a photo.

[1863] Press the "Upload" button and select the electronic receipt file on your device.

[1864] Step 2:

[1865] Terminal

[1866] The terminal temporarily stores the captured receipt image and then transmits the data to the server. The input is a photographed or uploaded receipt image. The output is image data that is sent to the server.

[1867] Specific operation:

[1868] The captured receipt image is previewed and the user presses the "Send" button.

[1869] The terminal transmits the image data to the server.

[1870] Step 3:

[1871] server

[1872] The server receives receipt image data sent from the terminal. The receipt image data is sent as input. The received image data is temporarily saved as output.

[1873] Specific operation:

[1874] The received image data is saved in a specific directory.

[1875] Step 4:

[1876] server

[1877] The server applies OCR (Optical Character Recognition) processing to the stored image data. Specifically, it uses Tesseract or Google Cloud Vision API to extract text information from the image. The input is the stored receipt image, and the output is the extracted text information.

[1878] Specific operation:

[1879] Start the OCR engine and input the image file.

[1880] Extract information such as product name, price, and purchase date and time.

[1881] Step 5:

[1882] server

[1883] The server uses natural language processing (NLP) to classify the extracted text information and assign specific categories and tags to it. It uses NLP libraries such as Python's NLTK or spaCy. The extracted text information is the input, and the classified text data is generated as the output.

[1884] Specific operation:

[1885] Analyzes text information and classifies product names and prices by category.

[1886] Add tags such as "food" and "daily necessities."

[1887] Step 6:

[1888] server

[1889] The server stores the classified text information in a database, using MySQL or PostgreSQL. The input is the classified text data, and the output is a new record stored in the database.

[1890] Specific operation:

[1891] Information such as user ID, product name, price, purchase date and time, etc. is inserted into the database via an SQL query.

[1892] Step 7:

[1893] server

[1894] The server assigns points to the user's account based on the information stored in the database. The input is the stored receipt information, and the output is the result of the points assignment.

[1895] Specific operation:

[1896] Run the point-granting algorithm and grant 50 points to user ID "12345".

[1897] A record of points awarded is stored in a database.

[1898] Step 8:

[1899] server

[1900] The server generates a notification message to notify the user of the point allocation result and sends it to the user terminal. The input is the point allocation result data, and the output is the generated notification message.

[1901] Specific operation:

[1902] The result data is sent to the terminal in JSON format.

[1903] Generates the message "50 points awarded."

[1904] Step 9:

[1905] Terminal

[1906] The device receives notification messages from the server and displays the notifications within the app. As input, it has the notification message sent by the server and as output, it generates the notification that is displayed to the user.

[1907] Specific operation:

[1908] The notification message is parsed and displayed in the user interface.

[1909] Step 10:

[1910] server

[1911] The server periodically inputs the saved purchase data into an AI model to analyze the user's purchasing trends. The AI ​​model uses TensorFlow and PyTorch. The input is the user's purchase history data, and the output is an analysis of purchasing trends.

[1912] Specific operation:

[1913] Purchase history data is fed into the AI ​​model, which then identifies purchasing patterns.

[1914] The analysis results include "User ID: 12345 frequently purchases food."

[1915] Step 11:

[1916] server

[1917] The server generates personalized advertisements and coupons based on the analysis results and sends them to the user's device. The input is the analysis results of purchasing trends, and the output is the generated advertisement and coupon data.

[1918] Specific operation:

[1919] Run ad generation algorithms to generate appropriate ads and coupons.

[1920] The generated advertisement is transmitted to the terminal.

[1921] Step 12:

[1922] Terminal

[1923] The device receives advertisements and coupon information sent from the server and displays them in a designated area within the app. The input is the advertisement data sent from the server, and the output is the advertisement displayed.

[1924] Specific operation:

[1925] The received advertising data is analyzed and displayed within the app.

[1926] The above are the processing steps of this system. The user, terminal, and server work together to carry out a series of processes to award points and provide personalized services.

[1927] (Application example 1)

[1928] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1929] While previous systems had the ability to use receipt information to award points and provide personalized advertisements and coupons, they lacked real-time support tools to help store associates improve their service. Furthermore, they lacked an immediate and intuitive interface for store associates to provide personalized service to customers. As a result, their effectiveness in improving customer satisfaction and maximizing store sales was limited.

[1930] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1931] In this invention, the server includes a means for acquiring a receipt image, an optical character recognition means for extracting text information from the acquired receipt image, and a means for classifying the extracted text information and assigning specific categories and tags. This enables a means for scanning receipt information and processing the data in real time. The server also includes a means installed in the smart glasses for providing display information when a store clerk provides customer service. This allows the store clerk to provide instant personalized service to customers.

[1932] definition statement

[1933] The "means for acquiring a receipt image" refers to a device such as a camera or scanner for electronically acquiring a receipt for a product purchased by a user.

[1934] An "optical character recognition means" is software or hardware for analyzing and extracting textual information from captured images.

[1935] The "means for assigning specific categories and tags" refers to a system that has the function of classifying extracted text information and automatically assigning appropriate categories and tags.

[1936] A "database storage means" is a data management device or system used to store classified information securely and efficiently.

[1937] The "means for adding points to a user account" is a system that automatically adds points to a user account based on the acquired and classified information.

[1938] "Artificial intelligence means" refers to machine learning models and algorithms used to analyze users' purchasing habits based on stored information.

[1939] The "means for generating personalized advertisements or coupons" is a system that generates advertisements and coupons optimized for each user based on analyzed purchasing data.

[1940] The "means for displaying on the user terminal" is an interface having a function for displaying the generated advertisements and coupons on the user terminal.

[1941] "Means installed on smart glasses to provide display information when store clerks provide customer service" refers to an application installed on smart glasses, which is a system that displays information necessary for store clerks to provide service to customers in real time.

[1942] "Means for scanning receipt information and processing data in real time" refers to hardware and software for quickly recognizing receipt information and immediately digitizing and processing that information.

[1943] MODE FOR CARRYING OUT THE INVENTION

[1944] The present invention provides an electronic payment system that acquires and analyzes receipt information, awards points to users, and provides personalized advertisements and coupons based on the points. The system includes smart glasses, a server, a database, an artificial intelligence model, and related communication means.

[1945] Capture and send receipt images

[1946] Terminal (smart glasses)

[1947] The store clerk puts on the smart glasses, launches the dedicated app, and scans the receipt for the item purchased by the user with the smart glasses' camera.

[1948] The scanned receipt image is temporarily stored in the smart glasses and the data is sent to the server.

[1949] Examples:

[1950] The store clerk presses the "scan" button and scans the receipt with the smart glasses' camera, after which the image data is automatically sent to the server.

[1951] Receipt data processing and analysis

[1952] server

[1953] The server receives the receipt image data sent from the terminal.

[1954] The received image data is processed using OCR (Optical Character Recognition) to extract text information, including the product name, price, purchase date, etc.

[1955] The extracted text information is classified into categories and tags.

[1956] Examples:

[1957] The server processes the receipt image and extracts the text data "Product A - 200 yen," "Product B - 300 yen," and "October 1, 2023." This is then automatically classified into categories such as "food" and "daily necessities."

[1958] Data storage and point allocation

[1959] server

[1960] The classified text information is stored in a database, including the user ID, purchased item, price, purchase date and time, etc.

[1961] After confirming that the receipt has been successfully processed and the data has been saved, points are credited to the user's account.

[1962] The results of point allocation are notified to the smart glasses.

[1963] Terminal (smart glasses)

[1964] The store clerk's smart glasses receive the point award notification from the server and display it as a message, allowing the store clerk to inform the customer that the user has earned points.

[1965] Examples:

[1966] The server awards 50 points to user ID "12345" and notifies the result to the smart glasses, which display the message "50 points have been awarded."

[1967] Data analytics and personalized advertising

[1968] server

[1969] The saved purchasing data is periodically input into the AI ​​model to analyze users' purchasing trends.

[1970] Based on the analysis results, optimal advertisements and coupons are generated for the user and sent to the smart glasses.

[1971] Terminal (smart glasses)

[1972] The smart glasses receive and display advertisements and coupon information sent from the server, and store clerks can use this information to suggest personalized products to customers.

[1973] Examples:

[1974] The server analyzes the user's purchase history and finds...

Claims

1. A means for acquiring a receipt image; Optical character recognition means for extracting text information from the captured receipt image; means for classifying the extracted text information and assigning specific categories and tags; means for storing the classified information in a database; means for awarding points to a user account based on the stored information; an artificial intelligence means for analyzing the user's purchasing trends based on the stored information; means for generating personalized advertisements or coupons for the user based on the analysis results; A system including means for displaying advertisements or coupons on a user terminal.

2. 10. The system of claim 1, further comprising a verification means for verifying the accuracy of the receipt information provided by the user.

3. The system of claim 1 , further comprising means for generating a dashboard for the business to provide analytical data and purchasing trends to the business.

Images