system

Abstract

The system according to this embodiment aims to automate the splitting of meal costs and simplify payment. [Solution] The system according to this embodiment comprises an image analysis unit, a payment calculation unit, and a payment completion unit. The image analysis unit analyzes images of the receipt and the meal. The payment calculation unit calculates the payment amount based on the order details analyzed by the image analysis unit. The payment completion unit performs electronic payment based on the payment amount calculated by the payment calculation unit.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to the description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] The In the conventional technology, it is troublesome to manually calculate the bill splitting and errors are likely to occur, resulting in payment troubles and time-consuming problems.

[0005] The system according to the embodiment aims to automate the bill splitting for meals and enable easy payment.

Means for Solving the Problems

[0006] The system according to the embodiment includes an image analysis unit, a payment calculation unit, and a payment completion unit. The image analysis unit analyzes the receipt and the image of the meal. The payment calculation unit calculates the payment amount based on the order content analyzed by the image analysis unit. The payment completion unit performs electronic payment based on the payment amount calculated by the payment calculation unit. [Effects of the Invention]

[0007] The system according to this embodiment can automate the splitting of the bill for meals, making payment easier. [Brief explanation of the drawing]

[0008] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Modes for carrying out the invention]

[0009] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0010] First, let's explain the terminology used in the following explanation.

[0011] In the following embodiments, the signed processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), APU (Accelerated Processing Unit), or TPU (Tensor Processing Unit).

[0012] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0013] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0014] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B". That is, "A and / or B" means that it may be only A, only B, or a combination of A and B. Also, in this specification, when expressing three or more matters connected by "and / or", the same concept as "A and / or B" is applied.

[0016] [First Embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] As shown in FIG. 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. Also, the database 24 and the communication I / F 26 are connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. Also, the reception device 38, the output device 40, and the camera 42 are connected to the bus 52.

[0020] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, and accepts user input. The touch panel 38A accepts user input via touch by detecting contact with an object (e.g., a pen or finger). The microphone 38B accepts user input via voice by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 (see Figure 2) acquires the data indicating the user input.

[0021] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user by outputting the data in a form perceptible to the user (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0022] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0023] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0024] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0025] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0026] In the smart device 14, specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as a control unit 46A according to the specific processing program 60 executed on the RAM 48. The smart device 14 also has a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0027] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device having the data generation model 58. The data processing device 12 may also be a server device or a terminal device owned by a user (e.g., a mobile phone, robot, home appliance, etc.). Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example of form 1) The Smart Bill-Splitting Assistant System according to an embodiment of the present invention is a system that uses AI to automate the splitting of meal bills and supports users in easily paying for meals. The Smart Bill-Splitting Assistant System uses AI to analyze images of receipts and meals and automatically recognize who ordered which menu item. For example, an image of the receipt is taken, and the AI ​​analyzes its contents to identify each person's order. It also analyzes images of the meals to supplement the order details and achieve more accurate recognition. Next, based on each person's order, the system automatically calculates the individual payment amount. The AI ​​matches the order contents and prices to calculate the amount each person should pay. For example, if person A orders a salad and pasta, and person B orders a steak, the system automatically calculates the respective amounts. Furthermore, based on the calculated payment amount, the system allows for easy payment completion using an electronic payment system. Users can complete the bill splitting effortlessly simply by making a payment through the app. For example, each person's payment amount is displayed, and payment via electronic payment is completed with a single click. This mechanism eliminates the need to manually calculate the split bill and reduces the likelihood of errors. Furthermore, it clarifies who should pay how much, preventing payment disputes. Additionally, it shortens the time required for payment, reducing inconvenience to the restaurant. For example, at a meal with friends, the AI ​​automatically recognizes each person's order and calculates the payment amount, significantly reducing the hassle of splitting the bill. Using electronic payment also ensures smooth transactions and prevents problems. In this way, the AI-powered smart bill-splitting assistant automates the process of splitting the bill, saving users time and making payments easy. This results in smoother payment processes and increased user satisfaction. Thus, the smart bill-splitting assistant system can automate the process of splitting the bill and support users in making payments easy.

[0029] The smart bill-splitting assistant system according to this embodiment comprises an image analysis unit, a payment calculation unit, and a payment completion unit. The image analysis unit analyzes images of receipts and meals. For example, the image analysis unit takes a picture of a receipt, and the AI ​​analyzes its contents to identify each person's order. The image analysis unit can also analyze images of meals to supplement the order details and achieve more accurate recognition. For example, the image analysis unit uses OCR technology to analyze the text information on the receipt to identify the order details. Furthermore, the image analysis unit can analyze images of meals to recognize the type and quantity of dishes. The payment calculation unit calculates the payment amount based on the order details analyzed by the image analysis unit. For example, the payment calculation unit matches the order details with the prices and calculates the amount each person should pay. For example, the payment calculation unit calculates the amount corresponding to each person's order based on the price information on the receipt. The payment calculation unit can also match the prices of dishes recognized from the meal images and calculate the payment amount. The payment completion unit performs electronic payment based on the payment amount calculated by the payment calculation unit. The payment completion unit completes the payment, for example, using an electronic payment system. For example, the payment completion unit can smoothly process payments using credit card payments or mobile payments. The payment completion unit can also complete payments using electronic money. As a result, the smart bill-splitting assistant system according to this embodiment can automate the splitting of meal bills and support users in easily making payments.

[0030] The image analysis unit analyzes images of receipts and meals. For example, the image analysis unit takes a picture of a receipt, and the AI ​​analyzes its contents to identify each person's order. Specifically, it takes a high-resolution image of the receipt and extracts text information using OCR (Optical Character Recognition) technology. This OCR technology uses a highly trained AI model that can handle differences in receipt format and font. The AI ​​analyzes the extracted text information and identifies information such as product name, quantity, and price to identify each person's order. In addition, the image analysis unit can supplement the order details and achieve more accurate recognition by analyzing images of meals. For example, it takes a picture of a meal, and the AI ​​uses image recognition technology to identify the type and quantity of the dish. The AI ​​has been trained in advance with a large dataset of food images and can identify the characteristics of dishes with high accuracy. This allows it to supplement information such as additional orders or shared dishes that are not listed on the receipt. Furthermore, the image analysis unit analyzes the text information of the receipt using OCR technology to identify the order details. For example, when analyzing text information on a receipt, the AI ​​performs contextual analysis to prevent misrecognition and corrects errors in product names and prices. Additionally, the image analysis unit can analyze images of meals to recognize the type and quantity of dishes. This improves the accuracy of order details and enhances the precision of data entered into the payment calculation unit.

[0031] The payment calculation unit calculates the payment amount based on the order details analyzed by the image analysis unit. For example, the payment calculation unit matches the order details with the prices and calculates the amount each person should pay. Specifically, the payment calculation unit calculates the amount corresponding to each person's order based on the price information on the receipt. The AI ​​uses multiple algorithms to match the order details with the price information and accurately calculate each person's payment amount. For example, if the order is shared among multiple people, the AI ​​calculates the proportion and fairly distributes the payment amount to each person. The payment calculation unit can also match the prices of dishes recognized from images of food and calculate the payment amount. For example, based on the type and quantity of dishes recognized from images of food, the AI ​​retrieves the price from a database and calculates each person's payment amount. Furthermore, the payment calculation unit can also take into account the application of discounts and coupons. For example, if a discount or coupon applies to a particular dish, the AI ​​takes that information into account and adjusts the final payment amount. This allows the payment calculation unit to calculate accurate and fair payment amounts and support users in making payments easily.

[0032] The payment completion unit performs electronic payment based on the payment amount calculated by the payment calculation unit. The payment completion unit completes the payment using, for example, an electronic payment system. Specifically, it can smoothly process payments using credit card payments or mobile payments. For example, when a user enters their credit card information, the payment completion unit securely processes that information and completes the payment. The payment completion unit can also complete payments using electronic money. For example, if a user has an electronic money account, the payment completion unit uses that account information to make the payment. Furthermore, the payment completion unit supports multiple payment methods, allowing users to pay in the most convenient way. For example, it improves user convenience by offering various payment methods such as QR code (registered trademark) payment and bank transfer. The payment completion unit also implements encryption technology and authentication processes to ensure payment security. This protects users' personal and payment information, allowing them to pay with peace of mind. Finally, the payment completion unit sends a notification to the user after payment is completed to confirm the payment. For example, it sends an email or SMS indicating that the payment has been completed, allowing the user to check the payment status. This allows the payment completion section to support users in making payments easily and securely, improving the convenience of the smart bill-splitting assistant system.

[0033] The image analysis unit includes a receipt analysis unit that analyzes the image of the receipt and identifies the order details. The receipt analysis unit, for example, analyzes the text information of the receipt using OCR technology to identify the order details. The receipt analysis unit can, for example, recognize the format of the receipt and identify the position of each item. The receipt analysis unit can, for example, convert the text information of the receipt into digital data and extract the order details. This improves the accuracy of calculating the payment amount by analyzing the image of the receipt to identify the order details. Some or all of the above processing in the receipt analysis unit may be performed using AI, for example, or without AI. For example, the receipt analysis unit can input the image data of the receipt into a generating AI and have the generating AI perform the analysis of the text information of the receipt.

[0034] The image analysis unit includes a food image analysis unit that analyzes images of meals and supplements the order details. The food image analysis unit, for example, analyzes images of meals and recognizes the type and quantity of dishes. The food image analysis unit can identify the type of dish using image recognition technology, for example. The food image analysis unit can measure the quantity of dishes from the meal images and supplement the order details. This makes it possible to calculate the payment amount more accurately by analyzing the meal images and supplementing the order details. Some or all of the above processing in the food image analysis unit may be performed using AI, for example, or without AI. For example, the food image analysis unit can input meal image data into a generating AI and have the generating AI perform the recognition of the type and quantity of dishes.

[0035] The payment calculation unit includes an order content matching unit that matches order details with prices. The order content matching unit, for example, matches order details with prices and calculates the amount each person should pay. The order content matching unit can, for example, calculate the amount corresponding to each person's order based on price information from a receipt. The order content matching unit can, for example, match the prices of dishes recognized from images of meals and calculate the payment amount. This makes it possible to calculate the payment amount accurately by matching order details with prices. Some or all of the above processing in the order content matching unit may be performed using AI, for example, or without AI. For example, the order content matching unit can input order details and price data into a generating AI and have the generating AI perform price matching.

[0036] The payment completion unit includes an electronic payment unit that completes payments using an electronic payment system. The electronic payment unit facilitates payments smoothly, for example, by using credit card payments or mobile payments. The electronic payment unit can complete payments using, for example, electronic money. The electronic payment unit can easily make payments using, for example, QR code payments. This ensures smooth payments by completing them using an electronic payment system. Some or all of the above-described processes in the electronic payment unit may be performed using, for example, AI, or not using AI. For example, the electronic payment unit can input payment amount data into a generating AI and have the generating AI execute the electronic payment procedure.

[0037] The image analysis unit improves analysis accuracy by integrating images taken under different lighting conditions and angles. For example, if an image of a receipt is dark, the image analysis unit can automatically adjust the brightness before analysis. For example, if an image of a meal is taken from an oblique angle, the image analysis unit can perform angle correction before analysis. For example, the image analysis unit can integrate multiple images, extract the clearest part, and perform analysis. This improves analysis accuracy by integrating images taken under different lighting conditions and angles. Some or all of the above processing in the image analysis unit may be performed using AI, for example, or without AI. For example, the image analysis unit can input image data from different lighting conditions and angles into a generating AI and have the generating AI perform image integration.

[0038] The image analysis unit optimizes the analysis algorithm by referring to past analysis results. For example, the image analysis unit can quickly analyze receipts of a similar format based on data from receipts that have been analyzed in the past. For example, the image analysis unit can quickly recognize similar dishes based on the analysis results of past food images. For example, the image analysis unit can learn from past analysis results and continuously improve its analysis accuracy. This makes it possible to optimize the analysis algorithm by referring to past analysis results. Some or all of the above processes in the image analysis unit may be performed using AI, for example, or without AI. For example, the image analysis unit can input past analysis result data into a generating AI and have the generating AI perform the optimization of the analysis algorithm.

[0039] The image analysis unit selects the optimal analysis method based on the user's device information. For example, if the user is using a smartphone, the image analysis unit can provide an analysis method optimized for smartphones. For example, if the user is using a tablet, the image analysis unit can provide an analysis method optimized for large screens. For example, if the user is using a desktop, the image analysis unit can provide high-resolution image analysis. This improves analysis accuracy by selecting the optimal analysis method based on the user's device information. Some or all of the above processing in the image analysis unit may be performed using AI, for example, or without AI. For example, the image analysis unit can input the user's device information into a generating AI and have the generating AI select the optimal analysis method.

[0040] The image analysis unit analyzes the user's social media activity and prioritizes the analysis of relevant images. For example, the image analysis unit prioritizes the analysis of images posted by the user on social media. For example, if the user has many followers on social media, the image analysis unit can prioritize the analysis of those images. For example, the image analysis unit can prioritize the analysis of relevant images based on the user's social media activity frequency. This improves the accuracy of the analysis by prioritizing the analysis of relevant images based on the user's social media activity. Some or all of the above processing in the image analysis unit may be performed using AI, for example, or without AI. For example, the image analysis unit can input the user's social media activity data into a generating AI and have the generating AI perform the priority analysis of relevant images.

[0041] The payment calculation unit selects the optimal calculation method by referring to past payment history. The payment calculation unit provides the optimal calculation method based on, for example, payment methods previously used by the user. The payment calculation unit can provide the most efficient calculation method based on, for example, the user's past payment history. The payment calculation unit can analyze, for example, the user's past payment history and provide the optimal method for splitting the bill. This allows the optimal calculation method to be selected by referring to past payment history. Some or all of the above processes in the payment calculation unit may be performed using, for example, AI, or not using AI. For example, the payment calculation unit can input past payment history data into a generating AI and have the generating AI perform the selection of the optimal calculation method.

[0042] The payment calculation unit improves calculation accuracy by considering different currencies and tax rates. For example, the payment calculation unit considers payments in different currencies and automatically applies exchange rates. For example, the payment calculation unit can calculate the correct payment amount by considering different tax rates. For example, the payment calculation unit can integrate multiple currencies and tax rates to provide the optimal payment amount. This improves calculation accuracy by considering different currencies and tax rates. Some or all of the above processes in the payment calculation unit may be performed using AI, for example, or not using AI. For example, the payment calculation unit can input data on different currencies and tax rates into a generating AI and have the generating AI perform the calculation accuracy improvement.

[0043] The payment calculation unit selects the optimal calculation method based on the user's geographical location information. For example, if the user is in a different region, the payment calculation unit will take into account the tax rate of that region. For example, if the user is traveling, the payment calculation unit can take into account payment in the local currency. For example, the payment calculation unit can provide the optimal payment method based on the user's geographical location information. This improves calculation accuracy by selecting the optimal calculation method based on the user's geographical location information. Some or all of the above processing in the payment calculation unit may be performed using AI, for example, or without AI. For example, the payment calculation unit can input the user's geographical location information into a generating AI and have the generating AI select the optimal calculation method.

[0044] The payment calculation unit analyzes the user's social media activity and prioritizes calculating relevant payment information. For example, the payment calculation unit provides the optimal calculation method based on payment information shared by the user on social media. For example, if a user has many followers on social media, the payment calculation unit can prioritize calculating that payment information. For example, the payment calculation unit can prioritize calculating relevant payment information based on the frequency of the user's social media activity. This improves calculation accuracy by prioritizing the calculation of relevant payment information based on the user's social media activity. Some or all of the above processing in the payment calculation unit may be performed using AI, for example, or without AI. For example, the payment calculation unit can input the user's social media activity data into a generating AI and have the generating AI perform the priority calculation of relevant payment information.

[0045] The payment completion unit selects the optimal completion method by referring to past payment history. The payment completion unit provides the optimal completion method based on, for example, payment methods previously used by the user. The payment completion unit can provide the most efficient completion method based on, for example, the user's past payment history. The payment completion unit can provide the optimal payment completion method by, for example, analyzing the user's past payment history. This allows the optimal completion method to be selected by referring to past payment history. Some or all of the above processing in the payment completion unit may be performed using, for example, AI, or not using AI. For example, the payment completion unit can input past payment history data into a generating AI and have the generating AI perform the selection of the optimal completion method.

[0046] The payment completion unit improves completion accuracy by integrating different payment methods. For example, the payment completion unit can complete a payment by integrating credit cards and electronic money. For example, the payment completion unit can complete a payment by integrating different electronic payment systems. For example, the payment completion unit can provide the optimal payment completion method by integrating multiple payment methods. This improves the accuracy of payment completion by integrating different payment methods. Some or all of the above processing in the payment completion unit may be performed using AI, for example, or without AI. For example, the payment completion unit can input data from different payment methods into a generating AI and have the generating AI perform the task of improving completion accuracy.

[0047] The payment completion unit selects the optimal completion method based on the user's device information. For example, if the user is using a smartphone, the payment completion unit can provide a completion method optimized for smartphones. For example, if the user is using a tablet, the payment completion unit can provide a completion method optimized for large screens. For example, if the user is using a desktop, the payment completion unit can provide a high-resolution completion method. This improves the accuracy of payment completion by selecting the optimal completion method based on the user's device information. Some or all of the above processing in the payment completion unit may be performed using AI, for example, or without AI. For example, the payment completion unit can input the user's device information into a generating AI and have the generating AI select the optimal completion method.

[0048] The payment completion unit analyzes the user's social media activity and prioritizes the completion of relevant payment information. For example, the payment completion unit provides the optimal completion method based on payment information shared by the user on social media. For example, if the user has many followers on social media, the payment completion unit can prioritize the completion of that payment information. For example, the payment completion unit can prioritize the completion of relevant payment information based on the frequency of the user's social media activity. This improves the accuracy of payment completion by prioritizing the completion of relevant payment information based on the user's social media activity. Some or all of the above processing in the payment completion unit may be performed using AI, for example, or without AI. For example, the payment completion unit can input the user's social media activity data into a generating AI and have the generating AI prioritize the completion of relevant payment information.

[0049] The receipt analysis unit improves analysis accuracy by integrating receipts of different formats. For example, the receipt analysis unit can automatically recognize and analyze receipt formats from different stores. For example, even if the receipt formats are different, the receipt analysis unit can extract common items and perform analysis. For example, the receipt analysis unit can integrate multiple receipts and analyze the overall payment information. This improves analysis accuracy by integrating receipts of different formats. Some or all of the above processes in the receipt analysis unit may be performed using AI, for example, or without AI. For example, the receipt analysis unit can input receipt data of different formats into a generating AI and have the generating AI perform integrated receipt analysis.

[0050] The receipt analysis unit optimizes its analysis algorithm by referring to past analysis results. For example, the receipt analysis unit can quickly analyze receipts of a similar format based on data from receipts that have been analyzed in the past. For example, the receipt analysis unit can quickly recognize similar items based on past receipt analysis results. For example, the receipt analysis unit can learn from past analysis results and continuously improve its analysis accuracy. This makes it possible to optimize the analysis algorithm by referring to past analysis results. Some or all of the above processes in the receipt analysis unit may be performed using AI, for example, or without AI. For example, the receipt analysis unit can input past analysis result data into a generating AI and have the generating AI perform the optimization of the analysis algorithm.

[0051] The receipt analysis unit selects the optimal analysis method based on the user's device information. For example, if the user is using a smartphone, the receipt analysis unit can provide an analysis method optimized for smartphones. For example, if the user is using a tablet, the receipt analysis unit can provide an analysis method optimized for larger screens. For example, if the user is using a desktop computer, the receipt analysis unit can provide high-resolution image analysis. This improves analysis accuracy by selecting the optimal analysis method based on the user's device information. Some or all of the above processing in the receipt analysis unit may be performed using AI, for example, or without AI. For example, the receipt analysis unit can input the user's device information into a generating AI and have the generating AI select the optimal analysis method.

[0052] The receipt analysis unit analyzes the user's social media activity and prioritizes the analysis of relevant receipts. For example, the receipt analysis unit prioritizes the analysis of receipt images posted by the user on social media. For example, the receipt analysis unit can prioritize the analysis of receipts if the user has a large number of social media followers. For example, the receipt analysis unit can prioritize the analysis of relevant receipts based on the user's social media activity frequency. This improves the accuracy of the analysis by prioritizing the analysis of relevant receipts based on the user's social media activity. Some or all of the above processing in the receipt analysis unit may be performed using AI, for example, or without AI. For example, the receipt analysis unit can input the user's social media activity data into a generating AI and have the generating AI perform the priority analysis of relevant receipts.

[0053] The food image analysis unit improves analysis accuracy by integrating images taken under different lighting conditions and angles. For example, if a food image is dark, the food image analysis unit automatically adjusts the brightness before analysis. For example, if a food image is taken from an oblique angle, the food image analysis unit can perform angle correction before analysis. For example, the food image analysis unit can integrate multiple images, extract the clearest part, and perform analysis. This improves analysis accuracy by integrating images taken under different lighting conditions and angles. Some or all of the above processing in the food image analysis unit may be performed using AI, for example, or without AI. For example, the food image analysis unit can input image data from different lighting conditions and angles into a generating AI and have the generating AI perform image integration.

[0054] The food image analysis unit optimizes its analysis algorithm by referring to past analysis results. For example, the food image analysis unit can quickly analyze similar dishes based on data from previously analyzed meals. For example, the food image analysis unit can quickly recognize similar dishes based on the analysis results of past meal images. For example, the food image analysis unit can learn from past analysis results and continuously improve its analysis accuracy. This makes it possible to optimize the analysis algorithm by referring to past analysis results. Some or all of the above processes in the food image analysis unit may be performed using AI, for example, or without AI. For example, the food image analysis unit can input past analysis result data into a generating AI and have the generating AI perform the optimization of the analysis algorithm.

[0055] The food image analysis unit selects the optimal analysis method based on the user's device information. For example, if the user is using a smartphone, the food image analysis unit can provide an analysis method optimized for smartphones. For example, if the user is using a tablet, the food image analysis unit can provide an analysis method optimized for larger screens. For example, if the user is using a desktop computer, the food image analysis unit can provide high-resolution image analysis. This improves analysis accuracy by selecting the optimal analysis method based on the user's device information. Some or all of the above processing in the food image analysis unit may be performed using AI, for example, or without AI. For example, the food image analysis unit can input the user's device information into a generating AI and have the generating AI select the optimal analysis method.

[0056] The food image analysis unit analyzes the user's social media activity and prioritizes the analysis of relevant food images. For example, the food image analysis unit prioritizes the analysis of food images posted by the user on social media. For example, if the user has many followers on social media, the food image analysis unit can prioritize the analysis of those food images. For example, the food image analysis unit can prioritize the analysis of relevant food images based on the user's social media activity frequency. This improves the accuracy of the analysis by prioritizing the analysis of relevant food images based on the user's social media activity. Some or all of the above processing in the food image analysis unit may be performed using AI, for example, or without AI. For example, the food image analysis unit can input the user's social media activity data into a generating AI and have the generating AI perform the priority analysis of relevant food images.

[0057] The order matching unit improves matching accuracy by referring to past order history. The order matching unit can quickly match similar orders based on past orders, for example. The order matching unit can quickly match similar items based on past order history, for example. The order matching unit can learn from past order history and continuously improve matching accuracy, for example. This improves matching accuracy by referring to past order history. Some or all of the above processing in the order matching unit may be performed using AI, for example, or without AI. For example, the order matching unit can input past order history data into a generating AI and have the generating AI perform the improvement of matching accuracy.

[0058] The order content matching unit improves matching accuracy by integrating order content in different formats. For example, the order content matching unit can automatically recognize and match order formats from different stores. For example, even if the order content formats are different, the order content matching unit can extract common items and perform matching. For example, the order content matching unit can integrate multiple order contents and match the overall order information. This improves matching accuracy by integrating order content in different formats. Some or all of the above processing in the order content matching unit may be performed using AI, for example, or without AI. For example, the order content matching unit can input order content data in different formats into a generating AI and have the generating AI perform integrated matching of the order content.

[0059] The order matching unit selects the optimal matching method based on the user's device information. For example, if the user is using a smartphone, the order matching unit can provide a matching method optimized for smartphones. For example, if the user is using a tablet, the order matching unit can provide a matching method optimized for larger screens. For example, if the user is using a desktop, the order matching unit can provide a high-resolution matching method. This improves matching accuracy by selecting the optimal matching method based on the user's device information. Some or all of the above processing in the order matching unit may be performed using AI, for example, or without AI. For example, the order matching unit can input the user's device information into a generating AI and have the generating AI select the optimal matching method.

[0060] The order matching unit analyzes the user's social media activity and prioritizes matching relevant order content. For example, the order matching unit prioritizes matching order content posted by the user on social media. For example, the order matching unit can prioritize matching order content if the user has many followers on social media. For example, the order matching unit can prioritize matching relevant order content based on the frequency of the user's social media activity. This improves matching accuracy by prioritizing the matching of relevant order content based on the user's social media activity. Some or all of the above processing in the order matching unit may be performed using AI, for example, or without AI. For example, the order matching unit can input the user's social media activity data into a generating AI and have the generating AI perform priority matching of relevant order content.

[0061] The electronic payment unit selects the optimal payment method by referring to past payment history. The electronic payment unit provides the optimal payment method based on the payment methods the user has used in the past, for example. The electronic payment unit can provide the most efficient payment method based on the user's past payment history, for example. The electronic payment unit can provide the optimal payment method by analyzing the user's past payment history, for example. This allows the optimal payment method to be selected by referring to past payment history. Some or all of the above processes in the electronic payment unit may be performed using AI, for example, or without AI. For example, the electronic payment unit can input past payment history data into a generating AI and have the generating AI perform the selection of the optimal payment method.

[0062] The electronic payment unit improves payment accuracy by integrating different payment methods. For example, the electronic payment unit can complete payments by integrating credit cards and electronic money. For example, the electronic payment unit can complete payments by integrating different electronic payment systems. For example, the electronic payment unit can provide the optimal payment method by integrating multiple payment methods. This improves payment accuracy by integrating different payment methods. Some or all of the above processes in the electronic payment unit may be performed using AI, for example, or without AI. For example, the electronic payment unit can input data from different payment methods into a generating AI and have the generating AI perform the improvement of payment accuracy.

[0063] The electronic payment unit selects the optimal payment method based on the user's device information. For example, if the user is using a smartphone, the electronic payment unit can provide a payment method optimized for smartphones. For example, if the user is using a tablet, the electronic payment unit can provide a payment method optimized for larger screens. For example, if the user is using a desktop computer, the electronic payment unit can provide a high-resolution payment method. This improves payment accuracy by selecting the optimal payment method based on the user's device information. Some or all of the above processing in the electronic payment unit may be performed using AI, for example, or without AI. For example, the electronic payment unit can input the user's device information into a generating AI and have the generating AI select the optimal payment method.

[0064] The electronic payment unit analyzes the user's social media activity and prioritizes the processing of relevant payment information. For example, the electronic payment unit provides the optimal payment method based on payment information shared by the user on social media. For example, if a user has many followers on social media, the electronic payment unit can prioritize the processing of that payment information. For example, the electronic payment unit can prioritize the processing of relevant payment information based on the user's frequency of social media activity. This improves payment accuracy by prioritizing the processing of relevant payment information based on the user's social media activity. Some or all of the above processing in the electronic payment unit may be performed using AI, for example, or without AI. For example, the electronic payment unit can input the user's social media activity data into a generating AI and have the generating AI perform the priority processing of relevant payment information.

[0065] The system according to the embodiment is not limited to the example described above, and various modifications are possible, for example, as follows.

[0066] The smart split-the-bill assistant system can suggest the most suitable payment method by referring to the user's past payment history. For example, if the user has frequently used credit cards in the past, the system will prioritize suggesting credit card payment. If the user frequently uses electronic money, it can suggest electronic money payment. Furthermore, if the user has a history of frequent payments at a particular store, it can suggest the most suitable payment method for that store. In this way, by suggesting the most suitable payment method based on the user's past payment history, the system can improve the convenience of payments.

[0067] The smart split-the-bill assistant system can suggest the optimal payment method based on the user's geographical location. For example, if the user is in a different region, it can suggest a payment method that takes into account the local currency and tax rate. If the user is traveling, it can suggest a payment method in the local currency. Furthermore, if the user frequently makes payments in a particular region, it can suggest the optimal payment method for that region. In this way, by suggesting the optimal payment method based on the user's geographical location, the system can improve the convenience of payments.

[0068] The smart bill-splitting assistant system can suggest the optimal payment method based on the user's device information. For example, if the user is using a smartphone, it can suggest a payment method optimized for smartphones. If the user is using a tablet, it can suggest a payment method optimized for larger screens. Furthermore, if the user is using a desktop computer, it can suggest a high-resolution payment method. This improves the convenience of payments by suggesting the most suitable payment method based on the user's device information.

[0069] The smart split-the-bill assistant system can analyze a user's social media activity and prioritize suggesting relevant payment information. For example, it can suggest the best payment method based on payment information shared by the user on social media. If a user has many followers on social media, it can prioritize suggesting payment information related to that account. It can also prioritize suggesting relevant payment information based on the user's social media activity. In this way, by prioritizing the suggestion of relevant payment information based on the user's social media activity, it can improve the convenience of payments.

[0070] The following briefly describes the processing flow for example form 1.

[0071] Step 1: The image analysis unit analyzes the receipt and the food images. The image analysis unit takes a picture of the receipt, and the AI ​​analyzes its contents to identify each person's order. The image analysis unit can also analyze the food images to supplement the order details and achieve more accurate recognition. For example, the image analysis unit uses OCR technology to analyze the text information on the receipt to identify the order details. Furthermore, the image analysis unit can analyze the food images to recognize the type and quantity of dishes. Step 2: The payment calculation unit calculates the payment amount based on the order details analyzed by the image analysis unit. The payment calculation unit matches the order details with the prices and calculates the amount each person should pay. For example, it calculates the amount corresponding to each person's order based on the price information on the receipt. It can also match the prices of the dishes recognized from the image of the food and calculate the payment amount. Step 3: The payment completion unit performs electronic payment based on the payment amount calculated by the payment calculation unit. The payment completion unit completes the payment using the electronic payment system. For example, payments can be made smoothly using credit card payments or mobile payments. Payments can also be completed using electronic money.

[0072] (Example of form 2) The Smart Bill-Splitting Assistant System according to an embodiment of the present invention is a system that uses AI to automate the splitting of meal bills and supports users in easily paying for meals. The Smart Bill-Splitting Assistant System uses AI to analyze images of receipts and meals and automatically recognize who ordered which menu item. For example, an image of the receipt is taken, and the AI ​​analyzes its contents to identify each person's order. It also analyzes images of the meals to supplement the order details and achieve more accurate recognition. Next, based on each person's order, the system automatically calculates the individual payment amount. The AI ​​matches the order contents and prices to calculate the amount each person should pay. For example, if person A orders a salad and pasta, and person B orders a steak, the system automatically calculates the respective amounts. Furthermore, based on the calculated payment amount, the system allows for easy payment completion using an electronic payment system. Users can complete the bill splitting effortlessly simply by making a payment through the app. For example, each person's payment amount is displayed, and payment via electronic payment is completed with a single click. This mechanism eliminates the need to manually calculate the split bill and reduces the likelihood of errors. Furthermore, it clarifies who should pay how much, preventing payment disputes. Additionally, it shortens the time required for payment, reducing inconvenience to the restaurant. For example, at a meal with friends, the AI ​​automatically recognizes each person's order and calculates the payment amount, significantly reducing the hassle of splitting the bill. Using electronic payment also ensures smooth transactions and prevents problems. In this way, the AI-powered smart bill-splitting assistant automates the process of splitting the bill, saving users time and making payments easy. This results in smoother payment processes and increased user satisfaction. Thus, the smart bill-splitting assistant system can automate the process of splitting the bill and support users in making payments easy.

[0073] The smart bill-splitting assistant system according to this embodiment comprises an image analysis unit, a payment calculation unit, and a payment completion unit. The image analysis unit analyzes images of receipts and meals. For example, the image analysis unit takes a picture of a receipt, and the AI ​​analyzes its contents to identify each person's order. The image analysis unit can also analyze images of meals to supplement the order details and achieve more accurate recognition. For example, the image analysis unit uses OCR technology to analyze the text information on the receipt to identify the order details. Furthermore, the image analysis unit can analyze images of meals to recognize the type and quantity of dishes. The payment calculation unit calculates the payment amount based on the order details analyzed by the image analysis unit. For example, the payment calculation unit matches the order details with the prices and calculates the amount each person should pay. For example, the payment calculation unit calculates the amount corresponding to each person's order based on the price information on the receipt. The payment calculation unit can also match the prices of dishes recognized from the meal images and calculate the payment amount. The payment completion unit performs electronic payment based on the payment amount calculated by the payment calculation unit. The payment completion unit completes the payment, for example, using an electronic payment system. For example, the payment completion unit can smoothly process payments using credit card payments or mobile payments. The payment completion unit can also complete payments using electronic money. As a result, the smart bill-splitting assistant system according to this embodiment can automate the splitting of meal bills and support users in easily making payments.

[0074] The image analysis unit analyzes images of receipts and meals. For example, the image analysis unit takes a picture of a receipt, and the AI ​​analyzes its contents to identify each person's order. Specifically, it takes a high-resolution image of the receipt and extracts text information using OCR (Optical Character Recognition) technology. This OCR technology uses a highly trained AI model that can handle differences in receipt format and font. The AI ​​analyzes the extracted text information and identifies information such as product name, quantity, and price to identify each person's order. In addition, the image analysis unit can supplement the order details and achieve more accurate recognition by analyzing images of meals. For example, it takes a picture of a meal, and the AI ​​uses image recognition technology to identify the type and quantity of the dish. The AI ​​has been trained in advance with a large dataset of food images and can identify the characteristics of dishes with high accuracy. This allows it to supplement information such as additional orders or shared dishes that are not listed on the receipt. Furthermore, the image analysis unit analyzes the text information of the receipt using OCR technology to identify the order details. For example, when analyzing text information on a receipt, the AI ​​performs contextual analysis to prevent misrecognition and corrects errors in product names and prices. Additionally, the image analysis unit can analyze images of meals to recognize the type and quantity of dishes. This improves the accuracy of order details and enhances the precision of data entered into the payment calculation unit.

[0075] The payment calculation unit calculates the payment amount based on the order details analyzed by the image analysis unit. For example, the payment calculation unit matches the order details with the prices and calculates the amount each person should pay. Specifically, the payment calculation unit calculates the amount corresponding to each person's order based on the price information on the receipt. The AI ​​uses multiple algorithms to match the order details with the price information and accurately calculate each person's payment amount. For example, if the order is shared among multiple people, the AI ​​calculates the proportion and fairly distributes the payment amount to each person. The payment calculation unit can also match the prices of dishes recognized from images of food and calculate the payment amount. For example, based on the type and quantity of dishes recognized from images of food, the AI ​​retrieves the price from a database and calculates each person's payment amount. Furthermore, the payment calculation unit can also take into account the application of discounts and coupons. For example, if a discount or coupon applies to a particular dish, the AI ​​takes that information into account and adjusts the final payment amount. This allows the payment calculation unit to calculate accurate and fair payment amounts and support users in making payments easily.

[0076] The payment completion unit performs electronic payment based on the payment amount calculated by the payment calculation unit. The payment completion unit completes the payment using, for example, an electronic payment system. Specifically, the payment completion unit can smoothly process payments using credit card payments or mobile payments. For example, when a user enters their credit card information, the payment completion unit securely processes that information and completes the payment. The payment completion unit can also complete payments using electronic money. For example, if a user has an electronic money account, the payment completion unit uses that account information to make the payment. Furthermore, the payment completion unit supports multiple payment methods, allowing users to pay in the most convenient way. For example, by offering various payment methods such as QR code payments and bank transfers, it improves user convenience. The payment completion unit also implements encryption technology and authentication processes to ensure payment security. This protects users' personal and payment information, allowing them to pay with peace of mind. Finally, the payment completion unit sends a notification to the user after payment is completed to confirm the payment. For example, it sends an email or SMS indicating that the payment has been completed, allowing the user to check the payment status. This allows the payment completion section to support users in making payments easily and securely, improving the convenience of the smart bill-splitting assistant system.

[0077] The image analysis unit includes a receipt analysis unit that analyzes the image of the receipt and identifies the order details. The receipt analysis unit, for example, analyzes the text information of the receipt using OCR technology to identify the order details. The receipt analysis unit can, for example, recognize the format of the receipt and identify the position of each item. The receipt analysis unit can, for example, convert the text information of the receipt into digital data and extract the order details. This improves the accuracy of calculating the payment amount by analyzing the image of the receipt to identify the order details. Some or all of the above processing in the receipt analysis unit may be performed using AI, for example, or without AI. For example, the receipt analysis unit can input the image data of the receipt into a generating AI and have the generating AI perform the analysis of the text information of the receipt.

[0078] The image analysis unit includes a food image analysis unit that analyzes images of meals and supplements the order details. The food image analysis unit, for example, analyzes images of meals and recognizes the type and quantity of dishes. The food image analysis unit can identify the type of dish using image recognition technology, for example. The food image analysis unit can measure the quantity of dishes from the meal images and supplement the order details. This makes it possible to calculate the payment amount more accurately by analyzing the meal images and supplementing the order details. Some or all of the above processing in the food image analysis unit may be performed using AI, for example, or without AI. For example, the food image analysis unit can input meal image data into a generating AI and have the generating AI perform the recognition of the type and quantity of dishes.

[0079] The payment calculation unit includes an order content matching unit that matches order details with prices. The order content matching unit, for example, matches order details with prices and calculates the amount each person should pay. The order content matching unit can, for example, calculate the amount corresponding to each person's order based on price information from a receipt. The order content matching unit can, for example, match the prices of dishes recognized from images of meals and calculate the payment amount. This makes it possible to calculate the payment amount accurately by matching order details with prices. Some or all of the above processing in the order content matching unit may be performed using AI, for example, or without AI. For example, the order content matching unit can input order details and price data into a generating AI and have the generating AI perform price matching.

[0080] The payment completion unit includes an electronic payment unit that completes payments using an electronic payment system. The electronic payment unit facilitates payments smoothly, for example, by using credit card payments or mobile payments. The electronic payment unit can complete payments using, for example, electronic money. The electronic payment unit can easily make payments using, for example, QR code payments. This ensures smooth payments by completing them using an electronic payment system. Some or all of the above-described processes in the electronic payment unit may be performed using, for example, AI, or not using AI. For example, the electronic payment unit can input payment amount data into a generating AI and have the generating AI execute the electronic payment procedure.

[0081] The image analysis unit estimates the user's emotions and adjusts the accuracy of the image analysis based on the estimated emotions. For example, if the user is stressed, the image analysis unit prioritizes analysis speed and provides results quickly. For example, if the user is relaxed, the image analysis unit prioritizes analysis accuracy and can perform a detailed analysis. For example, if the user is in a hurry, the image analysis unit can prioritize analyzing only the important information and provide results quickly. This allows for more appropriate analysis results by adjusting the accuracy of the image analysis according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the image analysis unit may be performed using AI, for example, or without AI. For example, the image analysis unit can input user emotion data into a generative AI and have the generative AI perform emotion estimation.

[0082] The image analysis unit improves analysis accuracy by integrating images taken under different lighting conditions and angles. For example, if an image of a receipt is dark, the image analysis unit can automatically adjust the brightness before analysis. For example, if an image of a meal is taken from an oblique angle, the image analysis unit can perform angle correction before analysis. For example, the image analysis unit can integrate multiple images, extract the clearest part, and perform analysis. This improves analysis accuracy by integrating images taken under different lighting conditions and angles. Some or all of the above processing in the image analysis unit may be performed using AI, for example, or without AI. For example, the image analysis unit can input image data from different lighting conditions and angles into a generating AI and have the generating AI perform image integration.

[0083] The image analysis unit optimizes the analysis algorithm by referring to past analysis results. For example, the image analysis unit can quickly analyze receipts of a similar format based on data from receipts that have been analyzed in the past. For example, the image analysis unit can quickly recognize similar dishes based on the analysis results of past food images. For example, the image analysis unit can learn from past analysis results and continuously improve its analysis accuracy. This makes it possible to optimize the analysis algorithm by referring to past analysis results. Some or all of the above processes in the image analysis unit may be performed using AI, for example, or without AI. For example, the image analysis unit can input past analysis result data into a generating AI and have the generating AI perform the optimization of the analysis algorithm.

[0084] The image analysis unit estimates the user's emotions and adjusts the display method of the analysis results based on the estimated user emotions. For example, if the user is tense, the image analysis unit can provide a simple and highly visible display method. For example, if the user is relaxed, the image analysis unit can provide a display method that includes detailed information. For example, if the user is in a hurry, the image analysis unit can provide a display method that gets straight to the point. By adjusting the display method of the analysis results according to the user's emotions, a more appropriate display becomes possible. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or a generative AI. The generative AI is, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. Some or all of the above processing in the image analysis unit may be performed using AI, for example, or without AI. For example, the image analysis unit can input user emotion data into a generative AI and have the generative AI perform emotion estimation.

[0085] The image analysis unit selects the optimal analysis method based on the user's device information. For example, if the user is using a smartphone, the image analysis unit can provide an analysis method optimized for smartphones. For example, if the user is using a tablet, the image analysis unit can provide an analysis method optimized for large screens. For example, if the user is using a desktop, the image analysis unit can provide high-resolution image analysis. This improves analysis accuracy by selecting the optimal analysis method based on the user's device information. Some or all of the above processing in the image analysis unit may be performed using AI, for example, or without AI. For example, the image analysis unit can input the user's device information into a generating AI and have the generating AI select the optimal analysis method.

[0086] The image analysis unit analyzes the user's social media activity and prioritizes the analysis of relevant images. For example, the image analysis unit prioritizes the analysis of images posted by the user on social media. For example, if the user has many followers on social media, the image analysis unit can prioritize the analysis of those images. For example, the image analysis unit can prioritize the analysis of relevant images based on the user's social media activity frequency. This improves the accuracy of the analysis by prioritizing the analysis of relevant images based on the user's social media activity. Some or all of the above processing in the image analysis unit may be performed using AI, for example, or without AI. For example, the image analysis unit can input the user's social media activity data into a generating AI and have the generating AI perform the priority analysis of relevant images.

[0087] The payment calculation unit estimates the user's emotions and adjusts the payment calculation method based on the estimated emotions. For example, if the user is stressed, the payment calculation unit can provide a simple calculation method. For example, if the user is relaxed, the payment calculation unit can provide a detailed calculation method. For example, if the user is in a hurry, the payment calculation unit can provide the calculation result quickly. By adjusting the payment calculation method according to the user's emotions, a more appropriate calculation result can be provided. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. The generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the payment calculation unit may be performed using AI, for example, or not using AI. For example, the payment calculation unit can input user emotion data into a generative AI and have the generative AI perform emotion estimation.

[0088] The payment calculation unit selects the optimal calculation method by referring to past payment history. The payment calculation unit provides the optimal calculation method based on, for example, payment methods previously used by the user. The payment calculation unit can provide the most efficient calculation method based on, for example, the user's past payment history. The payment calculation unit can analyze, for example, the user's past payment history and provide the optimal method for splitting the bill. This allows the optimal calculation method to be selected by referring to past payment history. Some or all of the above processes in the payment calculation unit may be performed using, for example, AI, or not using AI. For example, the payment calculation unit can input past payment history data into a generating AI and have the generating AI perform the selection of the optimal calculation method.

[0089] The payment calculation unit improves calculation accuracy by considering different currencies and tax rates. For example, the payment calculation unit considers payments in different currencies and automatically applies exchange rates. For example, the payment calculation unit can calculate the correct payment amount by considering different tax rates. For example, the payment calculation unit can integrate multiple currencies and tax rates to provide the optimal payment amount. This improves calculation accuracy by considering different currencies and tax rates. Some or all of the above processes in the payment calculation unit may be performed using AI, for example, or not using AI. For example, the payment calculation unit can input data on different currencies and tax rates into a generating AI and have the generating AI perform the calculation accuracy improvement.

[0090] The payment calculation unit estimates the user's emotions and adjusts the display method of the payment amount based on the estimated emotions. For example, if the user is nervous, the payment calculation unit can provide a simple and highly visible display method. For example, if the user is relaxed, the payment calculation unit can provide a display method that includes detailed information. For example, if the user is in a hurry, the payment calculation unit can provide a display method that gets straight to the point. By adjusting the display method of the payment amount according to the user's emotions, a more appropriate display becomes possible. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the payment calculation unit may be performed using AI, for example, or not using AI. For example, the payment calculation unit can input user emotion data into a generative AI and have the generative AI perform emotion estimation.

[0091] The payment calculation unit selects the optimal calculation method based on the user's geographical location information. For example, if the user is in a different region, the payment calculation unit will take into account the tax rate of that region. For example, if the user is traveling, the payment calculation unit can take into account payment in the local currency. For example, the payment calculation unit can provide the optimal payment method based on the user's geographical location information. This improves calculation accuracy by selecting the optimal calculation method based on the user's geographical location information. Some or all of the above processing in the payment calculation unit may be performed using AI, for example, or without AI. For example, the payment calculation unit can input the user's geographical location information into a generating AI and have the generating AI select the optimal calculation method.

[0092] The payment calculation unit analyzes the user's social media activity and prioritizes calculating relevant payment information. For example, the payment calculation unit provides the optimal calculation method based on payment information shared by the user on social media. For example, if a user has many followers on social media, the payment calculation unit can prioritize calculating that payment information. For example, the payment calculation unit can prioritize calculating relevant payment information based on the frequency of the user's social media activity. This improves calculation accuracy by prioritizing the calculation of relevant payment information based on the user's social media activity. Some or all of the above processing in the payment calculation unit may be performed using AI, for example, or without AI. For example, the payment calculation unit can input the user's social media activity data into a generating AI and have the generating AI perform the priority calculation of relevant payment information.

[0093] The payment completion unit estimates the user's emotions and adjusts the payment completion method based on the estimated emotions. For example, if the user is stressed, the payment completion unit can provide a method to complete the payment quickly. For example, if the user is relaxed, the payment completion unit can provide a detailed payment completion method. For example, if the user is in a hurry, the payment completion unit can provide a method to complete the payment with simple steps. This allows for more appropriate payment completion by adjusting the payment completion method according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the payment completion unit may be performed using AI or not using AI. For example, the payment completion unit can input user emotion data into a generative AI and have the generative AI perform emotion estimation.

[0094] The payment completion unit selects the optimal completion method by referring to past payment history. The payment completion unit provides the optimal completion method based on, for example, payment methods previously used by the user. The payment completion unit can provide the most efficient completion method based on, for example, the user's past payment history. The payment completion unit can provide the optimal payment completion method by, for example, analyzing the user's past payment history. This allows the optimal completion method to be selected by referring to past payment history. Some or all of the above processing in the payment completion unit may be performed using, for example, AI, or not using AI. For example, the payment completion unit can input past payment history data into a generating AI and have the generating AI perform the selection of the optimal completion method.

[0095] The payment completion unit improves completion accuracy by integrating different payment methods. For example, the payment completion unit can complete a payment by integrating credit cards and electronic money. For example, the payment completion unit can complete a payment by integrating different electronic payment systems. For example, the payment completion unit can provide the optimal payment completion method by integrating multiple payment methods. This improves the accuracy of payment completion by integrating different payment methods. Some or all of the above processing in the payment completion unit may be performed using AI, for example, or without AI. For example, the payment completion unit can input data from different payment methods into a generating AI and have the generating AI perform the task of improving completion accuracy.

[0096] The payment completion unit estimates the user's emotions and adjusts the display method for payment completion based on the estimated emotions. For example, if the user is nervous, the payment completion unit can provide a simple and highly visible display method. For example, if the user is relaxed, the payment completion unit can provide a display method that includes detailed information. For example, if the user is in a hurry, the payment completion unit can provide a display method that gets straight to the point. By adjusting the display method for payment completion according to the user's emotions, a more appropriate display becomes possible. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the payment completion unit may be performed using AI, for example, or not using AI. For example, the payment completion unit can input user emotion data into a generative AI and have the generative AI perform emotion estimation.

[0097] The payment completion unit selects the optimal completion method based on the user's device information. For example, if the user is using a smartphone, the payment completion unit can provide a completion method optimized for smartphones. For example, if the user is using a tablet, the payment completion unit can provide a completion method optimized for large screens. For example, if the user is using a desktop, the payment completion unit can provide a high-resolution completion method. This improves the accuracy of payment completion by selecting the optimal completion method based on the user's device information. Some or all of the above processing in the payment completion unit may be performed using AI, for example, or without AI. For example, the payment completion unit can input the user's device information into a generating AI and have the generating AI select the optimal completion method.

[0098] The payment completion unit analyzes the user's social media activity and prioritizes the completion of relevant payment information. For example, the payment completion unit provides the optimal completion method based on payment information shared by the user on social media. For example, if the user has many followers on social media, the payment completion unit can prioritize the completion of that payment information. For example, the payment completion unit can prioritize the completion of relevant payment information based on the frequency of the user's social media activity. This improves the accuracy of payment completion by prioritizing the completion of relevant payment information based on the user's social media activity. Some or all of the above processing in the payment completion unit may be performed using AI, for example, or without AI. For example, the payment completion unit can input the user's social media activity data into a generating AI and have the generating AI prioritize the completion of relevant payment information.

[0099] The receipt analysis unit estimates the user's emotions and adjusts the accuracy of the receipt analysis based on the estimated emotions. For example, if the user is stressed, the receipt analysis unit prioritizes analysis speed and provides results quickly. For example, if the user is relaxed, the receipt analysis unit prioritizes analysis accuracy and can perform a detailed analysis. For example, if the user is in a hurry, the receipt analysis unit can prioritize analyzing only the important information and provide results quickly. In this way, by adjusting the accuracy of the receipt analysis according to the user's emotions, more appropriate analysis results can be provided. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the receipt analysis unit may be performed using AI, for example, or without AI. For example, the receipt analysis unit can input user emotion data into a generative AI and have the generative AI perform emotion estimation.

[0100] The receipt analysis unit improves analysis accuracy by integrating receipts of different formats. For example, the receipt analysis unit can automatically recognize and analyze receipt formats from different stores. For example, even if the receipt formats are different, the receipt analysis unit can extract common items and perform analysis. For example, the receipt analysis unit can integrate multiple receipts and analyze the overall payment information. This improves analysis accuracy by integrating receipts of different formats. Some or all of the above processes in the receipt analysis unit may be performed using AI, for example, or without AI. For example, the receipt analysis unit can input receipt data of different formats into a generating AI and have the generating AI perform integrated receipt analysis.

[0101] The receipt analysis unit optimizes its analysis algorithm by referring to past analysis results. For example, the receipt analysis unit can quickly analyze receipts of a similar format based on data from receipts that have been analyzed in the past. For example, the receipt analysis unit can quickly recognize similar items based on past receipt analysis results. For example, the receipt analysis unit can learn from past analysis results and continuously improve its analysis accuracy. This makes it possible to optimize the analysis algorithm by referring to past analysis results. Some or all of the above processes in the receipt analysis unit may be performed using AI, for example, or without AI. For example, the receipt analysis unit can input past analysis result data into a generating AI and have the generating AI perform the optimization of the analysis algorithm.

[0102] The receipt analysis unit estimates the user's emotions and adjusts the display method of the receipt analysis results based on the estimated user emotions. For example, if the user is nervous, the receipt analysis unit can provide a simple and highly visible display method. For example, if the user is relaxed, the receipt analysis unit can provide a display method that includes detailed information. For example, if the user is in a hurry, the receipt analysis unit can provide a display method that gets straight to the point. By adjusting the display method of the receipt analysis results according to the user's emotions, a more appropriate display becomes possible. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or a generative AI. The generative AI is a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples. Some or all of the above processing in the receipt analysis unit may be performed using AI, for example, or without AI. For example, the receipt analysis unit can input user emotion data into a generative AI and have the generative AI perform emotion estimation.

[0103] The receipt analysis unit selects the optimal analysis method based on the user's device information. For example, if the user is using a smartphone, the receipt analysis unit can provide an analysis method optimized for smartphones. For example, if the user is using a tablet, the receipt analysis unit can provide an analysis method optimized for larger screens. For example, if the user is using a desktop computer, the receipt analysis unit can provide high-resolution image analysis. This improves analysis accuracy by selecting the optimal analysis method based on the user's device information. Some or all of the above processing in the receipt analysis unit may be performed using AI, for example, or without AI. For example, the receipt analysis unit can input the user's device information into a generating AI and have the generating AI select the optimal analysis method.

[0104] The receipt analysis unit analyzes the user's social media activity and prioritizes the analysis of relevant receipts. For example, the receipt analysis unit prioritizes the analysis of receipt images posted by the user on social media. For example, the receipt analysis unit can prioritize the analysis of receipts if the user has a large number of social media followers. For example, the receipt analysis unit can prioritize the analysis of relevant receipts based on the user's social media activity frequency. This improves the accuracy of the analysis by prioritizing the analysis of relevant receipts based on the user's social media activity. Some or all of the above processing in the receipt analysis unit may be performed using AI, for example, or without AI. For example, the receipt analysis unit can input the user's social media activity data into a generating AI and have the generating AI perform the priority analysis of relevant receipts.

[0105] The food image analysis unit estimates the user's emotions and adjusts the accuracy of the food image analysis based on the estimated emotions. For example, if the user is stressed, the food image analysis unit prioritizes analysis speed and provides results quickly. For example, if the user is relaxed, the food image analysis unit prioritizes analysis accuracy and can perform a detailed analysis. For example, if the user is in a hurry, the food image analysis unit can prioritize analyzing only the important information and provide results quickly. By adjusting the accuracy of the food image analysis according to the user's emotions, more appropriate analysis results can be provided. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the food image analysis unit may be performed using AI, for example, or without AI. For example, the food image analysis unit can input user emotion data into a generative AI and have the generative AI perform emotion estimation.

[0106] The food image analysis unit improves analysis accuracy by integrating images taken under different lighting conditions and angles. For example, if a food image is dark, the food image analysis unit automatically adjusts the brightness before analysis. For example, if a food image is taken from an oblique angle, the food image analysis unit can perform angle correction before analysis. For example, the food image analysis unit can integrate multiple images, extract the clearest part, and perform analysis. This improves analysis accuracy by integrating images taken under different lighting conditions and angles. Some or all of the above processing in the food image analysis unit may be performed using AI, for example, or without AI. For example, the food image analysis unit can input image data from different lighting conditions and angles into a generating AI and have the generating AI perform image integration.

[0107] The food image analysis unit optimizes its analysis algorithm by referring to past analysis results. For example, the food image analysis unit can quickly analyze similar dishes based on data from previously analyzed meals. For example, the food image analysis unit can quickly recognize similar dishes based on the analysis results of past meal images. For example, the food image analysis unit can learn from past analysis results and continuously improve its analysis accuracy. This makes it possible to optimize the analysis algorithm by referring to past analysis results. Some or all of the above processes in the food image analysis unit may be performed using AI, for example, or without AI. For example, the food image analysis unit can input past analysis result data into a generating AI and have the generating AI perform the optimization of the analysis algorithm.

[0108] The food image analysis unit estimates the user's emotions and adjusts the display method of the food image analysis results based on the estimated user emotions. For example, if the user is nervous, the food image analysis unit can provide a simple and highly visible display method. For example, if the user is relaxed, the food image analysis unit can provide a display method that includes detailed information. For example, if the user is in a hurry, the food image analysis unit can provide a display method that gets straight to the point. By adjusting the display method of the food image analysis results according to the user's emotions, a more appropriate display becomes possible. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or a generative AI. The generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the food image analysis unit may be performed using AI, for example, or without AI. For example, the food image analysis unit can input user emotion data into a generative AI and have the generative AI perform emotion estimation.

[0109] The food image analysis unit selects the optimal analysis method based on the user's device information. For example, if the user is using a smartphone, the food image analysis unit can provide an analysis method optimized for smartphones. For example, if the user is using a tablet, the food image analysis unit can provide an analysis method optimized for larger screens. For example, if the user is using a desktop computer, the food image analysis unit can provide high-resolution image analysis. This improves analysis accuracy by selecting the optimal analysis method based on the user's device information. Some or all of the above processing in the food image analysis unit may be performed using AI, for example, or without AI. For example, the food image analysis unit can input the user's device information into a generating AI and have the generating AI select the optimal analysis method.

[0110] The food image analysis unit analyzes the user's social media activity and prioritizes the analysis of relevant food images. For example, the food image analysis unit prioritizes the analysis of food images posted by the user on social media. For example, if the user has many followers on social media, the food image analysis unit can prioritize the analysis of those food images. For example, the food image analysis unit can prioritize the analysis of relevant food images based on the user's social media activity frequency. This improves the accuracy of the analysis by prioritizing the analysis of relevant food images based on the user's social media activity. Some or all of the above processing in the food image analysis unit may be performed using AI, for example, or without AI. For example, the food image analysis unit can input the user's social media activity data into a generating AI and have the generating AI perform the priority analysis of relevant food images.

[0111] The order matching unit estimates the user's emotions and adjusts the accuracy of order matching based on the estimated emotions. For example, if the user is stressed, the order matching unit prioritizes matching speed and provides results quickly. For example, if the user is relaxed, the order matching unit prioritizes matching accuracy and can perform detailed matching. For example, if the user is in a hurry, the order matching unit can prioritize matching only important information and provide results quickly. This allows for more appropriate matching results by adjusting the accuracy of order matching according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the order matching unit may be performed using AI or not. For example, the order matching unit can input user emotion data into a generative AI and have the generative AI perform emotion estimation.

[0112] The order matching unit improves matching accuracy by referring to past order history. The order matching unit can quickly match similar orders based on past orders, for example. The order matching unit can quickly match similar items based on past order history, for example. The order matching unit can learn from past order history and continuously improve matching accuracy, for example. This improves matching accuracy by referring to past order history. Some or all of the above processing in the order matching unit may be performed using AI, for example, or without AI. For example, the order matching unit can input past order history data into a generating AI and have the generating AI perform the improvement of matching accuracy.

[0113] The order content matching unit improves matching accuracy by integrating order content in different formats. For example, the order content matching unit can automatically recognize and match order formats from different stores. For example, even if the order content formats are different, the order content matching unit can extract common items and perform matching. For example, the order content matching unit can integrate multiple order contents and match the overall order information. This improves matching accuracy by integrating order content in different formats. Some or all of the above processing in the order content matching unit may be performed using AI, for example, or without AI. For example, the order content matching unit can input order content data in different formats into a generating AI and have the generating AI perform integrated matching of the order content.

[0114] The order matching unit estimates the user's emotions and adjusts the display method of the order matching results based on the estimated emotions. For example, if the user is nervous, the order matching unit can provide a simple and highly visible display method. For example, if the user is relaxed, the order matching unit can provide a display method that includes detailed information. For example, if the user is in a hurry, the order matching unit can provide a display method that gets straight to the point. By adjusting the display method of the order matching results according to the user's emotions, a more appropriate display becomes possible. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or a generative AI. The generative AI is a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples. Some or all of the above processing in the order matching unit may be performed using AI, for example, or without AI. For example, the order matching unit can input user emotion data into a generative AI and have the generative AI perform emotion estimation.

[0115] The order matching unit selects the optimal matching method based on the user's device information. For example, if the user is using a smartphone, the order matching unit can provide a matching method optimized for smartphones. For example, if the user is using a tablet, the order matching unit can provide a matching method optimized for larger screens. For example, if the user is using a desktop, the order matching unit can provide a high-resolution matching method. This improves matching accuracy by selecting the optimal matching method based on the user's device information. Some or all of the above processing in the order matching unit may be performed using AI, for example, or without AI. For example, the order matching unit can input the user's device information into a generating AI and have the generating AI select the optimal matching method.

[0116] The order matching unit analyzes the user's social media activity and prioritizes matching relevant order content. For example, the order matching unit prioritizes matching order content posted by the user on social media. For example, the order matching unit can prioritize matching order content if the user has many followers on social media. For example, the order matching unit can prioritize matching relevant order content based on the frequency of the user's social media activity. This improves matching accuracy by prioritizing the matching of relevant order content based on the user's social media activity. Some or all of the above processing in the order matching unit may be performed using AI, for example, or without AI. For example, the order matching unit can input the user's social media activity data into a generating AI and have the generating AI perform priority matching of relevant order content.

[0117] The electronic payment unit estimates the user's emotions and adjusts the electronic payment method based on the estimated emotions. For example, if the user is stressed, the electronic payment unit can provide a method to complete the payment quickly. For example, if the user is relaxed, the electronic payment unit can provide a detailed payment method. For example, if the user is in a hurry, the electronic payment unit can provide a method to complete the payment with simple steps. This allows for more appropriate payments by adjusting the electronic payment method according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the electronic payment unit may be performed using AI or not using AI. For example, the electronic payment unit can input user emotion data into a generative AI and have the generative AI perform emotion estimation.

[0118] The electronic payment unit selects the optimal payment method by referring to past payment history. The electronic payment unit provides the optimal payment method based on the payment methods the user has used in the past, for example. The electronic payment unit can provide the most efficient payment method based on the user's past payment history, for example. The electronic payment unit can provide the optimal payment method by analyzing the user's past payment history, for example. This allows the optimal payment method to be selected by referring to past payment history. Some or all of the above processes in the electronic payment unit may be performed using AI, for example, or without AI. For example, the electronic payment unit can input past payment history data into a generating AI and have the generating AI perform the selection of the optimal payment method.

[0119] The electronic payment unit improves payment accuracy by integrating different payment methods. For example, the electronic payment unit can complete payments by integrating credit cards and electronic money. For example, the electronic payment unit can complete payments by integrating different electronic payment systems. For example, the electronic payment unit can provide the optimal payment method by integrating multiple payment methods. This improves payment accuracy by integrating different payment methods. Some or all of the above processes in the electronic payment unit may be performed using AI, for example, or without AI. For example, the electronic payment unit can input data from different payment methods into a generating AI and have the generating AI perform the improvement of payment accuracy.

[0120] The electronic payment unit estimates the user's emotions and adjusts the display method of the electronic payment result based on the estimated user emotions. For example, if the user is nervous, the electronic payment unit can provide a simple and highly visible display method. For example, if the user is relaxed, the electronic payment unit can provide a display method that includes detailed information. For example, if the user is in a hurry, the electronic payment unit can provide a display method that gets straight to the point. By adjusting the display method of the electronic payment result according to the user's emotions, a more appropriate display becomes possible. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the electronic payment unit may be performed using AI, for example, or without AI. For example, the electronic payment unit can input user emotion data into a generative AI and have the generative AI perform emotion estimation.

[0121] The electronic payment unit selects the optimal payment method based on the user's device information. For example, if the user is using a smartphone, the electronic payment unit can provide a payment method optimized for smartphones. For example, if the user is using a tablet, the electronic payment unit can provide a payment method optimized for larger screens. For example, if the user is using a desktop computer, the electronic payment unit can provide a high-resolution payment method. This improves payment accuracy by selecting the optimal payment method based on the user's device information. Some or all of the above processing in the electronic payment unit may be performed using AI, for example, or without AI. For example, the electronic payment unit can input the user's device information into a generating AI and have the generating AI select the optimal payment method.

[0122] The electronic payment unit analyzes the user's social media activity and prioritizes the processing of relevant payment information. For example, the electronic payment unit provides the optimal payment method based on payment information shared by the user on social media. For example, if a user has many followers on social media, the electronic payment unit can prioritize the processing of that payment information. For example, the electronic payment unit can prioritize the processing of relevant payment information based on the user's frequency of social media activity. This improves payment accuracy by prioritizing the processing of relevant payment information based on the user's social media activity. Some or all of the above processing in the electronic payment unit may be performed using AI, for example, or without AI. For example, the electronic payment unit can input the user's social media activity data into a generating AI and have the generating AI perform the priority processing of relevant payment information.

[0123] The system according to the embodiment is not limited to the example described above, and various modifications are possible, for example, as follows.

[0124] The smart bill-splitting assistant system can estimate the user's emotions and suggest payment methods based on those emotions. For example, if the user is stressed, the system can suggest a quick and easy payment method. If the user is relaxed, it can offer more detailed payment options. And if the user is in a hurry, it can suggest the fastest payment method. This improves user satisfaction by suggesting payment methods that match the user's emotions.

[0125] The smart split-the-bill assistant system can suggest the most suitable payment method by referring to the user's past payment history. For example, if the user has frequently used credit cards in the past, the system will prioritize suggesting credit card payment. If the user frequently uses electronic money, it can suggest electronic money payment. Furthermore, if the user has a history of frequent payments at a particular store, it can suggest the most suitable payment method for that store. In this way, by suggesting the most suitable payment method based on the user's past payment history, the system can improve the convenience of payments.

[0126] The smart split-the-bill assistant system can suggest the optimal payment method based on the user's geographical location. For example, if the user is in a different region, it can suggest a payment method that takes into account the local currency and tax rate. If the user is traveling, it can suggest a payment method in the local currency. Furthermore, if the user frequently makes payments in a particular region, it can suggest the optimal payment method for that region. In this way, by suggesting the optimal payment method based on the user's geographical location, the system can improve the convenience of payments.

[0127] The smart bill-splitting assistant system can suggest the optimal payment method based on the user's device information. For example, if the user is using a smartphone, it can suggest a payment method optimized for smartphones. If the user is using a tablet, it can suggest a payment method optimized for larger screens. Furthermore, if the user is using a desktop computer, it can suggest a high-resolution payment method. This improves the convenience of payments by suggesting the most suitable payment method based on the user's device information.

[0128] The smart split-the-bill assistant system can analyze a user's social media activity and prioritize suggesting relevant payment information. For example, it can suggest the best payment method based on payment information shared by the user on social media. If a user has many followers on social media, it can prioritize suggesting payment information related to that account. It can also prioritize suggesting relevant payment information based on the user's social media activity. In this way, by prioritizing the suggestion of relevant payment information based on the user's social media activity, it can improve the convenience of payments.

[0129] The smart split-the-bill assistant system can estimate the user's emotions and adjust how the payment amount is displayed based on those emotions. For example, if the user is nervous, it can provide a simple and easy-to-read display. If the user is relaxed, it can provide a display that includes detailed information. And if the user is in a hurry, it can provide a display that gets straight to the point. By adjusting how the payment amount is displayed according to the user's emotions, a more appropriate display becomes possible.

[0130] The smart split-the-bill assistant system can estimate the user's emotions and adjust the payment method based on those emotions. For example, if the user is stressed, it can provide a quick payment method. If the user is relaxed, it can provide a more detailed payment method. And if the user is in a hurry, it can provide a simple payment method. This allows for a more appropriate payment process by adjusting the payment method according to the user's emotions.

[0131] The smart split-the-bill assistant system can estimate the user's emotions and adjust the payment calculation method based on those emotions. For example, if the user is stressed, it can provide a simple calculation method. If the user is relaxed, it can provide a more detailed calculation method. Also, if the user is in a hurry, it can provide the calculation result quickly. In this way, by adjusting the payment calculation method according to the user's emotions, it can provide a more appropriate calculation result.

[0132] The smart bill-splitting assistant system can estimate the user's emotions and suggest payment methods based on those emotions. For example, if the user is stressed, the system can suggest a quick and easy payment method. If the user is relaxed, it can offer more detailed payment options. And if the user is in a hurry, it can suggest the fastest payment method. This improves user satisfaction by suggesting payment methods that match the user's emotions.

[0133] The smart split-the-bill assistant system can estimate the user's emotions and adjust how the payment amount is displayed based on those emotions. For example, if the user is nervous, it can provide a simple and easy-to-read display. If the user is relaxed, it can provide a display that includes detailed information. And if the user is in a hurry, it can provide a display that gets straight to the point. By adjusting how the payment amount is displayed according to the user's emotions, a more appropriate display becomes possible.

[0134] The following briefly describes the processing flow for example form 2.

[0135] Step 1: The image analysis unit analyzes the receipt and the food images. The image analysis unit takes a picture of the receipt, and the AI ​​analyzes its contents to identify each person's order. The image analysis unit can also analyze the food images to supplement the order details and achieve more accurate recognition. For example, the image analysis unit uses OCR technology to analyze the text information on the receipt to identify the order details. Furthermore, the image analysis unit can analyze the food images to recognize the type and quantity of dishes. Step 2: The payment calculation unit calculates the payment amount based on the order details analyzed by the image analysis unit. The payment calculation unit matches the order details with the prices and calculates the amount each person should pay. For example, it calculates the amount corresponding to each person's order based on the price information on the receipt. It can also match the prices of the dishes recognized from the image of the food and calculate the payment amount. Step 3: The payment completion unit performs electronic payment based on the payment amount calculated by the payment calculation unit. The payment completion unit completes the payment using the electronic payment system. For example, payments can be made smoothly using credit card payments or mobile payments. Payments can also be completed using electronic money.

[0136] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0137] Data generation model 58 is a form of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> Examples of generative AI include text generation AI, image generation AI, and multimodal generation AI. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats from audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVMs), k-means clustering, convolutional neural networks (CNNs), recurrent neural networks (RNNs), generative adversarial networks (GANs), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each of the above parts is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example.Furthermore, processing performed by AI, including generative AI, may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by AI, including generative AI.

[0138] Furthermore, the processing performed by the data processing system 10 described above is carried out by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but it may also be carried out by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0139] Each of the multiple elements described above, including the image analysis unit, payment calculation unit, and payment completion unit, is implemented in at least one of the smart device 14 and the data processing unit 12. For example, the image analysis unit uses the camera 42 of the smart device 14 to capture images of receipts or meals, which are then analyzed by the processor 46. The payment calculation unit is implemented by the identification processing unit 290 of the data processing unit 12, which calculates the payment amount based on the analyzed order details. The payment completion unit can perform electronic payment using the control unit 46A of the smart device 14. The correspondence between each unit and the device or control unit is not limited to the example described above, and various modifications are possible.

[0140] [Second Embodiment] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0141] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0142] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0143] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0144] The microphone 238 receives voice signals from the user and accepts instructions from the user. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0145] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0146] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0147] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing by the processor 28. The storage 32 stores the specific processing program 56.

[0148] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0149] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0150] In the smart glasses 214, specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 acting as a control unit 46A according to the specific processing program 60 executed on the RAM 48. The smart glasses 214 also have a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0151] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0152] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0153] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0154] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but it may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart glasses 214 or an external device, and the smart glasses 214 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0155] Each of the multiple elements described above, including the image analysis unit, payment calculation unit, and payment completion unit, is implemented in at least one of the smart glasses 214 and the data processing unit 12. For example, the image analysis unit uses the camera 42 of the smart glasses 214 to capture images of receipts or meals, which are then analyzed by the processor 46. The payment calculation unit is implemented by the identification processing unit 290 of the data processing unit 12, which calculates the payment amount based on the analyzed order details. The payment completion unit can perform electronic payment using the control unit 46A of the smart glasses 214. The correspondence between each unit and the device or control unit is not limited to the example described above, and various modifications are possible.

[0156] [Third Embodiment] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0157] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0158] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0159] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0160] The microphone 238 receives voice signals from the user and accepts instructions from the user. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0161] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0162] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0163] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0164] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0165] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0166] In the headset terminal 314, specific processing is performed by the processor 46. The storage 50 stores a specific program 60. The processor 46 reads the specific program 60 from the storage 50 and executes the read specific program 60 on the RAM 48. The specific processing is realized by the processor 46 acting as a control unit 46A according to the specific program 60 executed on the RAM 48. The headset terminal 314 also has a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0167] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0168] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0169] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0170] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset terminal 314, but may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset terminal 314. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the headset terminal 314 or an external device, and the headset terminal 314 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0171] Each of the multiple elements described above, including the image analysis unit, payment calculation unit, and payment completion unit, is implemented in at least one of the headset terminal 314 and the data processing unit 12. For example, the image analysis unit uses the camera 42 of the headset terminal 314 to capture images of receipts or meals, which are then analyzed by the processor 46. The payment calculation unit is implemented by the specific processing unit 290 of the data processing unit 12, which calculates the payment amount based on the analyzed order details. The payment completion unit can perform electronic payment using the control unit 46A of the headset terminal 314. The correspondence between each unit and the device or control unit is not limited to the example described above, and various modifications are possible.

[0172] [Fourth Embodiment] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0173] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0174] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0175] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0176] The microphone 238 receives voice signals from the user and accepts instructions from the user. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0177] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS image sensor or CCD image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0178] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0179] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. The robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0180] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0181] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0182] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0183] In robot 414, specific processing is performed by processor 46. A specific program 60 is stored in storage 50. Processor 46 reads the specific program 60 from storage 50 and executes it on RAM 48. The specific processing is achieved by processor 46 acting as a control unit 46A according to the specific program 60 executed on RAM 48. Robot 414 also has data generation model 58 and emotion identification model 59, similar to those of the robot, and can perform processing similar to that of the specific processing unit 290 using these models.

[0184] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0185] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0186] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0187] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but it may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the robot 414 or an external device, and the robot 414 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0188] Each of the multiple elements described above, including the image analysis unit, payment calculation unit, and payment completion unit, is implemented by, for example, at least one of the robot 414 and the data processing unit 12. For example, the image analysis unit uses the camera 42 of the robot 414 to capture images of receipts or meals, which are then analyzed by the processor 46. The payment calculation unit is implemented by the specific processing unit 290 of the data processing unit 12, which calculates the payment amount based on the analyzed order details. The payment completion unit can perform electronic payment using the control unit 46A of the robot 414. The correspondence between each unit and the device or control unit is not limited to the example described above, and various modifications are possible.

[0189] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0190] Figure 9 shows the emotion map 400, in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0191] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0192] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0193] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, and motorcycles, emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated based, for example, on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0194] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0195] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0196] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing method for the specific process may be used, which includes computer 22 and multiple other computers.

[0197] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0198] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0199] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0200] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0201] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0202] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0203] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0204] Furthermore, although the above-described examples were divided into four embodiments, some or all of these embodiments may be combined. Also, the smart device 14, smart glasses 214, headset terminal 314, and robot 414 are just examples, and they may be combined, or other devices may be used. Also, although the above-described examples were divided into two embodiments, Embodiment 1 and Embodiment 2, these may be combined.

[0205] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and other things that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0206] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0207] (Note 1) An image analysis unit that analyzes receipts and images of meals, A payment calculation unit that calculates the payment amount based on the order details analyzed by the image analysis unit, The system includes a payment completion unit that performs electronic settlement based on the payment amount calculated by the payment calculation unit. A system characterized by the following features. (Note 2) The aforementioned image analysis unit, It includes a receipt analysis unit that analyzes images of receipts to identify order details. The system described in Appendix 1, characterized by the features described herein. (Note 3) The aforementioned image analysis unit, It includes a food image analysis unit that analyzes images of meals and supplements the order details. The system described in Appendix 1, characterized by the features described herein. (Note 4) The aforementioned payment calculation unit, It includes an order verification unit that matches the order details with the price. The system described in Appendix 1, characterized by the features described herein. (Note 5) The aforementioned payment completion section is, It includes an electronic payment section that completes payments using an electronic payment system. The system described in Appendix 1, characterized by the features described herein. (Note 6) The aforementioned image analysis unit, It estimates the user's emotions and adjusts the accuracy of image analysis based on the estimated user emotions. The system described in Appendix 1, characterized by the features described herein. (Note 7) The aforementioned image analysis unit, Integrating images from different lighting conditions and angles improves analysis accuracy. The system described in Appendix 1, characterized by the features described herein. (Note 8) The aforementioned image analysis unit, Optimize the analysis algorithm by referring to past analysis results. The system described in Appendix 1, characterized by the features described herein. (Note 9) The aforementioned image analysis unit, It estimates the user's emotions and adjusts how the analysis results are displayed based on the estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 10) The aforementioned image analysis unit, The optimal analysis method is selected based on the user's device information. The system described in Appendix 1, characterized by the features described herein. (Note 11) The aforementioned image analysis unit, Analyze users' social media activity and prioritize analyzing relevant images. The system described in Appendix 1, characterized by the features described herein. (Note 12) The aforementioned payment calculation unit, It estimates the user's emotions and adjusts the payment calculation method based on the estimated user emotions. The system described in Appendix 1, characterized by the features described herein. (Note 13) The aforementioned payment calculation unit, Refer to past payment history to select the most suitable calculation method. The system described in Appendix 1, characterized by the features described herein. (Note 14) The aforementioned payment calculation unit, Improve calculation accuracy by taking into account different currencies and tax rates. The system described in Appendix 1, characterized by the features described herein. (Note 15) The aforementioned payment calculation unit, The system estimates the user's emotions and adjusts how the payment amount is displayed based on those emotions. The system described in Appendix 1, characterized by the features described herein. (Note 16) The aforementioned payment calculation unit, The optimal calculation method is selected based on the user's geographical location information. The system described in Appendix 1, characterized by the features described herein. (Note 17) The aforementioned payment calculation unit, Analyze users' social media activity and prioritize calculating relevant payment information. The system described in Appendix 1, characterized by the features described herein. (Note 18) The aforementioned payment completion section is, It estimates the user's emotions and adjusts the payment completion method based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 19) The aforementioned payment completion section is, Refer to past payment history to select the most suitable payment method. The system described in Appendix 1, characterized by the features described herein. (Note 20) The aforementioned payment completion section is, Integrating different payment methods improves completion accuracy. The system described in Appendix 1, characterized by the features described herein. (Note 21) The aforementioned payment completion section is, The system estimates the user's emotions and adjusts how payment completion is displayed based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 22) The aforementioned payment completion section is, The optimal completion method is selected based on the user's device information. The system described in Appendix 1, characterized by the features described herein. (Note 23) The aforementioned payment completion section is, Analyze users' social media activity and prioritize processing related payment information. The system described in Appendix 1, characterized by the features described herein. (Note 24) The aforementioned receipt analysis unit is: The system estimates the user's emotions and adjusts the accuracy of receipt analysis based on those estimated emotions. The system described in Appendix 2, characterized by the features described herein. (Note 25) The aforementioned receipt analysis unit is: Integrating receipts in different formats improves analysis accuracy. The system described in Appendix 2, characterized by the features described herein. (Note 26) The aforementioned receipt analysis unit is: Optimize the analysis algorithm by referring to past analysis results. The system described in Appendix 2, characterized by the features described herein. (Note 27) The aforementioned receipt analysis unit is: The system estimates the user's emotions and adjusts how the receipt analysis results are displayed based on those estimated emotions. The system described in Appendix 2, characterized by the features described herein. (Note 28) The aforementioned receipt analysis unit is: The optimal analysis method is selected based on the user's device information. The system described in Appendix 2, characterized by the features described herein. (Note 29) The aforementioned receipt analysis unit is: Analyze users' social media activity and prioritize analyzing relevant receipts. The system described in Appendix 2, characterized by the features described herein. (Note 30) The aforementioned meal image analysis unit, The system estimates the user's emotions and adjusts the accuracy of the food image analysis based on those estimated emotions. The system described in Appendix 3, characterized by the features described herein. (Note 31) The aforementioned meal image analysis unit, Integrating images from different lighting conditions and angles improves analysis accuracy. The system described in Appendix 3, characterized by the features described herein. (Note 32) The aforementioned meal image analysis unit, Optimize the analysis algorithm by referring to past analysis results. The system described in Appendix 3, characterized by the features described herein. (Note 33) The aforementioned meal image analysis unit, The system estimates the user's emotions and adjusts how the food image analysis results are displayed based on those estimated emotions. The system described in Appendix 3, characterized by the features described herein. (Note 34) The aforementioned meal image analysis unit, The optimal analysis method is selected based on the user's device information. The system described in Appendix 3, characterized by the features described herein. (Note 35) The aforementioned meal image analysis unit, Analyze users' social media activity and prioritize analyzing relevant food images. The system described in Appendix 3, characterized by the features described herein. (Note 36) The aforementioned order content verification unit is: The system estimates the user's emotions and adjusts the accuracy of order matching based on those estimated emotions. The system described in Appendix 4, characterized by the features described herein. (Note 37) The aforementioned order content verification unit is: Improve matching accuracy by referring to past order history. The system described in Appendix 4, characterized by the features described herein. (Note 38) The aforementioned order content verification unit is: Integrating order details from different formats improves matching accuracy. The system described in Appendix 4, characterized by the features described herein. (Note 39) The aforementioned order content verification unit is: The system estimates the user's emotions and adjusts how the order matching results are displayed based on those estimated emotions. The system described in Appendix 4, characterized by the features described herein. (Note 40) The aforementioned order content verification unit is: The optimal matching method is selected based on the user's device information. The system described in Appendix 4, characterized by the features described herein. (Note 41) The aforementioned order content verification unit is: Analyze users' social media activity and prioritize matching relevant order details. The system described in Appendix 4, characterized by the features described herein. (Note 42) The aforementioned electronic payment unit is It estimates the user's emotions and adjusts the electronic payment method based on those estimated emotions. The system described in Appendix 5, characterized by the features described herein. (Note 43) The aforementioned electronic payment unit is Select the optimal payment method by referring to past payment history. The system described in Appendix 5, characterized by the features described herein. (Note 44) The aforementioned electronic payment unit is Integrating different payment methods improves payment accuracy. The system described in Appendix 5, characterized by the features described herein. (Note 45) The aforementioned electronic payment unit is The system estimates the user's emotions and adjusts how electronic payment results are displayed based on those estimated emotions. The system described in Appendix 5, characterized by the features described herein. (Note 46) The aforementioned electronic payment unit is The system selects the optimal payment method based on the user's device information. The system described in Appendix 5, characterized by the features described herein. (Note 47) The aforementioned electronic payment unit is Analyze users' social media activity and prioritize payments based on relevant payment information. The system described in Appendix 5, characterized by the features described herein. [Explanation of Symbols]

[0208] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots

Claims

1. An image analysis unit that analyzes receipts and images of meals, A payment calculation unit that calculates the payment amount based on the order details analyzed by the image analysis unit, The system includes a payment completion unit that performs electronic settlement based on the payment amount calculated by the payment calculation unit. A system characterized by the following features.

2. The aforementioned image analysis unit, It includes a receipt analysis unit that analyzes images of receipts to identify order details. The system according to feature 1.

3. The aforementioned image analysis unit, It includes a food image analysis unit that analyzes images of meals and supplements the order details. The system according to feature 1.

4. The aforementioned payment calculation unit, It includes an order verification unit that matches the order details with the price. The system according to feature 1.

5. The aforementioned payment completion section is, It includes an electronic payment section that completes payments using an electronic payment system. The system according to feature 1.

6. The aforementioned image analysis unit, It estimates the user's emotions and adjusts the accuracy of image analysis based on the estimated user emotions. The system according to feature 1.

7. The aforementioned image analysis unit, Integrating images from different lighting conditions and angles improves analysis accuracy. The system according to feature 1.

8. The aforementioned image analysis unit, Optimize the analysis algorithm by referring to past analysis results. The system according to feature 1.

9. The aforementioned image analysis unit, It estimates the user's emotions and adjusts how the analysis results are displayed based on the estimated emotions. The system according to feature 1.

10. The aforementioned image analysis unit, The optimal analysis method is selected based on the user's device information. The system according to feature 1.

Images