system

Abstract

We provide the system. [Solution] A means by which the user inputs and transmits registration information using a communication terminal, A means by which the server receives the registration information and performs authentication via a request from the communication provider, A system that includes a means for the server to generate a unique user ID based on the authentication result and send it back to the terminal.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance that responds to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Conventional accounting operations involve a lot of manual data entry and management, which are prone to human errors and a decrease in work efficiency. In addition, due to the lack of automatic expense classification, real-time data aggregation, and appropriate advice, it has been difficult for companies and individuals to perform accounting operations efficiently. Furthermore, in the cooperation with accountants, manual data sharing and conversations are required, making it difficult to respond quickly. There has been a demand for a system that solves such problems and supports efficient and accurate accounting operations.

Means for Solving the Problems

[0005] The present invention provides a system having means for a user to input registration information using a communication terminal, and for a server to generate a unique user ID after authentication. Furthermore, it includes means for the user to upload images related to transactions, for the server to convert them into text data using image recognition technology, and for automatic classification. The server has means for automatically generating financial statements based on aggregated accounting data, analyzing the accounting data using an AI algorithm, and transmitting the results to the communication terminal. In addition, by providing means for the user to send consultation details to an accountant, for the server to perform a preliminary analysis and generate an automatic response, and escalate to an accountant as necessary, it is possible to improve the efficiency and accuracy of accounting operations.

[0006] A "user" refers to an individual or legal entity that uses the system and operates it using a communication terminal.

[0007] A "communication terminal" is a device (such as a smartphone, tablet, or personal computer) used by a user to access and send and receive data with a server via the internet.

[0008] A "server" is a central computer system that receives, processes, stores, and transmits data sent by users.

[0009] "Registration information" refers to data that includes the user's personal information, contact details, and basic accounting information.

[0010] "Authentication" is the process of verifying that a user is indeed a user of the telecommunications provider's services.

[0011] A "User ID" is an identification code generated by the server to uniquely identify each user within the system.

[0012] "Transaction-related images" refer to images containing supporting documents necessary for accounting purposes, such as receipts and invoices for purchased goods.

[0013] "Image recognition technology" is a technology that extracts text information from uploaded images and converts it into digital text.

[0014] "Text data" refers to character information extracted using image recognition technology, represented in a data format.

[0015] "Automatic classification" is the process of automatically sorting text data into pre-defined categories.

[0016] "Accounting data" refers to all data related to accounting operations, such as income, expenses, and transaction records.

[0017] "Financial statements" are accounting reports that show the financial condition and operating results of a company or individual, and generally include trial balances and financial statements.

[0018] An "AI algorithm" refers to a computational method that uses artificial intelligence to perform data analysis and pattern recognition.

[0019] "Consultation content" refers to a message containing the questions or matters that the user wants to discuss with the accountant.

[0020] "Basic analysis" refers to the process in which a server uses AI to automatically analyze simple questions and problems and generate answers.

[0021] An "automated response" is a response message generated by the server based on basic analysis.

[0022] An "accountant" is a professional qualified to provide expert advice and answers to questions and problems related to a user's accounting operations.

[0023] "Escalation" is the process of forwarding questions or problems that the system cannot handle automatically to an accountant. [Brief explanation of the drawing]

[0024] [Figure 1] It is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] It is a conceptual diagram showing an example of the main functions of a data processing device and a smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

Embodiments for Carrying Out the Invention

[0025] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0026] First, let's explain the terminology used in the following explanation.

[0027] In the following embodiments, the signed processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), and APU (Accelerated Processing Unit).

[0028] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0029] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0030] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0031] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0032] [First Embodiment]

[0033] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0034] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0035] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0036] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0037] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0038] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0039] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0040] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0041] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0042] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0043] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0044] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0045] This invention is a system that improves the efficiency and accuracy of accounting operations by having a server receive and process data entered by a user using a communication terminal. The details of this system program and specific examples of its processing are shown below.

[0046] Program processing and its explanation

[0047] User registration and authentication

[0048] 1. The user accesses the application from their communication device and enters registration information such as their name, phone number, and email address. After entering the information, they click the registration button.

[0049] 2. The terminal temporarily stores the entered information and sends it to the server.

[0050] 3. The server verifies the received registration information via the communication provider's API and performs authentication.

[0051] 4. The server generates a unique user ID based on the authentication result and sends it back to the terminal.

[0052] 5. The terminal displays the authentication result and the generated user ID to the user and provides login information for future logins.

[0053] Data entry and management

[0054] 1. The user uses a communication device to take a picture of the receipt for the purchased items and uploads it to the system. They also enter relevant information (transaction date, amount, etc.) into the corresponding entry field.

[0055] 2. The device sends the uploaded image and input information to the server.

[0056] 3. The server uses image recognition technology to extract text data (such as transaction date, item, and amount) from the receipt image and saves it to the database.

[0057] 4. The server analyzes the stored data and automatically categorizes it into income and expense categories.

[0058] 5. The terminal displays the analysis results to the user and provides an interface for the user to review and correct them.

[0059] Accounting and Analysis

[0060] 1. The server aggregates daily and monthly accounting data and calculates total income and expenses.

[0061] 2. The server automatically generates financial statements such as trial balances and financial statements and saves them in the database.

[0062] 3. The server uses AI algorithms to analyze accounting data and identify unusual spending patterns and potential savings.

[0063] 4. The device displays the analysis results to the user in a visually easy-to-understand format (graphs and tables).

[0064] Collaboration with accountants

[0065] 1. The user enters and sends the details of what they want to discuss with the accountant via a communication terminal.

[0066] 2. The terminal sends the entered consultation details to the server.

[0067] 3. The server analyzes the received inquiry and generates an automated response using AI. If it cannot resolve the issue, it escalates it to an accountant.

[0068] 4. The server sends the response received from the accountant to the communication terminal and notifies the user.

[0069] 5. The device presents the user with detailed answers and provides information to help the user decide on their next action.

[0070] Specific example

[0071] 1. Specific Examples of User Registration and Authentication

[0072] The user accesses the new registration page from their communication device, enters the required information, and clicks the registration button.

[0073] When the terminal sends input information to the server, the server uses the communication provider's API to perform authentication, generate a user ID, and send it back.

[0074] The device displays the authentication result to the user, notifying them of a success message and user ID.

[0075] 2. Specific examples of receipt entry

[0076] Users take a picture of their supermarket shopping receipt with a communication device and upload it.

[0077] The terminal sends the receipt image and input information to the server, which uses image recognition technology to extract and analyze the data and automatically classify the expenses.

[0078] The user reviews the analysis results and makes corrections as needed.

[0079] 3. Specific Examples of Financial Statement Preparation

[0080] The server aggregates monthly accounting data and automatically creates a trial balance.

[0081] The terminal displays the generated trial balance to the user and provides it in a downloadable format.

[0082] 4. Specific examples of conversations with accountants

[0083] The user enters and submits their inquiry regarding year-end financial settlement procedures.

[0084] The server performs a basic analysis, and if it cannot provide an automated answer, it forwards the request to an accountant.

[0085] The accountant enters the appropriate answer, and the server notifies the user of that answer.

[0086] As described above, the present invention provides a system that offers total support for accounting operations and significantly reduces the burden on users. This system enables users to perform accounting operations more efficiently and accurately.

[0087] The following describes the processing flow.

[0088] User registration and authentication

[0089] Step 1:

[0090] The user accesses the application from their communication device and enters registration information such as their name, phone number, and email address. They then click the registration button.

[0091] Step 2:

[0092] The terminal temporarily stores the entered information and sends the registration information to the server.

[0093] Step 3:

[0094] The server sends the received registration information to the communication provider's API for authentication.

[0095] Step 4:

[0096] The server generates a unique user ID based on the authentication result and sends it back to the terminal.

[0097] Step 5:

[0098] The terminal displays the authentication result and the generated user ID to the user, and provides login information for future logins.

[0099] Data entry and management

[0100] Step 1:

[0101] The user uses a communication device to take a picture of the receipt for the purchased items and uploads it to the system. They also enter relevant information (transaction date, amount, etc.) into the corresponding entry field.

[0102] Step 2:

[0103] The device sends the uploaded image and input information to the server.

[0104] Step 3:

[0105] The server uses image recognition technology to extract text data (such as transaction date, item, and amount) from the receipt image and saves it to a database.

[0106] Step 4:

[0107] The server analyzes the stored data and automatically categorizes it into income and expense categories.

[0108] Step 5:

[0109] The server sends the analysis results to the terminal.

[0110] Step 6:

[0111] The terminal displays the analysis results to the user and provides an interface for the user to review and correct them.

[0112] Accounting and Analysis

[0113] Step 1:

[0114] The server aggregates daily and monthly accounting data and calculates total income and expenses.

[0115] Step 2:

[0116] The server automatically generates financial statements such as trial balances and financial statements and saves them in a database.

[0117] Step 3:

[0118] The server uses AI algorithms to analyze accounting data and identify unusual spending patterns and potential savings.

[0119] Step 4:

[0120] The server sends the analysis results to the terminal.

[0121] Step 5:

[0122] The device displays the analysis results to the user in a visually easy-to-understand format (graphs and tables).

[0123] Collaboration with accountants

[0124] Step 1:

[0125] The user inputs and sends the details of what they want to discuss with the accountant via a communication terminal.

[0126] Step 2:

[0127] The terminal sends the entered consultation details to the server.

[0128] Step 3:

[0129] The server analyzes the received inquiry and generates an automated response using AI. If the issue cannot be resolved, it is escalated to an accountant.

[0130] Step 4:

[0131] The server receives the response from the accountant and sends it to the communication terminal, notifying the user.

[0132] Step 5:

[0133] The device presents the user with detailed answers and provides information to help the user decide on their next action.

[0134] Through the above processing steps, this system can efficiently and accurately support accounting operations.

[0135] (Example 1)

[0136] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0137] For many small and medium-sized enterprises (SMEs) and sole proprietors, accounting is a time-consuming and difficult task to maintain accuracy in. Manual data entry and ledger management are time-consuming and costly, and there is a need for more efficient methods. Furthermore, advanced processing such as the automatic generation of necessary financial statements, data analysis, and identification of abnormal expenditure patterns are required, demanding specialized knowledge and skills. Against this backdrop, there is a challenge in building a system that can manage accounting tasks easily and accurately, and automatically generate necessary ledgers and financial statements.

[0138] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0139] In this invention, the server includes means for the user to input and transmit registration information using a communication terminal, means for authentication via a communication network interface, and means for generating a unique identification number based on the authentication result and returning it to the terminal. This enables the user to complete the registration process easily and accurately. Furthermore, the server includes means for the user to upload transaction-related images using a communication terminal, means for converting them into text data using image processing technology, means for analyzing and automatically classifying the text data, and means for transmitting the analysis results to the communication terminal for the user to confirm and correct. This enables the user to efficiently input and manage transaction information. In addition, the server includes means for automatically generating financial reports based on aggregated accounting data, means for transmitting the automatically generated financial reports to the communication terminal for the user to confirm, and means for analyzing accounting data using a machine learning algorithm and transmitting the analysis results to the communication terminal. This enables the user to easily create accurate financial reports and analyze accounting data.

[0140] A "user" is an individual or legal entity that uses a communication terminal to operate the system and input and verify necessary data.

[0141] A "communication terminal" is an electronic device that is connected to the internet and operated by the user, and specifically includes smartphones, tablets, and personal computers.

[0142] A "server" is a central computer system that receives data sent by users, processes it, and returns the results.

[0143] "Registration information" refers to personally identifiable information such as name, telephone number, and email address that the user provides to the system.

[0144] A "communication network interface" is a means of connection that allows a server to exchange data with external communication providers or other systems.

[0145] "Authentication" is the process of verifying the validity of registration information entered by a user via a communication network interface.

[0146] A "unique identification number" is a unique ID generated by the server to individually identify a user.

[0147] "Transaction-related images" refer to image files that users upload to the system, such as receipts and invoices, that represent transaction records.

[0148] "Image processing technology" refers to techniques for extracting text data from images, and specifically includes optical character recognition (OCR).

[0149] "Text data" refers to character information extracted from images related to a transaction using image processing technology.

[0150] "Automatic classification" is the process by which a server categorizes the text data it has analyzed.

[0151] "Accounting data" refers to transaction information entered by users and income and expense data compiled by the server.

[0152] A "financial report" is a document that shows the financial status, such as an income statement and balance sheet, which is automatically generated by a server based on accounting data.

[0153] A "machine learning algorithm" is an AI technology used by servers to analyze accounting data, identifying data patterns and performing predictions and classifications.

[0154] This invention is a system that improves the efficiency and accuracy of accounting operations by having a server receive data entered by a user using a communication terminal. This system is implemented in the following steps.

[0155] User registration and authentication

[0156] The user accesses the application using a communication device (e.g., smartphone, tablet, PC) and enters registration information such as name, phone number, and email address. Once the user submits the information, the device temporarily stores this information and sends it to the server via an HTTPS request. The server receives this registration information through the communication network interface and uses the communication provider's API to perform authentication to verify the legitimacy of the information. If authentication is successful, the server generates a unique identification number and sends it back to the device. The device then displays the authentication result and the generated identification number to the user.

[0157] Data entry and management

[0158] The user takes a picture of the receipt for their purchased items using a communication terminal and uploads it to the system. The terminal sends this receipt image and related information such as the transaction date and amount entered by the user to the server. The server uses image processing technology (specifically, optical character recognition technology - OCR) to extract text data (transaction date, items, amount, etc.) from the receipt image. The extracted text data is analyzed by the server and automatically categorized into income and expense categories. The terminal displays the analysis results to the user and provides an interface that allows the user to review the results and make corrections as needed.

[0159] Accounting and Analysis

[0160] The server aggregates daily and monthly accounting data and calculates total income and expenses. Furthermore, the server automatically generates financial reports such as trial balances and financial statements, and stores them in a database. These generated financial reports are displayed to the user via a terminal. The server also uses machine learning algorithms to analyze accounting data, identifying unusual spending patterns and potential savings. The analysis results are also displayed to the user via the terminal in a visually easy-to-understand format (graphs and tables).

[0161] Collaboration with accountants

[0162] The user uses a communication terminal to input the details of their inquiry to an accountant and sends them to the system. The terminal sends this inquiry to a server, which analyzes the received information and generates an automated response using AI. Inquiries that cannot be resolved are escalated to an accountant. The accountant's response is sent back to the terminal via the server and notified to the user. The terminal presents the user with detailed information about the response and provides information to help them decide on their next course of action.

[0163] Examples and prompts for generative AI models

[0164] Next, we will show specific examples of operation and prompt messages.

[0165] 1. Specific Examples of User Registration and Authentication

[0166] The user accesses the new registration page from their communication device, enters the required information, and clicks the registration button.

[0167] The terminal sends input information to the server, the server uses the communication provider's API to perform authentication, generates a user ID, and sends it back.

[0168] The device displays the authentication result to the user, notifying them of a success message and user ID.

[0169] Prompts for Generative AI Models

[0170] Please explain the procedure for new user registration in the following format.

[0171] 1. The user opens the application and enters the required information.

[0172] 2. Send the input information to the server.

[0173] 3. The server authenticates the information and generates a user ID.

[0174] 4. Notify the user of the authentication result and user ID.

[0175] 2. Specific examples of receipt entry

[0176] Users take a picture of their supermarket shopping receipt with a communication device and upload it.

[0177] The terminal sends the receipt image and input information to the server, which uses image recognition technology to extract and analyze the data and automatically classify the expenses.

[0178] The user reviews the analysis results and makes corrections as needed.

[0179] Prompts for Generative AI Models

[0180] Please explain the procedure for entering receipts at a supermarket in the following format.

[0181] 1. The user takes a picture of the receipt and uploads it.

[0182] 2. Send the image and information to the server.

[0183] 3. The server extracts, analyzes, and classifies the data.

[0184] 4. Display the results to the user for review and correction.

[0185] 3. Specific Examples of Financial Report Preparation

[0186] The server aggregates monthly accounting data and automatically creates a trial balance.

[0187] The terminal displays the generated trial balance to the user and provides it in a downloadable format.

[0188] Prompts for Generative AI Models

[0189] Please describe the procedure for creating a financial report from monthly accounting data in the following format.

[0190] 1. The server aggregates the data.

[0191] 2. Generate financial reports such as trial balances.

[0192] 3. Display the results to the user and provide them in a downloadable format.

[0193] 4. Specific examples of conversations with accountants

[0194] The user enters and submits their inquiry regarding year-end financial settlement procedures.

[0195] The server performs a basic analysis, and if it cannot provide an automated answer, it forwards the request to an accountant.

[0196] The accountant enters the appropriate answer, and the server notifies the user of that answer.

[0197] Prompts for Generative AI Models

[0198] Please describe the procedure for consulting with an accountant in the following format.

[0199] 1. The user enters their inquiry and submits it.

[0200] 2. The server analyzes the content and uses AI to automatically generate a response.

[0201] 3. If an automated response is not possible, the request will be forwarded to an accountant.

[0202] 4. The accountant enters the answer and notifies the user.

[0203] In this way, the present invention provides a system that allows users to perform accounting tasks easily and efficiently via a communication terminal. This improves the accuracy and efficiency of accounting tasks and significantly reduces the burden on users.

[0204] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0205] Step 1:

[0206] The user accesses the application from their communication device and enters registration information such as their name, phone number, and email address. Once the user has finished entering the information, they click the "Register" button. The entered data consists of personal information such as name, phone number, and email address.

[0207] Step 2:

[0208] The terminal temporarily stores the entered registration information and sends it to the server using an HTTPS request. The input here is the registration information provided by the user, and the output is an HTTPS request containing this information.

[0209] Step 3:

[0210] The server authenticates the registration information it receives via the communication network interface through the provider API. The input is the registration information from the terminal, which the server sends to the external API and receives the authentication result. The output is the authentication result.

[0211] Step 4:

[0212] The server generates a unique identification number (user ID) based on the authentication result and sends that ID back to the terminal. The input is the authentication result, and the output is the unique identification number (user ID).

[0213] Step 5:

[0214] The terminal displays the authentication result and the generated user ID to the user. The input is the identification number and authentication result from the server, and the output is the information displayed to the user. For example, a message such as "Registration complete. Your user ID is 123456."

[0215] Step 6:

[0216] The user takes a picture of the receipt for the purchased item using a communication device and enters relevant information such as the transaction date and amount. They then click the "Upload" button to confirm the image data and entered information. The input data consists of the receipt image and transaction information.

[0217] Step 7:

[0218] The device sends the uploaded receipt image and input information to the server via an HTTPS request. The input consists of the image and transaction information provided by the user to the device, and the output is the request sent to the server.

[0219] Step 8:

[0220] The server uses image recognition technology to extract text data (such as transaction date, item, and amount) from a receipt image. The input is a receipt image sent from the terminal, and the output is the extracted text data.

[0221] Step 9:

[0222] The server analyzes the extracted text data, automatically categorizes it into income and expense categories, and stores it in a database. The input is text data after OCR processing, and the output is categorized database entries.

[0223] Step 10:

[0224] The terminal displays the analysis results to the user, providing an interface for the user to review the results and make corrections as needed. The input is the analysis results sent from the server, and the output is the display information and correction interface for the user.

[0225] Step 11:

[0226] The server aggregates daily and monthly accounting data and calculates total income and expenses. The input is transaction information stored in the database, and the output is aggregated income and expense data.

[0227] Step 12:

[0228] The server automatically generates financial reports such as trial balances and financial statements and saves them to the database. The input is aggregated income and expenditure data, and the output is the generated financial reports.

[0229] Step 13:

[0230] The terminal displays the generated financial report to the user and provides it in a downloadable format as needed. The input is the financial report from the server, and the output is a display and download link for the user.

[0231] Step 14:

[0232] The server uses machine learning algorithms to analyze accounting data and identify unusual spending patterns and potential savings. The input is historical accounting data in the database, and the output is the analysis results and their insights.

[0233] Step 15:

[0234] The terminal displays the analysis results to the user in a visually easy-to-understand format (graphs and tables). The input is the analysis results from the server, and the output is displayed in graph and table format.

[0235] Step 16:

[0236] The user uses a communication terminal to input the details of their consultation with an accountant and sends them to the system. The input is the user's consultation content, and the output is the request sent to the server.

[0237] Step 17:

[0238] The terminal sends the entered consultation details to the server. The input is the user's consultation details, and the output is an HTTPS request.

[0239] Step 18:

[0240] The server analyzes the received inquiry and generates an automated response using AI. If the issue cannot be resolved, it escalates it to an accountant. The input is the user's inquiry, and the output is either an AI-generated response or an escalation request to an accountant.

[0241] Step 19:

[0242] The accountant enters the appropriate answer, the server sends that answer to the communication terminal, and the user is notified. The input is the accountant's answer, and the output is the notification to the user.

[0243] Step 20:

[0244] The terminal presents the user with detailed answers and provides information to help the user decide on their next action. The input is the detailed answers from the server, and the output is the information presented to the user.

[0245] (Application Example 1)

[0246] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0247] Modern brick-and-mortar stores demand efficient and accurate accounting and inventory management. However, traditional methods often involve manual processes, leading to frequent input errors and delays in data updates, thus hindering improvements in efficiency and accuracy. Furthermore, users spend a significant amount of time managing accounting data, and real-time inventory management remains challenging.

[0248] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0249] In this invention, the server includes means for a user to input and transmit registration information using a communication terminal, means for the server to receive the registration information and perform authentication via a request from a communication provider, means for the server to generate a unique user ID based on the authentication result and send it back to the terminal, means for the user to input transaction information at a physical store using a communication terminal, means for the user to take a picture of a receipt using smart glasses and send it to the server, means for the server to convert the receipt image into text data using image recognition technology and automatically classify it, and means for the server to update inventory data in real time based on the text data. This makes it possible to improve the efficiency and accuracy of accounting operations and inventory management at physical stores.

[0250] A "communication terminal" is a device used by a user to input and transmit information, and includes smartphones, tablets, and personal computers.

[0251] A "server" is a central computer system that receives data sent by users and performs authentication, data analysis, and storage.

[0252] "Authentication" is the process of verifying a user's registration information through a request from the communication provider to confirm that they are a legitimate user.

[0253] A "User ID" is a unique string or number generated by the server, used to uniquely identify a user.

[0254] A "physical store" is a commercial facility that provides goods and services in a physical location.

[0255] "Transaction information" refers to detailed data about the purchase of goods or services, including the transaction date, item, and amount.

[0256] "Smart glasses" are devices that allow the wearer to collect information about their environment and utilize display and camera functions.

[0257] A "receipt image" is an image of a paper receipt that shows the details of the purchased items.

[0258] "Image recognition technology" is a computer vision technology that extracts text and objects from images and treats them as data.

[0259] "Text data" refers to character information extracted using image recognition technology and stored in digital format.

[0260] "Automatic classification" is the process of automatically sorting analyzed data into specific categories.

[0261] "Inventory data" refers to the quantity and detailed information of products held by a physical store.

[0262] "Real-time" refers to data processing and updates occurring immediately at the moment an event takes place.

[0263] "Accounting work" refers to all tasks involving the recording, aggregation, and analysis of transactions, including the preparation of financial statements.

[0264] This invention relates to a system that streamlines and improves the accuracy of accounting and inventory management in physical stores. Specific embodiments are described below.

[0265] System Configuration

[0266] 1. Communication terminal

[0267] A device used by users to input and transmit information. This includes smartphones, tablets, and personal computers.

[0268] 2. Server

[0269] This is a central computer system that receives data sent from users and performs authentication, data analysis, and storage. The server is responsible for authenticating registration information, generating unique user IDs, aggregating accounting data, and performing AI-based data analysis.

[0270] 3. Smart Glasses

[0271] This device allows the wearer to collect information about their environment and utilize its display and camera functions. It is used for taking pictures of receipts and scanning barcodes.

[0272] Specific examples of hardware and software

[0273] Communication terminal:

[0274] Smartphones: iPhone (registered trademark) or Android (registered trademark) devices

[0275] Computers and tablets: Windows PCs, Macs, iPads (registered trademark), etc.

[0276] server:

[0277] Cloud servers: AWS (Amazon Web Services), Google Cloud Platform, Microsoft Azure, etc.

[0278] Software: Python, TENSORFLOW® (registered trademark) (used for AI algorithm implementation), OpenCV (for image recognition)

[0279] Smart glasses:

[0280] Devices: Google Glass®, Vuzix, etc.

[0281] Contents of data processing and data calculations

[0282] User Authentication:

[0283] 1. The user inputs and transmits registration information (such as name, phone number, email address, etc.) from the communication terminal.

[0284] 2. The server authenticates the received information via the API of the communication provider, generates a unique user ID, and returns it.

[0285] Analysis and Classification of Receipt Images:

[0286] 1. The user uses smart glasses to take a picture of the receipt as transaction information and transmits it to the server through the communication terminal.

[0287] 2. The server extracts text data from the receipt image using image recognition technology (using OpenCV).

[0288] 3. The extracted data is analyzed and automatically classified.

[0289] Real-time Update of Inventory Data:

[0290] 1. The server updates the inventory data in real-time based on the text data obtained from the photographed receipt.

[0291] Specific Example

[0292] The store clerk wears smart glasses and scans the barcode of the product purchased by the customer. After issuing the receipt, the receipt is photographed with the camera of the smart glasses, and the data is transmitted to the server. The server analyzes the receipt information using image recognition technology and updates the inventory information in real-time.

[0293] Examples of Prompt Sentences

[0294] "Create an application that uses smart glasses to scan barcodes of items purchased by customers and take photos of receipts. This application sends the scanned product data and the captured receipt images to a server, where the server analyzes the data and updates inventory information in real time."

[0295] This invention improves the efficiency and accuracy of accounting operations and inventory management in physical stores, significantly reducing the burden on users.

[0296] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0297] Step 1:

[0298] The user enters and submits registration information (name, phone number, email address) using a communication terminal. The information entered by the user is temporarily stored on the communication terminal and then sent to the server. The registration information is used as input data, and as a result, the registration information is sent to the server.

[0299] Step 2:

[0300] The server uses the communication provider's API to perform authentication based on the registration information it receives. The server sends a request to the API and receives the authentication result from the communication provider. The registration information is used as input data, and the authentication result is obtained as output data.

[0301] Step 3:

[0302] The server generates a unique user ID based on the authentication result and sends that user ID back to the terminal. The server verifies the authentication result and uses an algorithm to generate the user ID. The authentication result is used as input data, and the user ID is generated as output data.

[0303] Step 4:

[0304] The user takes a picture of the receipt image of the purchased goods using smart glasses and sends it to the server via the communication terminal. The user takes a picture of the receipt image using the camera of the smart glasses and saves it in the communication terminal. The receipt image is used as input data, and as a result, the receipt image is sent to the server.

[0305] Step 5:

[0306] The server converts the received receipt image into character data using image recognition technology. The server uses image recognition software such as OpenCV to extract character information from the image. The receipt image is used as input data, and text data is generated as output data.

[0307] Step 6:

[0308] The server analyzes the converted text data and automatically classifies the purchase information. The server uses an analysis algorithm to classify the text data into categories of income and expenditure. The text data is used as input data, and classified data is obtained as output data.

[0309] Step 7:

[0310] The server updates the inventory data in real time based on the analysis results. The server accesses the database and subtracts the inventory quantity of the sold goods for updating. The classified purchase information is used as input data, and updated inventory data is obtained as output data.

[0311] Step 8:

[0312] The server automatically generates financial statements based on the aggregated accounting data and sends them to the user's communication terminal. The server uses the collected data to generate trial balance sheets and financial statements and sends them to the communication terminal. The classified accounting information is used as input data, and the generated financial statements are obtained as output data.

[0313] Step 9:

[0314] The server uses an AI algorithm to analyze accounting data and sends the analysis results to a communication terminal. The server uses a generated AI model to identify unusual spending patterns and potential savings. Accounting data is used as input data, and the analysis results are obtained as output data.

[0315] The above processing steps improve the efficiency and accuracy of accounting operations and inventory management in physical stores.

[0316] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0317] This invention provides a system that improves the efficiency and accuracy of accounting operations by having a server receive and process data entered by a user using a communication terminal. Furthermore, by combining it with an emotion engine, it provides a function that allows the system to respond dynamically based on the user's emotions.

[0318] Program processing and its explanation

[0319] User registration and authentication

[0320] 1. The user accesses the application from their communication device and enters registration information such as their name, phone number, and email address. After entering the information, they click the registration button.

[0321] 2. The terminal temporarily stores the entered information and sends it to the server.

[0322] 3. The server sends the received registration information to the communication provider's API for authentication.

[0323] 4. The server generates a unique user ID based on the authentication result and sends it back to the terminal.

[0324] 5. The terminal displays the authentication result and the generated user ID to the user and provides login information for future logins.

[0325] Data entry and management

[0326] 1. The user uses a communication device to take a picture of the receipt for the purchased items and uploads it to the system. They also enter relevant information (transaction date, amount, etc.) into the corresponding entry field.

[0327] 2. The device sends the uploaded image and input information to the server.

[0328] 3. The server uses image recognition technology to extract text data (such as transaction date, item, and amount) from the receipt image and saves it to the database.

[0329] 4. The server analyzes the stored data and automatically categorizes it into income and expense categories.

[0330] 5. The server sends the analysis results to the terminal.

[0331] 6. The terminal displays the analysis results to the user and provides an interface for the user to review and correct them.

[0332] Accounting and Analysis

[0333] 1. The server aggregates daily and monthly accounting data and calculates total income and expenses.

[0334] 2. The server automatically generates financial statements such as trial balances and financial statements and saves them in the database.

[0335] 3. The server uses AI algorithms to analyze accounting data and identify unusual spending patterns and potential savings.

[0336] 4. The server sends the analysis results to the terminal.

[0337] 5. The device displays the analysis results to the user in a visually easy-to-understand format (graphs and tables).

[0338] Collaboration with accountants

[0339] 1. The user enters and sends the details of what they want to discuss with the accountant via a communication terminal.

[0340] 2. The terminal sends the entered consultation details to the server.

[0341] 3. The server analyzes the received inquiry and generates an automated response using AI. If the issue cannot be resolved, it escalates to an accountant.

[0342] 4. The server sends the response received from the accountant to the communication terminal and notifies the user.

[0343] 5. The device presents the user with detailed answers and provides information to help the user decide on their next action.

[0344] Emotional engine integration

[0345] 1. The system collects facial expressions and voice data using the camera and microphone built into the user's communication terminal.

[0346] 2. The device sends the collected data to the emotion engine, which analyzes the user's emotional state.

[0347] 3. The server receives emotion data recognized by the emotion engine and dynamically adjusts the UI / UX based on it.

[0348] 4. The server optimizes the content and timing of accounting advice and alerts based on the analyzed sentiment data.

[0349] 5. The server accumulates emotional data, analyzes long-term emotional patterns, and provides personalized plans and advice tailored to the user's stress level.

[0350] Specific example

[0351] 1. Specific Examples of User Registration and Authentication

[0352] The user accesses the new registration page from their communication device, enters the required information, and clicks the registration button.

[0353] When the terminal sends input information to the server, the server uses the communication provider's API to perform authentication, generate a user ID, and send it back.

[0354] The device displays the authentication result to the user, notifying them of a success message and user ID.

[0355] 2. Specific examples of receipt entry

[0356] Users take a picture of their supermarket shopping receipt with a communication device and upload it.

[0357] The terminal sends receipt images and input information to the server, which uses image recognition technology to extract and analyze the data and automatically classify the expenses.

[0358] The user reviews the analysis results and makes corrections as needed.

[0359] 3. Specific Examples of Financial Statement Preparation

[0360] The server aggregates monthly accounting data and automatically creates a trial balance.

[0361] The terminal displays the generated trial balance to the user and provides it in a downloadable format.

[0362] 4. Specific examples of consultations with an accountant

[0363] The user enters and submits their inquiry regarding year-end financial settlement procedures.

[0364] The server performs a basic analysis, and if it cannot provide an automated answer, it forwards the request to an accountant.

[0365] The accountant enters the appropriate answer, and the server notifies the user of that answer.

[0366] 5. Specific Examples of Emotional Engines

[0367] While the user is using the accounting system, the camera and microphone on the communication terminal capture the user's facial expressions and voice.

[0368] The device analyzes the captured data using an emotion engine to detect when the user is experiencing stress.

[0369] Based on the emotion data detected by the server, measures such as simplifying the system's UI will be taken.

[0370] The server will attempt to reduce user stress by providing accounting advice in a gentler tone.

[0371] Through the processing steps described above, this system not only efficiently and accurately supports accounting operations but also enables flexible responses that take into account the user's emotional state.

[0372] The following describes the processing flow.

[0373] User registration and authentication

[0374] Step 1:

[0375] The user accesses the application from their communication device and enters registration information such as their name, phone number, and email address. After entering the information, they click the registration button.

[0376] Step 2:

[0377] The terminal temporarily stores the entered information and sends it to the server.

[0378] Step 3:

[0379] The server sends the received registration information to the communication provider's API for authentication.

[0380] Step 4:

[0381] The server generates a unique user ID based on the authentication result and sends it back to the terminal.

[0382] Step 5:

[0383] The terminal displays the authentication result and the generated user ID to the user, and provides login information for future logins.

[0384] Data entry and management

[0385] Step 1:

[0386] The user uses a communication device to take a picture of the receipt for the purchased items and uploads it to the system. They also enter relevant information (transaction date, amount, etc.) into the corresponding entry field.

[0387] Step 2:

[0388] The device sends the uploaded image and input information to the server.

[0389] Step 3:

[0390] The server uses image recognition technology to extract text data (such as transaction date, item, and amount) from the receipt image and saves it to a database.

[0391] Step 4:

[0392] The server analyzes the stored data and automatically categorizes it into income and expense categories.

[0393] Step 5:

[0394] The server sends the analysis results to the terminal.

[0395] Step 6:

[0396] The terminal displays the analysis results to the user and provides an interface for the user to review and make corrections.

[0397] Accounting and Analysis

[0398] Step 1:

[0399] The server aggregates daily and monthly accounting data and calculates total income and expenses.

[0400] Step 2:

[0401] The server automatically generates financial statements such as trial balances and financial statements and saves them in a database.

[0402] Step 3:

[0403] The server uses AI algorithms to analyze accounting data and identify unusual spending patterns and potential savings.

[0404] Step 4:

[0405] The server sends the analysis results to the terminal.

[0406] Step 5:

[0407] The device displays the analysis results to the user in a visually easy-to-understand format (graphs and tables).

[0408] Collaboration with accountants

[0409] Step 1:

[0410] The user inputs and sends the details of what they want to discuss with the accountant via a communication terminal.

[0411] Step 2:

[0412] The terminal sends the entered consultation details to the server.

[0413] Step 3:

[0414] The server analyzes the received inquiry and generates an automated response using AI. If the issue cannot be resolved, it is escalated to an accountant.

[0415] Step 4:

[0416] The accountant enters their answers to the user's questions using a dedicated accountant terminal and sends them to the server.

[0417] Step 5:

[0418] The server receives the response from the accountant and sends it to the communication terminal, notifying the user.

[0419] Step 6:

[0420] The device presents the user with detailed answers and provides information to help the user decide on their next action.

[0421] Emotional engine integration

[0422] Step 1:

[0423] The system collects the user's facial expressions and voice data using the camera and microphone built into the communication terminal. If the user does not perform any specific actions, the system periodically captures data.

[0424] Step 2:

[0425] The device collects facial and voice data and transmits it to an emotion engine in real time to analyze the user's emotional state.

[0426] Step 3:

[0427] The server receives emotional data generated by the emotion engine and dynamically adjusts the UI / UX based on it.

[0428] Step 4:

[0429] The server optimizes the content and timing of accounting advice and alerts based on the user's emotional state. For example, if the user is feeling stressed, it will provide more helpful messages and advice.

[0430] Step 5:

[0431] The server tracks the user's emotional data over a long period and provides special plans and advice based on stress levels and emotional patterns. For example, if a user is consistently experiencing high stress levels, it will provide advice on relaxation techniques and efficient financial management methods.

[0432] Specific example

[0433] 1. Specific Examples of User Registration and Authentication

[0434] The user accesses the new registration page from their communication device, enters the required information, and clicks the registration button.

[0435] When the terminal sends input information to the server, the server uses the communication provider's API to perform authentication, generate a user ID, and send it back.

[0436] The device displays the authentication result to the user, notifying them of a success message and user ID.

[0437] 2. Specific examples of receipt entry

[0438] Users take a picture of their supermarket shopping receipt with a communication device and upload it.

[0439] The terminal sends receipt images and input information to the server, which uses image recognition technology to extract and analyze the data and automatically classify the expenses.

[0440] The user reviews the analysis results and makes corrections as needed.

[0441] 3. Specific Examples of Financial Statement Preparation

[0442] The server aggregates monthly accounting data and automatically creates a trial balance.

[0443] The terminal displays the generated trial balance to the user and provides it in a downloadable format.

[0444] 4. Specific examples of consultations with an accountant

[0445] The user enters and submits their inquiry regarding year-end financial settlement procedures.

[0446] The server performs a basic analysis, and if it cannot provide an automated answer, it forwards the request to an accountant.

[0447] The accountant enters the appropriate answer, and the server notifies the user of that answer.

[0448] 5. Specific Examples of Emotional Engines

[0449] While the user is using the accounting system, the camera and microphone on the communication terminal capture the user's facial expressions and voice.

[0450] The device analyzes the captured data using an emotion engine to detect when the user is experiencing stress.

[0451] Based on the emotion data detected by the server, measures such as simplifying the system's UI will be taken.

[0452] The server will attempt to reduce user stress by providing accounting advice in a gentler tone.

[0453] Through the above processes, this system not only efficiently and accurately supports accounting operations but also enables flexible responses that take into account the user's feelings.

[0454] (Example 2)

[0455] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0456] Traditional accounting systems require users to manually input and aggregate data, which is time-consuming, labor-intensive, and prone to human error. Furthermore, few systems consider the user's emotional state, leading to stress accumulation due to prolonged work. Additionally, collaboration with experts is often difficult, making timely consultation challenging. There is a need for a flexible system that addresses these issues, supports efficient and accurate accounting operations, and is considerate of the user's emotional state.

[0457] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0458] In this invention, the server includes means for a user to input and transmit registration information using a communication terminal; means for the server to receive the registration information and perform authentication via a request from a communication provider; means for the server to generate a unique user ID based on the authentication result and send it back to the terminal; means for a user to upload transaction-related images using a communication terminal; means for the server to receive the images and convert them into text data using image recognition technology; means for the server to analyze and automatically classify the text data; means for the server to send the analysis results to the communication terminal for the user to confirm and correct; means for the server to automatically generate financial statements based on aggregated accounting data; means for the server to send the automatically generated financial statements to the communication terminal for the user to confirm; means for the server to analyze accounting data using an artificial intelligence algorithm and send the analysis results to the communication terminal; means for the communication terminal to collect facial and voice data; means for the server to analyze the user's emotional state and dynamically adjust the UI / UX; and means for the server to optimize advice and alert content based on emotional data. This will enable more efficient user data entry, improved data accuracy, and accounting operations that take emotional states into consideration.

[0459] A "user" is a person who uses the system to input data, register information, and perform sentiment analysis.

[0460] A "communication terminal" is a device operated by a user that can input, transmit, and receive data, and collect data using a camera or microphone.

[0461] A "server" is a central computer system that receives data transmitted from communication terminals and performs processing such as storage, analysis, authentication, classification, aggregation, and sentiment analysis.

[0462] "Registration information" refers to personal information such as name, phone number, and email address that users enter using their communication devices.

[0463] A "communication provider" is an external service provider that authenticates users based on registration information and communicates with the system via an API.

[0464] A "unique user ID" is an identifier generated by the server based on the user's authentication result, used to uniquely identify each user.

[0465] "Transaction-related images" refer to image data necessary for accounting processing, including receipts and invoices for purchased goods.

[0466] "Image recognition technology" refers to technologies used by servers to extract text data from images related to transactions, and includes technologies such as OCR (Optical Character Recognition).

[0467] "Text data" refers to character information extracted using image recognition technology, and includes data such as transaction date, item, and amount.

[0468] A "database" is a system for systematically storing and managing data acquired by a server.

[0469] "Automatic classification" is the process by which a server classifies text data it has acquired into specific categories, and this includes the classification of income and expenses.

[0470] "Financial statements" are documents that show financial information, such as trial balances and financial statements, generated based on accounting data compiled by a server.

[0471] An "artificial intelligence algorithm" is a computational method that uses technologies such as machine learning and deep learning, which a server uses to analyze accounting data.

[0472] "Facial and voice data" refers to data that shows the user's facial expressions and voice characteristics, collected using the camera and microphone of the communication device.

[0473] "Emotional state" refers to the user's current psychological state, obtained by analyzing collected facial and voice data, and can include states such as stress or joy.

[0474] "UI / UX" refers to the user interface and user experience, and is a set of design elements that optimize the interaction between a system and its user.

[0475] "Advice and alerts" refer to the advice and warnings that the server provides to the user based on sentiment data and financial data.

[0476] A "specialist" is someone who possesses the knowledge and experience to provide consultation services through the system, and this primarily refers to individuals such as accountants.

[0477] A "prompt" is an input sentence or instruction sent to a generative AI model to obtain a specific output.

[0478] This invention is a system that improves the efficiency and accuracy of accounting operations by having a server receive and process data entered by a user using a communication terminal. Furthermore, by incorporating an emotion engine, it provides a function that allows the system to respond dynamically based on the user's emotions. The following hardware and software are used to implement this invention.

[0479] Hardware and software configuration

[0480] Communication devices (smartphones, tablets, personal computers)

[0481] server

[0482] Camera and microphone (built into the communication terminal)

[0483] Database systems (e.g., MySQL (registered trademark), MongoDB)

[0484] API services (e.g., Twilio API, Google Cloud Vision API, Microsoft Azure Emotion API)

[0485] Programming languages ​​and frameworks (e.g., Node.js, Flask, Pandas, TensorFlow)

[0486] User registration and authentication

[0487] Users access the application using a communication device and enter registration information such as their name, phone number, and email address. The entered information is sent from the device to the server. The server authenticates the user via a request from a communication provider such as the Twilio API and generates a unique user ID based on the authentication result. The generated user ID is sent back to the device, and the user can use it to log in from the next time onward.

[0488] Data entry and management

[0489] The user uses a communication device to take a picture of the receipt for their purchased items and upload it to the system. They also enter information such as the transaction date and amount into the corresponding entry field. The device sends the uploaded image and entered information to the server. The server uses the Google Cloud Vision API to extract text data from the receipt image and saves it to a database. The server then analyzes the saved data and automatically categorizes it into income and expense categories. The analysis results are sent to the device, where the user can review the results and make corrections as needed.

[0490] Accounting and Analysis

[0491] The server aggregates daily and monthly accounting data and calculates total income and expenses. It also automatically generates financial statements such as trial balances and financial statements and stores them in a database. Furthermore, the server analyzes the accounting data using AI algorithms such as TensorFlow to identify unusual spending patterns and potential savings. The analysis results are sent to the terminal, and the user can view the results in a visually easy-to-understand format.

[0492] Collaboration with experts

[0493] When a user enters a specific inquiry and sends it to the system using a communication terminal, the server analyzes the input and generates an automatic response based on a generated AI model. If the server cannot resolve the issue, it escalates it to an expert. After receiving the response from the expert, the server sends the content to the terminal and notifies the user.

[0494] Emotional engine integration

[0495] While a user is using the accounting system, the camera and microphone on the communication terminal collect the user's facial expressions and voice data. The collected data is sent to a server, where the Microsoft Azure Emotion API analyzes the user's emotional state. Based on the emotional data, the server dynamically adjusts the UI / UX and optimizes accounting advice and alerts to reduce user stress. Furthermore, the server accumulates emotional data over the long term and provides special plans and advice tailored to the user's stress level.

[0496] Examples of specific cases and prompt statements

[0497] 1. Specific examples of user registration and authentication:

[0498] The user accesses the new registration page from their communication device, enters the required information, and clicks the registration button. The device then sends the information to the server, which uses the Twilio API to perform authentication, generates a user ID, and sends it back. The user receives a "Registration Successful" message and the user ID.

[0499] 2. Specific examples of receipt entry:

[0500] The user takes a picture of their supermarket shopping receipt and uploads it. The device sends the receipt image and input information to the server, which uses the Google Cloud Vision API to extract and analyze the data and automatically categorize expenses. The user reviews the analysis results and makes corrections as needed.

[0501] 3. Specific examples of financial statement preparation:

[0502] The server aggregates monthly accounting data and automatically generates a trial balance. The generated trial balance is displayed to the user via their terminal and provided in a downloadable format.

[0503] 4. Specific examples of consultations with experts:

[0504] The user enters and submits a question such as, "Why did this month's expenses exceed the budget?" The server generates a prompt and inputs it into a generating AI model (e.g., GPT-3®) to automatically generate an answer. If the server cannot resolve the issue, it forwards the question to an expert and notifies the user of the answer.

[0505] 5. Specific examples of the emotional engine:

[0506] While a user is using the accounting system, the camera and microphone on the communication terminal capture the user's facial expressions and voice. The server analyzes this data using the Microsoft Azure Emotion API to detect if the user is experiencing stress. As a result, the server simplifies the UI and provides accounting advice in a more friendly tone.

[0507] Through the processing steps described above, this system not only efficiently and accurately supports accounting operations but also enables flexible responses that take into account the user's emotional state.

[0508] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0509] Step 1:

[0510] The user accesses the application from their communication device and enters registration information such as their name, phone number, and email address. After entering the information, they click the registration button.

[0511] Input: Registration information such as name, phone number, and email address.

[0512] Output: Request sent to the server by the terminal

[0513] Specific operation: The user enters the required information into the input form and presses the "Register" button in the browser or application on their communication device.

[0514] Step 2:

[0515] The terminal temporarily stores the entered information and sends it to the server using the secure HTTP protocol.

[0516] Input: User-entered registration information

[0517] Output: HTTP POST request to the server

[0518] Specific operation: The terminal temporarily stores the input information in memory and then sends the data to the server via secure HTTPS communication.

[0519] Step 3:

[0520] The server verifies the received information and sends an authentication request to perform phone number authentication using the Twilio API.

[0521] Input: Registration information sent from the device

[0522] Output: Authentication request and authentication result to the Twilio API

[0523] Specific operation: A Node.js script is executed on the server side, formatting the received registration information and sending a request to the Twilio API. This performs phone number verification.

[0524] Step 4:

[0525] The server receives the authentication result, generates a unique user ID, saves it to the database, and simultaneously sends it back to the terminal.

[0526] Input: Authentication result from Twilio API

[0527] Output: Unique user ID, saved to database, and returned to terminal.

[0528] Specific operation: The server confirms successful authentication, generates a unique user ID and saves it to MongoDB, and then sends an authentication success response containing that user ID back to the terminal.

[0529] Step 5:

[0530] The device displays a "Registration successful" message and the generated user ID to the user.

[0531] Input: User ID and authentication result returned from the server.

[0532] Output: "Registration successful" message and user ID displayed.

[0533] Specific operation: The application on the terminal receives a response from the server and displays a "registration successful" message and user ID to the user in the form of a pop-up or notification.

[0534] Step 6:

[0535] The user uses a communication device to take a picture of the receipt for the purchased items and uploads it to the system. They also enter information such as the transaction date and amount into the corresponding entry field.

[0536] Input: Receipt image, transaction date, amount, and other input data.

[0537] Output: Request sent to the server by the terminal

[0538] Specific operation: The user takes a picture of the receipt using the camera on their communication device, enters the transaction date and amount into the application's form, and then clicks the "Upload" button.

[0539] Step 7:

[0540] The terminal temporarily stores the image file and input data, and then sends an HTTP POST request to the server.

[0541] Input: User-submitted receipt image and associated transaction information

[0542] Output: HTTP POST request to the server

[0543] Specific operation: The terminal temporarily stores the receipt image and input information in memory, and then sends the image and data to the server using secure HTTPS communication.

[0544] Step 8:

[0545] The server uses the Google Cloud Vision API to extract text data from the receipt image and saves it to a database.

[0546] Input: Receipt image and transaction information sent from the device.

[0547] Output: Extraction results of character data and saving to the database.

[0548] Specific operation: A Python script is executed on the server side, sending an image to the Google Cloud Vision API, converting the extracted text data into a Pandas DataFrame, and saving it to a MySQL database.

[0549] Step 9:

[0550] The server analyzes the stored data and automatically categorizes it into income and expense categories.

[0551] Input: Text data extracted from receipts

[0552] Output: Accounting data categorized by type

[0553] Specific operation: An analysis script is executed on the server side, automatically classifying the data based on income and expense categories, and updating the appropriate fields in the database with the results.

[0554] Step 10:

[0555] The server returns the analysis results to the terminal in JSON format.

[0556] Input: Accounting data categorized by type

[0557] Output: JSON response of the analysis results

[0558] Specific operation: The server formats the analysis results in JSON format and sends the response to the terminal using secure HTTPS communication.

[0559] Step 11:

[0560] The terminal displays the analysis results it has received and provides an interface for the user to review and correct them.

[0561] Input: Analysis results sent from the server

[0562] Output: Provides an interface for displaying and modifying analysis results.

[0563] Specific operation: The application on the terminal side parses the JSON response, displays the parsing results to the user, and provides an interface for correcting the data as needed.

[0564] Step 12:

[0565] The server aggregates daily and monthly accounting data and calculates total income and expenses.

[0566] Input: Accounting data in the database

[0567] Output: Total income and expenses

[0568] Specific operation: A scheduled script is executed on the server side, using Pandas to aggregate accounting data in the database and calculate daily and monthly totals of income and expenses.

[0569] Step 13:

[0570] The server automatically generates financial statements such as trial balances and financial statements based on aggregated data and saves them in a database.

[0571] Input: Daily and monthly aggregated data

[0572] Output: Auto-generated financial statements

[0573] Specific operation: The generation script is executed on the server side, generating financial statements using Pandas and Matplotlib, and saving them in PDF format.

[0574] Step 14:

[0575] The server uses an AI algorithm based on TensorFlow to analyze accounting data and identify unusual spending patterns and potential savings.

[0576] Input: Automatically generated financial statements and accounting data

[0577] Output: Analysis results

[0578] Specific operation: The server runs an AI model, uses accounting data as input to detect anomalies and suggest cost-saving measures, analyzes the results, and saves them to a database.

[0579] Step 15:

[0580] The server returns the analysis results to the terminal in JSON format.

[0581] Input: Analysis results by AI algorithm

[0582] Output: JSON response

[0583] Specific operation: The server formats the analysis results in JSON format and sends the response to the terminal using secure HTTPS communication.

[0584] Step 16:

[0585] The device displays the analysis results to the user in graph or table format.

[0586] Input: Analysis results sent from the server

[0587] Output: Display in graph or tabular format

[0588] Specific operation: The application on the terminal receives the analysis results and displays them visually using a graph library.

[0589] Step 17:

[0590] The user inputs and sends the details of what they want to discuss with the accountant via a communication terminal.

[0591] Input: Consultation details

[0592] Output: Request sent to the server by the terminal

[0593] Specific action: The user enters their inquiry into the consultation form on their communication device and presses the "Send" button.

[0594] Step 18:

[0595] The terminal temporarily saves the entered consultation details and sends them to the server.

[0596] Input: User-entered consultation details

[0597] Output: HTTP POST request to the server

[0598] Specific operation: The terminal temporarily stores the consultation details in memory and then sends the data to the server via secure HTTPS communication.

[0599] Step 19:

[0600] The server analyzes the received inquiry and generates an automated response based on an AI model. If it cannot resolve the issue, it escalates it to an expert.

[0601] Input: Consultation content sent from the device

[0602] Output: Automated response and escalation results

[0603] Specific operation: The server uses NLP technology to analyze the consultation content and inputs it as a prompt into a generating AI model (e.g., GPT-3) to automatically generate an answer. If a solution cannot be found, the issue is escalated to an expert.

[0604] Step 20:

[0605] The server receives the answer from the expert and sends it to the communication terminal, notifying the user.

[0606] Input: Answer from an expert

[0607] Output: Response to terminal

[0608] Specific operation: After receiving a response from an expert, the server formats the content in JSON format and sends the response to the terminal using secure HTTPS communication.

[0609] Step 21:

[0610] The device presents the user with detailed answers and provides information to help them decide on their next action.

[0611] Input: Response received from the server

[0612] Output: Display to the user

[0613] Specific operation: The application on the device receives the response and displays it to the user in the form of a pop-up or notification, providing information for the next action.

[0614] Step 22:

[0615] While the user is using the accounting system, the camera and microphone on the communication terminal collect facial expressions and voice data.

[0616] Input: User's facial expressions and voice data

[0617] Output: Request sent to the server by the terminal

[0618] Specific operation: The camera and microphone built into the communication terminal capture the user's facial expressions and voice in real time, and temporarily store this data.

[0619] Step 23:

[0620] The device sends the collected data to the emotion engine for analysis.

[0621] Input: Facial expressions and voice data

[0622] Output: Emotion analysis results

[0623] Specific operation: The device sends the collected data to the Microsoft Azure Emotion API to analyze the user's emotional state in real time.

[0624] Step 24:

[0625] The server analyzes the user's emotional state and dynamically adjusts the UI / UX.

[0626] Input: Analysis results from the emotion engine

[0627] Output: UI / UX adjustment results

[0628] Specific operation: The server dynamically adjusts UI elements and tone based on emotional data to optimize the user experience.

[0629] Step 25:

[0630] The server optimizes accounting advice and alerts based on sentiment data.

[0631] Input: Sentimental data and financial data

[0632] Output: Optimized accounting advice and alerts

[0633] Specific operation: The server analyzes emotional data and generates specific accounting advice and alerts tailored to the user's emotional state.

[0634] Step 26:

[0635] The server accumulates emotional data, analyzes long-term emotional patterns, and provides personalized plans and advice.

[0636] Input: Accumulated emotional data

[0637] Output: Special plans or advice

[0638] Specific operation: The server analyzes emotional data accumulated over a long period and provides special countermeasures and advice tailored to the user's stress level.

[0639] (Application Example 2)

[0640] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0641] Currently, many accounting systems do not take into account the emotional state of the user, leading to high levels of user stress. Furthermore, users often encounter complex operations and take considerable time to understand when performing accounting tasks. Therefore, cumbersome user interfaces contribute to increased stress. This situation highlights the need for efficient and stress-free accounting operations.

[0642] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0643] In this invention, the server includes means for the user to input and transmit registration information using a communication terminal; means for the server to receive the registration information and perform authentication via a request from a communication provider; means for the server to generate a unique user ID based on the authentication result and send it back to the terminal; means for collecting the user's facial and voice data from the communication terminal and identifying emotions; means for dynamically adjusting the user interface based on the identified emotions; and means for analyzing the identified emotion data and evaluating the user's stress level. As a result, the user can perform accounting tasks through a simplified user interface, improving work efficiency while reducing stress.

[0644] A "communication terminal" is an electronic device used for inputting, transmitting, and receiving information, and includes smartphones and head-mounted displays.

[0645] A "server" is a computing system used for storing, processing, and managing data, and it performs user registration information and sentiment data analysis.

[0646] "Registration information" refers to information necessary to identify a user, such as their name, phone number, and email address, which they enter using a communication device.

[0647] "Authentication" is the process by which a server verifies the authenticity of a user's registration information through a request from a communication provider.

[0648] A "User ID" is a unique identifier generated by the server based on the authentication result and is used to identify the user.

[0649] "Facial expression data" refers to data obtained by capturing a user's facial expressions with the camera on a communication device and analyzing it to determine the user's emotions.

[0650] "Voice data" refers to data obtained by capturing the user's voice with the microphone of a communication terminal and analyzing it to determine the user's emotions.

[0651] "Emotion recognition" is the process of analyzing a user's facial expression and voice data to identify their current emotional state.

[0652] "Dynamic adjustment" is a feature that changes elements of the user interface in real time based on the user's identified emotions.

[0653] A "user interface" is a means of display and operation for a user to interact with a system, and includes screen design and operation buttons.

[0654] "Emotional data" refers to data that indicates the user's emotional state, obtained through emotion recognition.

[0655] "Stress level" refers to the degree of stress a user is experiencing, evaluated based on emotional data.

[0656] "Image recognition technology" is a technology that analyzes information such as characters from uploaded images and converts it into text data.

[0657] This invention is a system that streamlines and improves the accuracy of accounting operations by allowing users to input and transmit registration information using a communication terminal, which is then received by a server. Furthermore, this system provides a function to analyze the user's emotional state and dynamically adjust the user interface based on that analysis.

[0658] Hardware and software to be used

[0659] Hardware: Smartphones, head-mounted displays (HMDs)

[0660] Software: Flask (a Python®-based web framework), SQLite (database system), APIs for external authentication, image recognition, and sentiment analysis services.

[0661] Program Processing Overview

[0662] User-performed processes

[0663] Users first access the system using a communication terminal and enter registration information such as their name, phone number, and email address. This information is sent to the server, where authentication is performed via the communication provider. If authentication is successful, the server generates a unique user ID and sends it back to the terminal. From then on, users can log in to the system using this user ID.

[0664] Uploading and managing receipts

[0665] Users take a picture of their purchase receipt with a communication device and upload it to the system. The server receives this receipt image and uses image recognition technology to convert it into text data (e.g., transaction date, item, amount). This data is stored in a database and automatically categorized. The analysis results are displayed on the device, allowing the user to review and modify them.

[0666] Aggregation and analysis of accounting data

[0667] The server aggregates daily and monthly accounting data and calculates total income and expenses. It automatically generates financial documents such as trial balances and financial statements and stores them in a database. Furthermore, the server uses AI algorithms to analyze the accounting data, identifying unusual spending patterns and potential savings. These analysis results are transmitted to the communication terminal and displayed visually to the user.

[0668] Emotional engine integration

[0669] When a user uses the camera and microphone built into their communication device to collect facial and voice data, it is sent to a server. The server analyzes the user's emotions using an emotion analysis algorithm and dynamically adjusts the user interface based on the results. For example, if the user is feeling stressed, the UI is simplified and accounting advice is provided in a gentler tone.

[0670] Examples of usage and prompt statements

[0671] User registration usage example

[0672] The user accesses the new registration page from their communication device, enters the required information, and clicks the registration button. The server performs authentication, generates a user ID, and sends it back to the device.

[0673] Prompt example: "A new user is attempting to register with the name 'User A', phone number '090-1234-5678', and email address 'user@example.com'."

[0674] Example of using receipt entry

[0675] Users take photos of their supermarket shopping receipts and upload them. The server uses image recognition technology to extract and analyze the data, automatically classifying expenses.

[0676] Example prompt: "The user has taken a picture of a receipt and uploaded it. Extract the date, item, and amount from the image."

[0677] Examples of using an emotion engine

[0678] While a user is using the accounting system, the camera and microphone on the communication terminal capture the user's facial expressions and voice. The server analyzes this data to detect stress and simplifies the UI to make it easier to use.

[0679] Example prompt: "Analyze data from the user's camera and microphone to assess their stress level. If stress is high, simplify the user interface."

[0680] Through the processing steps described above, this system not only efficiently and accurately supports accounting operations but also enables flexible responses that take into account the user's emotional state.

[0681] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0682] Step 1:

[0683] The user accesses the new registration page from their communication device, enters registration information such as their name, phone number, and email address, and submits it. This entered data is temporarily stored by the device and then sent to the server.

[0684] Input: Name, phone number, email address

[0685] Output: Registration information sent to the server

[0686] Step 2:

[0687] The server authenticates the received registration information via the communication provider's API. It then retrieves the authentication result from the communication provider and generates a unique user ID based on it.

[0688] Input: Registration information (name, phone number, email address)

[0689] Data processing: Authentication using the communication provider's API

[0690] Output: Authentication result, unique user ID

[0691] Step 3:

[0692] The server sends a generated user ID back to the terminal, and the terminal displays the authentication result and user ID to the user. The user uses this user ID for future logins.

[0693] Input: Unique User ID

[0694] Output: Authentication result and user ID displayed on the terminal

[0695] Step 4:

[0696] The user uses a communication device to take a picture of the receipt related to the transaction, enters relevant information (transaction date, amount, etc.) into the corresponding entry field, and uploads it to the system.

[0697] Input: Receipt image, transaction date, amount

[0698] Output: Receipt image and related information sent to the server

[0699] Step 5:

[0700] The server converts the received receipt image into text data using image recognition technology. The converted data is then stored in a database.

[0701] Input: Receipt image

[0702] Data processing: Text conversion using image recognition technology

[0703] Output: Converted text data

[0704] Step 6:

[0705] The server analyzes the stored data and automatically categorizes it into income and expense categories. The analysis results are sent to the terminal, where the user can review and modify them.

[0706] Input: Text data

[0707] Data processing: Automated classification algorithms

[0708] Output: Automated classification of analysis results

[0709] Step 7:

[0710] The user collects facial and audio data using the camera and microphone built into their communication terminal. This data is then transmitted from the terminal to a server.

[0711] Input: Facial expression data, audio data

[0712] Output: Facial expression and audio data sent to the server

[0713] Step 8:

[0714] The server uses an emotion analysis algorithm to analyze the user's emotional state and dynamically adjusts the user interface based on the analysis results.

[0715] Input: Facial expression data, audio data

[0716] Data processing: Sentiment analysis algorithm

[0717] Output: Adjusted user interface

[0718] Step 9:

[0719] The server accumulates long-term emotional data and evaluates user stress levels. Based on the evaluation, it generates data to provide special plans and advice.

[0720] Input: Analyzed sentiment data

[0721] Data processing: Stress level evaluation algorithm

[0722] Output: Special plans and advice provided to the user.

[0723] Through these processing steps, this system not only efficiently and accurately supports accounting operations but also enables flexible responses that take into account the user's emotional state.

[0724] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0725] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0726] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0727] [Second Embodiment]

[0728] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0729] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0730] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0731] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0732] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0733] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0734] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0735] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0736] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0737] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0738] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0739] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0740] This invention is a system that improves the efficiency and accuracy of accounting operations by having a server receive and process data entered by a user using a communication terminal. The details of this system program and specific examples of its processing are shown below.

[0741] Program processing and its explanation

[0742] User registration and authentication

[0743] 1. The user accesses the application from their communication device and enters registration information such as their name, phone number, and email address. After entering the information, they click the registration button.

[0744] 2. The terminal temporarily stores the entered information and sends it to the server.

[0745] 3. The server verifies the received registration information via the communication provider's API and performs authentication.

[0746] 4. The server generates a unique user ID based on the authentication result and sends it back to the terminal.

[0747] 5. The terminal displays the authentication result and the generated user ID to the user and provides login information for future logins.

[0748] Data entry and management

[0749] 1. The user uses a communication device to take a picture of the receipt for the purchased items and uploads it to the system. They also enter relevant information (transaction date, amount, etc.) into the corresponding entry field.

[0750] 2. The device sends the uploaded image and input information to the server.

[0751] 3. The server uses image recognition technology to extract text data (such as transaction date, item, and amount) from the receipt image and saves it to the database.

[0752] 4. The server analyzes the stored data and automatically categorizes it into income and expense categories.

[0753] 5. The terminal displays the analysis results to the user and provides an interface for the user to review and correct them.

[0754] Accounting and Analysis

[0755] 1. The server aggregates daily and monthly accounting data and calculates total income and expenses.

[0756] 2. The server automatically generates financial statements such as trial balances and financial statements and saves them in the database.

[0757] 3. The server uses AI algorithms to analyze accounting data and identify unusual spending patterns and potential savings.

[0758] 4. The device displays the analysis results to the user in a visually easy-to-understand format (graphs and tables).

[0759] Collaboration with accountants

[0760] 1. The user enters and sends the details of what they want to discuss with the accountant via a communication terminal.

[0761] 2. The terminal sends the entered consultation details to the server.

[0762] 3. The server analyzes the received inquiry and generates an automated response using AI. If it cannot resolve the issue, it escalates it to an accountant.

[0763] 4. The server sends the response received from the accountant to the communication terminal and notifies the user.

[0764] 5. The device presents the user with detailed answers and provides information to help the user decide on their next action.

[0765] Specific example

[0766] 1. Specific Examples of User Registration and Authentication

[0767] The user accesses the new registration page from their communication device, enters the required information, and clicks the registration button.

[0768] When the terminal sends input information to the server, the server uses the communication provider's API to perform authentication, generate a user ID, and send it back.

[0769] The device displays the authentication result to the user, notifying them of a success message and user ID.

[0770] 2. Specific examples of receipt entry

[0771] Users take a picture of their supermarket shopping receipt with a communication device and upload it.

[0772] The terminal sends the receipt image and input information to the server, which uses image recognition technology to extract and analyze the data and automatically classify the expenses.

[0773] The user reviews the analysis results and makes corrections as needed.

[0774] 3. Specific Examples of Financial Statement Preparation

[0775] The server aggregates monthly accounting data and automatically creates a trial balance.

[0776] The terminal displays the generated trial balance to the user and provides it in a downloadable format.

[0777] 4. Specific examples of conversations with accountants

[0778] The user enters and submits their inquiry regarding year-end financial settlement procedures.

[0779] The server performs a basic analysis, and if it cannot provide an automated answer, it forwards the request to an accountant.

[0780] The accountant enters the appropriate answer, and the server notifies the user of that answer.

[0781] As described above, the present invention provides a system that offers total support for accounting operations and significantly reduces the burden on users. This system enables users to perform accounting operations more efficiently and accurately.

[0782] The following describes the processing flow.

[0783] User registration and authentication

[0784] Step 1:

[0785] The user accesses the application from their communication device and enters registration information such as their name, phone number, and email address. They then click the registration button.

[0786] Step 2:

[0787] The terminal temporarily stores the entered information and sends the registration information to the server.

[0788] Step 3:

[0789] The server sends the received registration information to the communication provider's API for authentication.

[0790] Step 4:

[0791] The server generates a unique user ID based on the authentication result and sends it back to the terminal.

[0792] Step 5:

[0793] The terminal displays the authentication result and the generated user ID to the user, and provides login information for future logins.

[0794] Data entry and management

[0795] Step 1:

[0796] The user uses a communication device to take a picture of the receipt for the purchased items and uploads it to the system. They also enter relevant information (transaction date, amount, etc.) into the corresponding entry field.

[0797] Step 2:

[0798] The device sends the uploaded image and input information to the server.

[0799] Step 3:

[0800] The server uses image recognition technology to extract text data (such as transaction date, item, and amount) from the receipt image and saves it to a database.

[0801] Step 4:

[0802] The server analyzes the stored data and automatically categorizes it into income and expense categories.

[0803] Step 5:

[0804] The server sends the analysis results to the terminal.

[0805] Step 6:

[0806] The terminal displays the analysis results to the user and provides an interface for the user to review and correct them.

[0807] Accounting and Analysis

[0808] Step 1:

[0809] The server aggregates daily and monthly accounting data and calculates total income and expenses.

[0810] Step 2:

[0811] The server automatically generates financial statements such as trial balances and financial statements and saves them in a database.

[0812] Step 3:

[0813] The server uses AI algorithms to analyze accounting data and identify unusual spending patterns and potential savings.

[0814] Step 4:

[0815] The server sends the analysis results to the terminal.

[0816] Step 5:

[0817] The device displays the analysis results to the user in a visually easy-to-understand format (graphs and tables).

[0818] Collaboration with accountants

[0819] Step 1:

[0820] The user inputs and sends the details of what they want to discuss with the accountant via a communication terminal.

[0821] Step 2:

[0822] The terminal sends the entered consultation details to the server.

[0823] Step 3:

[0824] The server analyzes the received inquiry and generates an automated response using AI. If the issue cannot be resolved, it is escalated to an accountant.

[0825] Step 4:

[0826] The server receives the response from the accountant and sends it to the communication terminal, notifying the user.

[0827] Step 5:

[0828] The device presents the user with detailed answers and provides information to help the user decide on their next action.

[0829] Through the above processing steps, this system can efficiently and accurately support accounting operations.

[0830] (Example 1)

[0831] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0832] For many small and medium-sized enterprises (SMEs) and sole proprietors, accounting is a time-consuming and difficult task to maintain accuracy in. Manual data entry and ledger management are time-consuming and costly, and there is a need for more efficient methods. Furthermore, advanced processing such as the automatic generation of necessary financial statements, data analysis, and identification of abnormal expenditure patterns are required, demanding specialized knowledge and skills. Against this backdrop, there is a challenge in building a system that can manage accounting tasks easily and accurately, and automatically generate necessary ledgers and financial statements.

[0833] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0834] In this invention, the server includes means for the user to input and transmit registration information using a communication terminal, means for authentication via a communication network interface, and means for generating a unique identification number based on the authentication result and returning it to the terminal. This enables the user to complete the registration process easily and accurately. Furthermore, the server includes means for the user to upload transaction-related images using a communication terminal, means for converting them into text data using image processing technology, means for analyzing and automatically classifying the text data, and means for transmitting the analysis results to the communication terminal for the user to confirm and correct. This enables the user to efficiently input and manage transaction information. In addition, the server includes means for automatically generating financial reports based on aggregated accounting data, means for transmitting the automatically generated financial reports to the communication terminal for the user to confirm, and means for analyzing accounting data using a machine learning algorithm and transmitting the analysis results to the communication terminal. This enables the user to easily create accurate financial reports and analyze accounting data.

[0835] A "user" is an individual or legal entity that uses a communication terminal to operate the system and input and verify necessary data.

[0836] A "communication terminal" is an electronic device that is connected to the internet and operated by the user, and specifically includes smartphones, tablets, and personal computers.

[0837] A "server" is a central computer system that receives data sent by users, processes it, and returns the results.

[0838] "Registration information" refers to personally identifiable information such as name, telephone number, and email address that the user provides to the system.

[0839] A "communication network interface" is a means of connection that allows a server to exchange data with external communication providers or other systems.

[0840] "Authentication" is the process of verifying the validity of registration information entered by a user via a communication network interface.

[0841] A "unique identification number" is a unique ID generated by the server to individually identify a user.

[0842] "Transaction-related images" refer to image files that users upload to the system, such as receipts and invoices, that represent transaction records.

[0843] "Image processing technology" refers to techniques for extracting text data from images, and specifically includes optical character recognition (OCR).

[0844] "Text data" refers to character information extracted from images related to a transaction using image processing technology.

[0845] "Automatic classification" is the process by which a server categorizes the text data it has analyzed.

[0846] "Accounting data" refers to transaction information entered by users and income and expense data compiled by the server.

[0847] A "financial report" is a document that shows the financial status, such as an income statement and balance sheet, which is automatically generated by a server based on accounting data.

[0848] A "machine learning algorithm" is an AI technology used by servers to analyze accounting data, identifying data patterns and performing predictions and classifications.

[0849] This invention is a system that improves the efficiency and accuracy of accounting operations by having a server receive data entered by a user using a communication terminal. This system is implemented in the following steps.

[0850] User registration and authentication

[0851] The user accesses the application using a communication device (e.g., smartphone, tablet, PC) and enters registration information such as name, phone number, and email address. Once the user submits the information, the device temporarily stores this information and sends it to the server via an HTTPS request. The server receives this registration information through the communication network interface and uses the communication provider's API to perform authentication to verify the legitimacy of the information. If authentication is successful, the server generates a unique identification number and sends it back to the device. The device then displays the authentication result and the generated identification number to the user.

[0852] Data entry and management

[0853] The user takes a picture of the receipt for their purchased items using a communication terminal and uploads it to the system. The terminal sends this receipt image and related information such as the transaction date and amount entered by the user to the server. The server uses image processing technology (specifically, optical character recognition technology - OCR) to extract text data (transaction date, items, amount, etc.) from the receipt image. The extracted text data is analyzed by the server and automatically categorized into income and expense categories. The terminal displays the analysis results to the user and provides an interface that allows the user to review the results and make corrections as needed.

[0854] Accounting and Analysis

[0855] The server aggregates daily and monthly accounting data and calculates total income and expenses. Furthermore, the server automatically generates financial reports such as trial balances and financial statements, and stores them in a database. These generated financial reports are displayed to the user via a terminal. The server also uses machine learning algorithms to analyze accounting data, identifying unusual spending patterns and potential savings. The analysis results are also displayed to the user via the terminal in a visually easy-to-understand format (graphs and tables).

[0856] Collaboration with accountants

[0857] The user uses a communication terminal to input the details of their inquiry to an accountant and sends them to the system. The terminal sends this inquiry to a server, which analyzes the received information and generates an automated response using AI. Inquiries that cannot be resolved are escalated to an accountant. The accountant's response is sent back to the terminal via the server and notified to the user. The terminal presents the user with detailed information about the response and provides information to help them decide on their next course of action.

[0858] Examples and prompts for generative AI models

[0859] Next, we will show specific examples of operation and prompt messages.

[0860] 1. Specific Examples of User Registration and Authentication

[0861] The user accesses the new registration page from their communication device, enters the required information, and clicks the registration button.

[0862] The terminal sends input information to the server, the server uses the communication provider's API to perform authentication, generates a user ID, and sends it back.

[0863] The device displays the authentication result to the user, notifying them of a success message and user ID.

[0864] Prompts for Generative AI Models

[0865] Please explain the procedure for new user registration in the following format.

[0866] 1. The user opens the application and enters the required information.

[0867] 2. Send the input information to the server.

[0868] 3. The server authenticates the information and generates a user ID.

[0869] 4. Notify the user of the authentication result and user ID.

[0870] 2. Specific examples of receipt entry

[0871] Users take a picture of their supermarket shopping receipt with a communication device and upload it.

[0872] The terminal sends the receipt image and input information to the server, which uses image recognition technology to extract and analyze the data and automatically classify the expenses.

[0873] The user reviews the analysis results and makes corrections as needed.

[0874] Prompts for Generative AI Models

[0875] Please explain the procedure for entering receipts at a supermarket in the following format.

[0876] 1. The user takes a picture of the receipt and uploads it.

[0877] 2. Send the image and information to the server.

[0878] 3. The server extracts, analyzes, and classifies the data.

[0879] 4. Display the results to the user for review and correction.

[0880] 3. Specific Examples of Financial Report Preparation

[0881] The server aggregates monthly accounting data and automatically creates a trial balance.

[0882] The terminal displays the generated trial balance to the user and provides it in a downloadable format.

[0883] Prompts for Generative AI Models

[0884] Please describe the procedure for creating a financial report from monthly accounting data in the following format.

[0885] 1. The server aggregates the data.

[0886] 2. Generate financial reports such as trial balances.

[0887] 3. Display the results to the user and provide them in a downloadable format.

[0888] 4. Specific examples of conversations with accountants

[0889] The user enters and submits their inquiry regarding year-end financial settlement procedures.

[0890] The server performs a basic analysis, and if it cannot provide an automated answer, it forwards the request to an accountant.

[0891] The accountant enters the appropriate answer, and the server notifies the user of that answer.

[0892] Prompts for Generative AI Models

[0893] Please describe the procedure for consulting with an accountant in the following format.

[0894] 1. The user enters their inquiry and submits it.

[0895] 2. The server analyzes the content and uses AI to automatically generate a response.

[0896] 3. If an automated response is not possible, the request will be forwarded to an accountant.

[0897] 4. The accountant enters the answer and notifies the user.

[0898] In this way, the present invention provides a system that allows users to perform accounting tasks easily and efficiently via a communication terminal. This improves the accuracy and efficiency of accounting tasks and significantly reduces the burden on users.

[0899] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0900] Step 1:

[0901] The user accesses the application from their communication device and enters registration information such as their name, phone number, and email address. Once the user has finished entering the information, they click the "Register" button. The entered data consists of personal information such as name, phone number, and email address.

[0902] Step 2:

[0903] The terminal temporarily stores the entered registration information and sends it to the server using an HTTPS request. The input here is the registration information provided by the user, and the output is an HTTPS request containing this information.

[0904] Step 3:

[0905] The server authenticates the registration information it receives via the communication network interface through the provider API. The input is the registration information from the terminal, which the server sends to the external API and receives the authentication result. The output is the authentication result.

[0906] Step 4:

[0907] The server generates a unique identification number (user ID) based on the authentication result and sends that ID back to the terminal. The input is the authentication result, and the output is the unique identification number (user ID).

[0908] Step 5:

[0909] The terminal displays the authentication result and the generated user ID to the user. The input is the identification number and authentication result from the server, and the output is the information displayed to the user. For example, a message such as "Registration complete. Your user ID is 123456."

[0910] Step 6:

[0911] The user takes a picture of the receipt for the purchased item using a communication device and enters relevant information such as the transaction date and amount. They then click the "Upload" button to confirm the image data and entered information. The input data consists of the receipt image and transaction information.

[0912] Step 7:

[0913] The device sends the uploaded receipt image and input information to the server via an HTTPS request. The input consists of the image and transaction information provided by the user to the device, and the output is the request sent to the server.

[0914] Step 8:

[0915] The server uses image recognition technology to extract text data (such as transaction date, item, and amount) from a receipt image. The input is a receipt image sent from the terminal, and the output is the extracted text data.

[0916] Step 9:

[0917] The server analyzes the extracted text data, automatically categorizes it into income and expense categories, and stores it in a database. The input is text data after OCR processing, and the output is categorized database entries.

[0918] Step 10:

[0919] The terminal displays the analysis results to the user, providing an interface for the user to review the results and make corrections as needed. The input is the analysis results sent from the server, and the output is the display information and correction interface for the user.

[0920] Step 11:

[0921] The server aggregates daily and monthly accounting data and calculates total income and expenses. The input is transaction information stored in the database, and the output is aggregated income and expense data.

[0922] Step 12:

[0923] The server automatically generates financial reports such as trial balances and financial statements and saves them to the database. The input is aggregated income and expenditure data, and the output is the generated financial reports.

[0924] Step 13:

[0925] The terminal displays the generated financial report to the user and provides it in a downloadable format as needed. The input is the financial report from the server, and the output is a display and download link for the user.

[0926] Step 14:

[0927] The server uses machine learning algorithms to analyze accounting data and identify unusual spending patterns and potential savings. The input is historical accounting data in the database, and the output is the analysis results and their insights.

[0928] Step 15:

[0929] The terminal displays the analysis results to the user in a visually easy-to-understand format (graphs and tables). The input is the analysis results from the server, and the output is displayed in graph and table format.

[0930] Step 16:

[0931] The user uses a communication terminal to input the details of their consultation with an accountant and sends them to the system. The input is the user's consultation content, and the output is the request sent to the server.

[0932] Step 17:

[0933] The terminal sends the entered consultation details to the server. The input is the user's consultation details, and the output is an HTTPS request.

[0934] Step 18:

[0935] The server analyzes the received inquiry and generates an automated response using AI. If the issue cannot be resolved, it escalates it to an accountant. The input is the user's inquiry, and the output is either an AI-generated response or an escalation request to an accountant.

[0936] Step 19:

[0937] The accountant enters the appropriate answer, the server sends that answer to the communication terminal, and the user is notified. The input is the accountant's answer, and the output is the notification to the user.

[0938] Step 20:

[0939] The terminal presents the user with detailed answers and provides information to help the user decide on their next action. The input is the detailed answers from the server, and the output is the information presented to the user.

[0940] (Application Example 1)

[0941] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0942] Modern brick-and-mortar stores demand efficient and accurate accounting and inventory management. However, traditional methods often involve manual processes, leading to frequent input errors and delays in data updates, thus hindering improvements in efficiency and accuracy. Furthermore, users spend a significant amount of time managing accounting data, and real-time inventory management remains challenging.

[0943] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0944] In this invention, the server includes means for a user to input and transmit registration information using a communication terminal, means for the server to receive the registration information and perform authentication via a request from a communication provider, means for the server to generate a unique user ID based on the authentication result and send it back to the terminal, means for the user to input transaction information at a physical store using a communication terminal, means for the user to take a picture of a receipt using smart glasses and send it to the server, means for the server to convert the receipt image into text data using image recognition technology and automatically classify it, and means for the server to update inventory data in real time based on the text data. This makes it possible to improve the efficiency and accuracy of accounting operations and inventory management at physical stores.

[0945] A "communication terminal" is a device used by a user to input and transmit information, and includes smartphones, tablets, and personal computers.

[0946] A "server" is a central computer system that receives data sent by users and performs authentication, data analysis, and storage.

[0947] "Authentication" is the process of verifying a user's registration information through a request from the communication provider to confirm that they are a legitimate user.

[0948] A "User ID" is a unique string or number generated by the server, used to uniquely identify a user.

[0949] A "physical store" is a commercial facility that provides goods and services in a physical location.

[0950] "Transaction information" refers to detailed data about the purchase of goods or services, including the transaction date, item, and amount.

[0951] "Smart glasses" are devices that allow the wearer to collect information about their environment and utilize display and camera functions.

[0952] A "receipt image" is an image of a paper receipt that shows the details of the purchased items.

[0953] "Image recognition technology" is a computer vision technology that extracts text and objects from images and treats them as data.

[0954] "Text data" refers to character information extracted using image recognition technology and stored in digital format.

[0955] "Automatic classification" is the process of automatically sorting analyzed data into specific categories.

[0956] "Inventory data" refers to the quantity and detailed information of products held by a physical store.

[0957] "Real-time" refers to data processing and updates occurring immediately at the moment an event takes place.

[0958] "Accounting work" refers to all tasks involving the recording, aggregation, and analysis of transactions, including the preparation of financial statements.

[0959] This invention relates to a system that streamlines and improves the accuracy of accounting and inventory management in physical stores. Specific embodiments are described below.

[0960] System Configuration

[0961] 1. Communication terminal

[0962] A device used by users to input and transmit information. This includes smartphones, tablets, and personal computers.

[0963] 2. Server

[0964] This is a central computer system that receives data sent from users and performs authentication, data analysis, and storage. The server is responsible for authenticating registration information, generating unique user IDs, aggregating accounting data, and performing AI-based data analysis.

[0965] 3. Smart Glasses

[0966] This device allows the wearer to collect information about their environment and utilize its display and camera functions. It is used for taking pictures of receipts and scanning barcodes.

[0967] Specific examples of hardware and software

[0968] Communication terminal:

[0969] Smartphone: iPhone or Android device

[0970] Computers and Tablets: Windows PCs, Macs, iPads, etc.

[0971] server:

[0972] Cloud servers: AWS (Amazon Web Services), Google Cloud Platform, Microsoft Azure, etc.

[0973] Software: Python, TensorFlow (used for implementing AI algorithms), OpenCV (for image recognition)

[0974] Smart glasses:

[0975] Devices: Google Glass, Vuzix, etc.

[0976] Contents of data processing and data calculations

[0977] User authentication:

[0978] 1. The user enters and submits registration information (name, phone number, email address, etc.) from their communication terminal.

[0979] 2. The server authenticates the received information via the communication provider's API, generates a unique user ID, and sends it back.

[0980] Analysis and classification of receipt images:

[0981] 1. The user uses smart glasses to take a picture of the receipt as transaction information and sends it to the server via a communication terminal.

[0982] 2. The server uses image recognition technology (using OpenCV) to extract text data from the receipt image.

[0983] 3. The extracted data is analyzed and automatically classified.

[0984] Real-time updates of inventory data:

[0985] 1. The server updates inventory data in real time based on text data obtained from the scanned receipts.

[0986] Specific example

[0987] The store clerk wears smart glasses and scans the barcode of the customer's purchased items. After issuing a receipt, the clerk takes a picture of the receipt with the smart glasses' camera, and the data is sent to a server. The server uses image recognition technology to analyze the receipt information and updates inventory information in real time.

[0988] Example of a prompt

[0989] "Create an application that uses smart glasses to scan barcodes of items purchased by customers and take photos of receipts. This application sends the scanned product data and the captured receipt images to a server, where the server analyzes the data and updates inventory information in real time."

[0990] This invention improves the efficiency and accuracy of accounting operations and inventory management in physical stores, significantly reducing the burden on users.

[0991] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0992] Step 1:

[0993] The user enters and submits registration information (name, phone number, email address) using a communication terminal. The information entered by the user is temporarily stored on the communication terminal and then sent to the server. The registration information is used as input data, and as a result, the registration information is sent to the server.

[0994] Step 2:

[0995] The server uses the communication provider's API to perform authentication based on the registration information it receives. The server sends a request to the API and receives the authentication result from the communication provider. The registration information is used as input data, and the authentication result is obtained as output data.

[0996] Step 3:

[0997] The server generates a unique user ID based on the authentication result and sends that user ID back to the terminal. The server verifies the authentication result and uses an algorithm to generate the user ID. The authentication result is used as input data, and the user ID is generated as output data.

[0998] Step 4:

[0999] The user takes a picture of the receipt for the purchased items using smart glasses and sends it to the server via a communication terminal. The user takes a picture of the receipt using the camera on the smart glasses and saves it to the communication terminal. The receipt image is used as input data, and as a result, the receipt image is sent to the server.

[1000] Step 5:

[1001] The server converts the received receipt image into text data using image recognition technology. The server uses image recognition software such as OpenCV to extract text information from the image. A receipt image is used as input data, and text data is generated as output data.

[1002] Step 6:

[1003] The server analyzes the converted text data and automatically classifies the purchase information. The server uses an analysis algorithm to categorize the text data into income and expense categories. Text data is used as input, and the classified data is obtained as output.

[1004] Step 7:

[1005] The server updates inventory data in real time based on the analysis results. The server accesses the database and updates the inventory by subtracting the number of sold items. Classified purchase information is used as input data, and updated inventory data is obtained as output data.

[1006] Step 8:

[1007] The server automatically generates financial statements based on the aggregated accounting data and sends them to the user's communication terminal. The server uses the collected data to generate trial balances and financial statements and sends them to the communication terminal. The classified accounting information is used as input data, and the generated financial statements are obtained as output data.

[1008] Step 9:

[1009] The server uses an AI algorithm to analyze accounting data and sends the analysis results to a communication terminal. The server uses a generated AI model to identify unusual spending patterns and potential savings. Accounting data is used as input data, and the analysis results are obtained as output data.

[1010] The above processing steps improve the efficiency and accuracy of accounting operations and inventory management in physical stores.

[1011] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1012] This invention provides a system that improves the efficiency and accuracy of accounting operations by having a server receive and process data entered by a user using a communication terminal. Furthermore, by combining it with an emotion engine, it provides a function that allows the system to respond dynamically based on the user's emotions.

[1013] Program processing and its explanation

[1014] User registration and authentication

[1015] 1. The user accesses the application from their communication device and enters registration information such as their name, phone number, and email address. After entering the information, they click the registration button.

[1016] 2. The terminal temporarily stores the entered information and sends it to the server.

[1017] 3. The server sends the received registration information to the communication provider's API for authentication.

[1018] 4. The server generates a unique user ID based on the authentication result and sends it back to the terminal.

[1019] 5. The terminal displays the authentication result and the generated user ID to the user and provides login information for future logins.

[1020] Data entry and management

[1021] 1. The user uses a communication device to take a picture of the receipt for the purchased items and uploads it to the system. They also enter relevant information (transaction date, amount, etc.) into the corresponding entry field.

[1022] 2. The device sends the uploaded image and input information to the server.

[1023] 3. The server uses image recognition technology to extract text data (such as transaction date, item, and amount) from the receipt image and saves it to the database.

[1024] 4. The server analyzes the stored data and automatically categorizes it into income and expense categories.

[1025] 5. The server sends the analysis results to the terminal.

[1026] 6. The terminal displays the analysis results to the user and provides an interface for the user to review and correct them.

[1027] Accounting and Analysis

[1028] 1. The server aggregates daily and monthly accounting data and calculates total income and expenses.

[1029] 2. The server automatically generates financial statements such as trial balances and financial statements and saves them in the database.

[1030] 3. The server uses AI algorithms to analyze accounting data and identify unusual spending patterns and potential savings.

[1031] 4. The server sends the analysis results to the terminal.

[1032] 5. The device displays the analysis results to the user in a visually easy-to-understand format (graphs and tables).

[1033] Collaboration with accountants

[1034] 1. The user enters and sends the details of what they want to discuss with the accountant via a communication terminal.

[1035] 2. The terminal sends the entered consultation details to the server.

[1036] 3. The server analyzes the received inquiry and generates an automated response using AI. If the issue cannot be resolved, it escalates to an accountant.

[1037] 4. The server sends the response received from the accountant to the communication terminal and notifies the user.

[1038] 5. The device presents the user with detailed answers and provides information to help the user decide on their next action.

[1039] Emotional engine integration

[1040] 1. The system collects facial expressions and voice data using the camera and microphone built into the user's communication terminal.

[1041] 2. The device sends the collected data to the emotion engine, which analyzes the user's emotional state.

[1042] 3. The server receives emotion data recognized by the emotion engine and dynamically adjusts the UI / UX based on it.

[1043] 4. The server optimizes the content and timing of accounting advice and alerts based on the analyzed sentiment data.

[1044] 5. The server accumulates emotional data, analyzes long-term emotional patterns, and provides personalized plans and advice tailored to the user's stress level.

[1045] Specific example

[1046] 1. Specific Examples of User Registration and Authentication

[1047] The user accesses the new registration page from their communication device, enters the required information, and clicks the registration button.

[1048] When the terminal sends input information to the server, the server uses the communication provider's API to perform authentication, generate a user ID, and send it back.

[1049] The device displays the authentication result to the user, notifying them of a success message and user ID.

[1050] 2. Specific examples of receipt entry

[1051] Users take a picture of their supermarket shopping receipt with a communication device and upload it.

[1052] The terminal sends receipt images and input information to the server, which uses image recognition technology to extract and analyze the data and automatically classify the expenses.

[1053] The user reviews the analysis results and makes corrections as needed.

[1054] 3. Specific Examples of Financial Statement Preparation

[1055] The server aggregates monthly accounting data and automatically creates a trial balance.

[1056] The terminal displays the generated trial balance to the user and provides it in a downloadable format.

[1057] 4. Specific examples of consultations with an accountant

[1058] The user enters and submits their inquiry regarding year-end financial settlement procedures.

[1059] The server performs a basic analysis, and if it cannot provide an automated answer, it forwards the request to an accountant.

[1060] The accountant enters the appropriate answer, and the server notifies the user of that answer.

[1061] 5. Specific Examples of Emotional Engines

[1062] While the user is using the accounting system, the camera and microphone on the communication terminal capture the user's facial expressions and voice.

[1063] The device analyzes the captured data using an emotion engine to detect when the user is experiencing stress.

[1064] Based on the emotion data detected by the server, measures such as simplifying the system's UI will be taken.

[1065] The server will attempt to reduce user stress by providing accounting advice in a gentler tone.

[1066] Through the processing steps described above, this system not only efficiently and accurately supports accounting operations but also enables flexible responses that take into account the user's emotional state.

[1067] The following describes the processing flow.

[1068] User registration and authentication

[1069] Step 1:

[1070] The user accesses the application from their communication device and enters registration information such as their name, phone number, and email address. After entering the information, they click the registration button.

[1071] Step 2:

[1072] The terminal temporarily stores the entered information and sends it to the server.

[1073] Step 3:

[1074] The server sends the received registration information to the communication provider's API for authentication.

[1075] Step 4:

[1076] The server generates a unique user ID based on the authentication result and sends it back to the terminal.

[1077] Step 5:

[1078] The terminal displays the authentication result and the generated user ID to the user, and provides login information for future logins.

[1079] Data entry and management

[1080] Step 1:

[1081] The user uses a communication device to take a picture of the receipt for the purchased items and uploads it to the system. They also enter relevant information (transaction date, amount, etc.) into the corresponding entry field.

[1082] Step 2:

[1083] The device sends the uploaded image and input information to the server.

[1084] Step 3:

[1085] The server uses image recognition technology to extract text data (such as transaction date, item, and amount) from the receipt image and saves it to a database.

[1086] Step 4:

[1087] The server analyzes the stored data and automatically categorizes it into income and expense categories.

[1088] Step 5:

[1089] The server sends the analysis results to the terminal.

[1090] Step 6:

[1091] The terminal displays the analysis results to the user and provides an interface for the user to review and make corrections.

[1092] Accounting and Analysis

[1093] Step 1:

[1094] The server aggregates daily and monthly accounting data and calculates total income and expenses.

[1095] Step 2:

[1096] The server automatically generates financial statements such as trial balances and financial statements and saves them in a database.

[1097] Step 3:

[1098] The server uses AI algorithms to analyze accounting data and identify unusual spending patterns and potential savings.

[1099] Step 4:

[1100] The server sends the analysis results to the terminal.

[1101] Step 5:

[1102] The device displays the analysis results to the user in a visually easy-to-understand format (graphs and tables).

[1103] Collaboration with accountants

[1104] Step 1:

[1105] The user inputs and sends the details of what they want to discuss with the accountant via a communication terminal.

[1106] Step 2:

[1107] The terminal sends the entered consultation details to the server.

[1108] Step 3:

[1109] The server analyzes the received inquiry and generates an automated response using AI. If the issue cannot be resolved, it is escalated to an accountant.

[1110] Step 4:

[1111] The accountant enters their answers to the user's questions using a dedicated accountant terminal and sends them to the server.

[1112] Step 5:

[1113] The server receives the response from the accountant and sends it to the communication terminal, notifying the user.

[1114] Step 6:

[1115] The device presents the user with detailed answers and provides information to help the user decide on their next action.

[1116] Emotional engine integration

[1117] Step 1:

[1118] The system collects the user's facial expressions and voice data using the camera and microphone built into the communication terminal. If the user does not perform any specific actions, the system periodically captures data.

[1119] Step 2:

[1120] The device collects facial and voice data and transmits it to an emotion engine in real time to analyze the user's emotional state.

[1121] Step 3:

[1122] The server receives emotional data generated by the emotion engine and dynamically adjusts the UI / UX based on it.

[1123] Step 4:

[1124] The server optimizes the content and timing of accounting advice and alerts based on the user's emotional state. For example, if the user is feeling stressed, it will provide more helpful messages and advice.

[1125] Step 5:

[1126] The server tracks the user's emotional data over a long period and provides special plans and advice based on stress levels and emotional patterns. For example, if a user is consistently experiencing high stress levels, it will provide advice on relaxation techniques and efficient financial management methods.

[1127] Specific example

[1128] 1. Specific Examples of User Registration and Authentication

[1129] The user accesses the new registration page from their communication device, enters the required information, and clicks the registration button.

[1130] When the terminal sends input information to the server, the server uses the communication provider's API to perform authentication, generate a user ID, and send it back.

[1131] The device displays the authentication result to the user, notifying them of a success message and user ID.

[1132] 2. Specific examples of receipt entry

[1133] Users take a picture of their supermarket shopping receipt with a communication device and upload it.

[1134] The terminal sends receipt images and input information to the server, which uses image recognition technology to extract and analyze the data and automatically classify the expenses.

[1135] The user reviews the analysis results and makes corrections as needed.

[1136] 3. Specific Examples of Financial Statement Preparation

[1137] The server aggregates monthly accounting data and automatically creates a trial balance.

[1138] The terminal displays the generated trial balance to the user and provides it in a downloadable format.

[1139] 4. Specific examples of consultations with an accountant

[1140] The user enters and submits their inquiry regarding year-end financial settlement procedures.

[1141] The server performs a basic analysis, and if it cannot provide an automated answer, it forwards the request to an accountant.

[1142] The accountant enters the appropriate answer, and the server notifies the user of that answer.

[1143] 5. Specific Examples of Emotional Engines

[1144] While the user is using the accounting system, the camera and microphone on the communication terminal capture the user's facial expressions and voice.

[1145] The device analyzes the captured data using an emotion engine to detect when the user is experiencing stress.

[1146] Based on the emotion data detected by the server, measures such as simplifying the system's UI will be taken.

[1147] The server will attempt to reduce user stress by providing accounting advice in a gentler tone.

[1148] Through the above processes, this system not only efficiently and accurately supports accounting operations but also enables flexible responses that take into account the user's feelings.

[1149] (Example 2)

[1150] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[1151] Traditional accounting systems require users to manually input and aggregate data, which is time-consuming, labor-intensive, and prone to human error. Furthermore, few systems consider the user's emotional state, leading to stress accumulation due to prolonged work. Additionally, collaboration with experts is often difficult, making timely consultation challenging. There is a need for a flexible system that addresses these issues, supports efficient and accurate accounting operations, and is considerate of the user's emotional state.

[1152] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1153] In this invention, the server includes means for a user to input and transmit registration information using a communication terminal; means for the server to receive the registration information and perform authentication via a request from a communication provider; means for the server to generate a unique user ID based on the authentication result and send it back to the terminal; means for a user to upload transaction-related images using a communication terminal; means for the server to receive the images and convert them into text data using image recognition technology; means for the server to analyze and automatically classify the text data; means for the server to send the analysis results to the communication terminal for the user to confirm and correct; means for the server to automatically generate financial statements based on aggregated accounting data; means for the server to send the automatically generated financial statements to the communication terminal for the user to confirm; means for the server to analyze accounting data using an artificial intelligence algorithm and send the analysis results to the communication terminal; means for the communication terminal to collect facial and voice data; means for the server to analyze the user's emotional state and dynamically adjust the UI / UX; and means for the server to optimize advice and alert content based on emotional data. This will enable more efficient user data entry, improved data accuracy, and accounting operations that take emotional states into consideration.

[1154] A "user" is a person who uses the system to input data, register information, and perform sentiment analysis.

[1155] A "communication terminal" is a device operated by a user that can input, transmit, and receive data, and collect data using a camera or microphone.

[1156] A "server" is a central computer system that receives data transmitted from communication terminals and performs processing such as storage, analysis, authentication, classification, aggregation, and sentiment analysis.

[1157] "Registration information" refers to personal information such as name, phone number, and email address that users enter using their communication devices.

[1158] A "communication provider" is an external service provider that authenticates users based on registration information and communicates with the system via an API.

[1159] A "unique user ID" is an identifier generated by the server based on the user's authentication result, used to uniquely identify each user.

[1160] "Transaction-related images" refer to image data necessary for accounting processing, including receipts and invoices for purchased goods.

[1161] "Image recognition technology" refers to technologies used by servers to extract text data from images related to transactions, and includes technologies such as OCR (Optical Character Recognition).

[1162] "Text data" refers to character information extracted using image recognition technology, and includes data such as transaction date, item, and amount.

[1163] A "database" is a system for systematically storing and managing data acquired by a server.

[1164] "Automatic classification" is the process by which a server classifies text data it has acquired into specific categories, and this includes the classification of income and expenses.

[1165] "Financial statements" are documents that show financial information, such as trial balances and financial statements, generated based on accounting data compiled by a server.

[1166] An "artificial intelligence algorithm" is a computational method that uses technologies such as machine learning and deep learning, which a server uses to analyze accounting data.

[1167] "Facial and voice data" refers to data that shows the user's facial expressions and voice characteristics, collected using the camera and microphone of the communication device.

[1168] "Emotional state" refers to the user's current psychological state, obtained by analyzing collected facial and voice data, and can include states such as stress or joy.

[1169] "UI / UX" refers to the user interface and user experience, and is a set of design elements that optimize the interaction between a system and its user.

[1170] "Advice and alerts" refer to the advice and warnings that the server provides to the user based on sentiment data and financial data.

[1171] A "specialist" is someone who possesses the knowledge and experience to provide consultation services through the system, and this primarily refers to individuals such as accountants.

[1172] A "prompt" is an input sentence or instruction sent to a generative AI model to obtain a specific output.

[1173] This invention is a system that improves the efficiency and accuracy of accounting operations by having a server receive and process data entered by a user using a communication terminal. Furthermore, by incorporating an emotion engine, it provides a function that allows the system to respond dynamically based on the user's emotions. The following hardware and software are used to implement this invention.

[1174] Hardware and software configuration

[1175] Communication devices (smartphones, tablets, personal computers)

[1176] server

[1177] Camera and microphone (built into the communication terminal)

[1178] Database systems (e.g., MySQL, MongoDB)

[1179] API services (e.g., Twilio API, Google Cloud Vision API, Microsoft Azure Emotion API)

[1180] Programming languages ​​and frameworks (e.g., Node.js, Flask, Pandas, TensorFlow)

[1181] User registration and authentication

[1182] Users access the application using a communication device and enter registration information such as their name, phone number, and email address. The entered information is sent from the device to the server. The server authenticates the user via a request from a communication provider such as the Twilio API and generates a unique user ID based on the authentication result. The generated user ID is sent back to the device, and the user can use it to log in from the next time onward.

[1183] Data entry and management

[1184] The user uses a communication device to take a picture of the receipt for their purchased items and upload it to the system. They also enter information such as the transaction date and amount into the corresponding entry field. The device sends the uploaded image and entered information to the server. The server uses the Google Cloud Vision API to extract text data from the receipt image and saves it to a database. The server then analyzes the saved data and automatically categorizes it into income and expense categories. The analysis results are sent to the device, where the user can review the results and make corrections as needed.

[1185] Accounting and Analysis

[1186] The server aggregates daily and monthly accounting data and calculates total income and expenses. It also automatically generates financial statements such as trial balances and financial statements and stores them in a database. Furthermore, the server analyzes the accounting data using AI algorithms such as TensorFlow to identify unusual spending patterns and potential savings. The analysis results are sent to the terminal, and the user can view the results in a visually easy-to-understand format.

[1187] Collaboration with experts

[1188] When a user enters a specific inquiry and sends it to the system using a communication terminal, the server analyzes the input and generates an automatic response based on a generated AI model. If the server cannot resolve the issue, it escalates it to an expert. After receiving the response from the expert, the server sends the content to the terminal and notifies the user.

[1189] Emotional engine integration

[1190] While a user is using the accounting system, the camera and microphone on the communication terminal collect the user's facial expressions and voice data. The collected data is sent to a server, where the Microsoft Azure Emotion API analyzes the user's emotional state. Based on the emotional data, the server dynamically adjusts the UI / UX and optimizes accounting advice and alerts to reduce user stress. Furthermore, the server accumulates emotional data over the long term and provides special plans and advice tailored to the user's stress level.

[1191] Examples of specific cases and prompt statements

[1192] 1. Specific examples of user registration and authentication:

[1193] The user accesses the new registration page from their communication device, enters the required information, and clicks the registration button. The device then sends the information to the server, which uses the Twilio API to perform authentication, generates a user ID, and sends it back. The user receives a "Registration Successful" message and the user ID.

[1194] 2. Specific examples of receipt entry:

[1195] The user takes a picture of their supermarket shopping receipt and uploads it. The device sends the receipt image and input information to the server, which uses the Google Cloud Vision API to extract and analyze the data and automatically categorize expenses. The user reviews the analysis results and makes corrections as needed.

[1196] 3. Specific examples of financial statement preparation:

[1197] The server aggregates monthly accounting data and automatically generates a trial balance. The generated trial balance is displayed to the user via their terminal and provided in a downloadable format.

[1198] 4. Specific examples of consultations with experts:

[1199] The user enters and submits a question such as, "Why did this month's expenses exceed the budget?" The server generates a prompt and inputs it into a generating AI model (e.g., GPT-3) to automatically generate an answer. If it cannot resolve the issue, it forwards the question to an expert and notifies the user of the answer.

[1200] 5. Specific examples of the emotional engine:

[1201] While a user is using the accounting system, the camera and microphone on the communication terminal capture the user's facial expressions and voice. The server analyzes this data using the Microsoft Azure Emotion API to detect if the user is experiencing stress. As a result, the server simplifies the UI and provides accounting advice in a more friendly tone.

[1202] Through the processing steps described above, this system not only efficiently and accurately supports accounting operations but also enables flexible responses that take into account the user's emotional state.

[1203] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1204] Step 1:

[1205] The user accesses the application from their communication device and enters registration information such as their name, phone number, and email address. After entering the information, they click the registration button.

[1206] Input: Registration information such as name, phone number, and email address.

[1207] Output: Request sent to the server by the terminal

[1208] Specific operation: The user enters the required information into the input form and presses the "Register" button in the browser or application on their communication device.

[1209] Step 2:

[1210] The terminal temporarily stores the entered information and sends it to the server using the secure HTTP protocol.

[1211] Input: User-entered registration information

[1212] Output: HTTP POST request to the server

[1213] Specific operation: The terminal temporarily stores the input information in memory and then sends the data to the server via secure HTTPS communication.

[1214] Step 3:

[1215] The server verifies the received information and sends an authentication request to perform phone number authentication using the Twilio API.

[1216] Input: Registration information sent from the device

[1217] Output: Authentication request and authentication result to the Twilio API

[1218] Specific operation: A Node.js script is executed on the server side, formatting the received registration information and sending a request to the Twilio API. This performs phone number verification.

[1219] Step 4:

[1220] The server receives the authentication result, generates a unique user ID, saves it to the database, and simultaneously sends it back to the terminal.

[1221] Input: Authentication result from Twilio API

[1222] Output: Unique user ID, saved to database, and returned to terminal.

[1223] Specific operation: The server confirms successful authentication, generates a unique user ID and saves it to MongoDB, and then sends an authentication success response containing that user ID back to the terminal.

[1224] Step 5:

[1225] The device displays a "Registration successful" message and the generated user ID to the user.

[1226] Input: User ID and authentication result returned from the server.

[1227] Output: "Registration successful" message and user ID displayed.

[1228] Specific operation: The application on the terminal receives a response from the server and displays a "registration successful" message and user ID to the user in the form of a pop-up or notification.

[1229] Step 6:

[1230] The user uses a communication device to take a picture of the receipt for the purchased items and uploads it to the system. They also enter information such as the transaction date and amount into the corresponding entry field.

[1231] Input: Receipt image, transaction date, amount, and other input data.

[1232] Output: Request sent to the server by the terminal

[1233] Specific operation: The user takes a picture of the receipt using the camera on their communication device, enters the transaction date and amount into the application's form, and then clicks the "Upload" button.

[1234] Step 7:

[1235] The terminal temporarily stores the image file and input data, and then sends an HTTP POST request to the server.

[1236] Input: User-submitted receipt image and associated transaction information

[1237] Output: HTTP POST request to the server

[1238] Specific operation: The terminal temporarily stores the receipt image and input information in memory, and then sends the image and data to the server using secure HTTPS communication.

[1239] Step 8:

[1240] The server uses the Google Cloud Vision API to extract text data from the receipt image and saves it to a database.

[1241] Input: Receipt image and transaction information sent from the device.

[1242] Output: Extraction results of character data and saving to the database.

[1243] Specific operation: A Python script is executed on the server side, sending an image to the Google Cloud Vision API, converting the extracted text data into a Pandas DataFrame, and saving it to a MySQL database.

[1244] Step 9:

[1245] The server analyzes the stored data and automatically categorizes it into income and expense categories.

[1246] Input: Text data extracted from receipts

[1247] Output: Accounting data categorized by type

[1248] Specific operation: An analysis script is executed on the server side, automatically classifying the data based on income and expense categories, and updating the appropriate fields in the database with the results.

[1249] Step 10:

[1250] The server returns the analysis results to the terminal in JSON format.

[1251] Input: Accounting data categorized by type

[1252] Output: JSON response of the analysis results

[1253] Specific operation: The server formats the analysis results in JSON format and sends the response to the terminal using secure HTTPS communication.

[1254] Step 11:

[1255] The terminal displays the analysis results it has received and provides an interface for the user to review and correct them.

[1256] Input: Analysis results sent from the server

[1257] Output: Provides an interface for displaying and modifying analysis results.

[1258] Specific operation: The application on the terminal side parses the JSON response, displays the parsing results to the user, and provides an interface for correcting the data as needed.

[1259] Step 12:

[1260] The server aggregates daily and monthly accounting data and calculates total income and expenses.

[1261] Input: Accounting data in the database

[1262] Output: Total income and expenses

[1263] Specific operation: A scheduled script is executed on the server side, using Pandas to aggregate accounting data in the database and calculate daily and monthly totals of income and expenses.

[1264] Step 13:

[1265] The server automatically generates financial statements such as trial balances and financial statements based on aggregated data and saves them in a database.

[1266] Input: Daily and monthly aggregated data

[1267] Output: Automated financial statements

[1268] Specific operation: The generation script is executed on the server side, generating financial statements using Pandas and Matplotlib, and saving them in PDF format.

[1269] Step 14:

[1270] The server uses an AI algorithm based on TensorFlow to analyze accounting data and identify unusual spending patterns and potential savings.

[1271] Input: Automatically generated financial statements and accounting data

[1272] Output: Analysis results

[1273] Specific operation: The server runs an AI model, uses accounting data as input to detect anomalies and suggest cost-saving measures, analyzes the results, and saves them to a database.

[1274] Step 15:

[1275] The server returns the analysis results to the terminal in JSON format.

[1276] Input: Analysis results by AI algorithm

[1277] Output: JSON response

[1278] Specific operation: The server formats the analysis results in JSON format and sends the response to the terminal using secure HTTPS communication.

[1279] Step 16:

[1280] The device displays the analysis results to the user in graph or table format.

[1281] Input: Analysis results sent from the server

[1282] Output: Display in graph or tabular format

[1283] Specific operation: The application on the terminal receives the analysis results and displays them visually using a graph library.

[1284] Step 17:

[1285] The user inputs and sends the details of what they want to discuss with the accountant via a communication terminal.

[1286] Input: Consultation details

[1287] Output: Request sent to the server by the terminal

[1288] Specific action: The user enters their inquiry into the consultation form on their communication device and presses the "Send" button.

[1289] Step 18:

[1290] The terminal temporarily saves the entered consultation details and sends them to the server.

[1291] Input: User-entered consultation details

[1292] Output: HTTP POST request to the server

[1293] Specific operation: The terminal temporarily stores the consultation details in memory and then sends the data to the server via secure HTTPS communication.

[1294] Step 19:

[1295] The server analyzes the received inquiry and generates an automated response based on an AI model. If it cannot resolve the issue, it escalates it to an expert.

[1296] Input: Consultation content sent from the device

[1297] Output: Automated response and escalation results

[1298] Specific operation: The server uses NLP technology to analyze the consultation content and inputs it as a prompt into a generating AI model (e.g., GPT-3) to automatically generate an answer. If a solution cannot be found, the issue is escalated to an expert.

[1299] Step 20:

[1300] The server receives the answer from the expert and sends it to the communication terminal, notifying the user.

[1301] Input: Answer from an expert

[1302] Output: Response to terminal

[1303] Specific operation: After receiving a response from an expert, the server formats the content in JSON format and sends the response to the terminal using secure HTTPS communication.

[1304] Step 21:

[1305] The device presents the user with detailed answers and provides information to help them decide on their next action.

[1306] Input: Response received from the server

[1307] Output: Display to the user

[1308] Specific operation: The application on the device receives the response and displays it to the user in the form of a pop-up or notification, providing information for the next action.

[1309] Step 22:

[1310] While the user is using the accounting system, the camera and microphone on the communication terminal collect facial expressions and voice data.

[1311] Input: User's facial expressions and voice data

[1312] Output: Request sent to the server by the terminal

[1313] Specific operation: The camera and microphone built into the communication terminal capture the user's facial expressions and voice in real time, and temporarily store this data.

[1314] Step 23:

[1315] The device sends the collected data to the emotion engine for analysis.

[1316] Input: Facial expressions and voice data

[1317] Output: Emotion analysis results

[1318] Specific operation: The device sends the collected data to the Microsoft Azure Emotion API to analyze the user's emotional state in real time.

[1319] Step 24:

[1320] The server analyzes the user's emotional state and dynamically adjusts the UI / UX.

[1321] Input: Analysis results from the emotion engine

[1322] Output: UI / UX adjustment results

[1323] Specific operation: The server dynamically adjusts UI elements and tone based on emotional data to optimize the user experience.

[1324] Step 25:

[1325] The server optimizes accounting advice and alerts based on sentiment data.

[1326] Input: Sentimental data and financial data

[1327] Output: Optimized accounting advice and alerts

[1328] Specific operation: The server analyzes emotional data and generates specific accounting advice and alerts tailored to the user's emotional state.

[1329] Step 26:

[1330] The server accumulates emotional data, analyzes long-term emotional patterns, and provides personalized plans and advice.

[1331] Input: Accumulated emotional data

[1332] Output: Special plans or advice

[1333] Specific operation: The server analyzes emotional data accumulated over a long period and provides special countermeasures and advice tailored to the user's stress level.

[1334] (Application Example 2)

[1335] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[1336] Currently, many accounting systems do not take into account the emotional state of the user, leading to high levels of user stress. Furthermore, users often encounter complex operations and take considerable time to understand when performing accounting tasks. Therefore, cumbersome user interfaces contribute to increased stress. This situation highlights the need for efficient and stress-free accounting operations.

[1337] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1338] In this invention, the server includes means for the user to input and transmit registration information using a communication terminal; means for the server to receive the registration information and perform authentication via a request from a communication provider; means for the server to generate a unique user ID based on the authentication result and send it back to the terminal; means for collecting the user's facial and voice data from the communication terminal and identifying emotions; means for dynamically adjusting the user interface based on the identified emotions; and means for analyzing the identified emotion data and evaluating the user's stress level. As a result, the user can perform accounting tasks through a simplified user interface, improving work efficiency while reducing stress.

[1339] A "communication terminal" is an electronic device used for inputting, transmitting, and receiving information, and includes smartphones and head-mounted displays.

[1340] A "server" is a computing system used for storing, processing, and managing data, and it performs user registration information and sentiment data analysis.

[1341] "Registration information" refers to information necessary to identify a user, such as their name, phone number, and email address, which they enter using a communication device.

[1342] "Authentication" is the process by which a server verifies the authenticity of a user's registration information through a request from a communication provider.

[1343] A "User ID" is a unique identifier generated by the server based on the authentication result and is used to identify the user.

[1344] "Facial expression data" refers to data obtained by capturing a user's facial expressions with the camera on a communication device and analyzing it to determine the user's emotions.

[1345] "Voice data" refers to data obtained by capturing the user's voice with the microphone of a communication terminal and analyzing it to determine the user's emotions.

[1346] "Emotion recognition" is the process of analyzing a user's facial expression and voice data to identify their current emotional state.

[1347] "Dynamic adjustment" is a feature that changes elements of the user interface in real time based on the user's identified emotions.

[1348] A "user interface" is a means of display and operation for a user to interact with a system, and includes screen design and operation buttons.

[1349] "Emotional data" refers to data that indicates the user's emotional state, obtained through emotion recognition.

[1350] "Stress level" refers to the degree of stress a user is experiencing, evaluated based on emotional data.

[1351] "Image recognition technology" is a technology that analyzes information such as characters from uploaded images and converts it into text data.

[1352] This invention is a system that streamlines and improves the accuracy of accounting operations by allowing users to input and transmit registration information using a communication terminal, which is then received by a server. Furthermore, this system provides a function to analyze the user's emotional state and dynamically adjust the user interface based on that analysis.

[1353] Hardware and software to be used

[1354] Hardware: Smartphones, head-mounted displays (HMDs)

[1355] Software: Flask (Python-based web framework), SQLite (database system), external authentication, image recognition, and sentiment analysis service APIs.

[1356] Program Processing Overview

[1357] User-performed processes

[1358] Users first access the system using a communication terminal and enter registration information such as their name, phone number, and email address. This information is sent to the server, where authentication is performed via the communication provider. If authentication is successful, the server generates a unique user ID and sends it back to the terminal. From then on, users can log in to the system using this user ID.

[1359] Uploading and managing receipts

[1360] Users take a picture of their purchase receipt with a communication device and upload it to the system. The server receives this receipt image and uses image recognition technology to convert it into text data (e.g., transaction date, item, amount). This data is stored in a database and automatically categorized. The analysis results are displayed on the device, allowing the user to review and modify them.

[1361] Aggregation and analysis of accounting data

[1362] The server aggregates daily and monthly accounting data and calculates total income and expenses. It automatically generates financial documents such as trial balances and financial statements and stores them in a database. Furthermore, the server uses AI algorithms to analyze the accounting data, identifying unusual spending patterns and potential savings. These analysis results are transmitted to the communication terminal and displayed visually to the user.

[1363] Emotional engine integration

[1364] When a user uses the camera and microphone built into their communication device to collect facial and voice data, it is sent to a server. The server analyzes the user's emotions using an emotion analysis algorithm and dynamically adjusts the user interface based on the results. For example, if the user is feeling stressed, the UI is simplified and accounting advice is provided in a gentler tone.

[1365] Examples of usage and prompt statements

[1366] User registration usage example

[1367] The user accesses the new registration page from their communication device, enters the required information, and clicks the registration button. The server performs authentication, generates a user ID, and sends it back to the device.

[1368] Prompt example: "A new user is attempting to register with the name 'User A', phone number '090-1234-5678', and email address 'user@example.com'."

[1369] Example of using receipt entry

[1370] Users take photos of their supermarket shopping receipts and upload them. The server uses image recognition technology to extract and analyze the data, automatically classifying expenses.

[1371] Example prompt: "The user has taken a picture of a receipt and uploaded it. Extract the date, item, and amount from the image."

[1372] Examples of using an emotion engine

[1373] While a user is using the accounting system, the camera and microphone on the communication terminal capture the user's facial expressions and voice. The server analyzes this data to detect stress and simplifies the UI to make it easier to use.

[1374] Example prompt: "Analyze data from the user's camera and microphone to assess their stress level. If stress is high, simplify the user interface."

[1375] Through the processing steps described above, this system not only efficiently and accurately supports accounting operations but also enables flexible responses that take into account the user's emotional state.

[1376] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1377] Step 1:

[1378] The user accesses the new registration page from their communication device, enters registration information such as their name, phone number, and email address, and submits it. This entered data is temporarily stored by the device and then sent to the server.

[1379] Input: Name, phone number, email address

[1380] Output: Registration information sent to the server

[1381] Step 2:

[1382] The server authenticates the received registration information via the communication provider's API. It then retrieves the authentication result from the communication provider and generates a unique user ID based on it.

[1383] Input: Registration information (name, phone number, email address)

[1384] Data processing: Authentication using the communication provider's API

[1385] Output: Authentication result, unique user ID

[1386] Step 3:

[1387] The server sends a generated user ID back to the terminal, and the terminal displays the authentication result and user ID to the user. The user uses this user ID for future logins.

[1388] Input: Unique User ID

[1389] Output: Authentication result and user ID displayed on the terminal

[1390] Step 4:

[1391] The user uses a communication device to take a picture of the receipt related to the transaction, enters relevant information (transaction date, amount, etc.) into the corresponding entry field, and uploads it to the system.

[1392] Input: Receipt image, transaction date, amount

[1393] Output: Receipt image and related information sent to the server

[1394] Step 5:

[1395] The server converts the received receipt image into text data using image recognition technology. The converted data is then stored in a database.

[1396] Input: Receipt image

[1397] Data processing: Text conversion using image recognition technology

[1398] Output: Converted text data

[1399] Step 6:

[1400] The server analyzes the stored data and automatically categorizes it into income and expense categories. The analysis results are sent to the terminal, where the user can review and modify them.

[1401] Input: Text data

[1402] Data processing: Automated classification algorithms

[1403] Output: Automated classification of analysis results

[1404] Step 7:

[1405] The user collects facial and audio data using the camera and microphone built into their communication terminal. This data is then transmitted from the terminal to a server.

[1406] Input: Facial expression data, audio data

[1407] Output: Facial expression and audio data sent to the server

[1408] Step 8:

[1409] The server uses an emotion analysis algorithm to analyze the user's emotional state and dynamically adjusts the user interface based on the analysis results.

[1410] Input: Facial expression data, audio data

[1411] Data processing: Sentiment analysis algorithm

[1412] Output: Adjusted user interface

[1413] Step 9:

[1414] The server accumulates long-term emotional data and evaluates user stress levels. Based on the evaluation, it generates data to provide special plans and advice.

[1415] Input: Analyzed sentiment data

[1416] Data processing: Stress level evaluation algorithm

[1417] Output: Special plans and advice provided to the user.

[1418] Through these processing steps, this system not only efficiently and accurately supports accounting operations but also enables flexible responses that take into account the user's emotional state.

[1419] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1420] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include those described above. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions shown by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1421] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[1422] [Third Embodiment]

[1423] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[1424] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1425] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1426] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[1427] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1428] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1429] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1430] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1431] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1432] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1433] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1434] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[1435] This invention is a system that improves the efficiency and accuracy of accounting operations by having a server receive and process data entered by a user using a communication terminal. The details of this system program and specific examples of its processing are shown below.

[1436] Program processing and its explanation

[1437] User registration and authentication

[1438] 1. The user accesses the application from their communication device and enters registration information such as their name, phone number, and email address. After entering the information, they click the registration button.

[1439] 2. The terminal temporarily stores the entered information and sends it to the server.

[1440] 3. The server verifies the received registration information via the communication provider's API and performs authentication.

[1441] 4. The server generates a unique user ID based on the authentication result and sends it back to the terminal.

[1442] 5. The terminal displays the authentication result and the generated user ID to the user and provides login information for future logins.

[1443] Data entry and management

[1444] 1. The user uses a communication device to take a picture of the receipt for the purchased items and uploads it to the system. They also enter relevant information (transaction date, amount, etc.) into the corresponding entry field.

[1445] 2. The device sends the uploaded image and input information to the server.

[1446] 3. The server uses image recognition technology to extract text data (such as transaction date, item, and amount) from the receipt image and saves it to the database.

[1447] 4. The server analyzes the stored data and automatically categorizes it into income and expense categories.

[1448] 5. The terminal displays the analysis results to the user and provides an interface for the user to review and correct them.

[1449] Accounting and Analysis

[1450] 1. The server aggregates daily and monthly accounting data and calculates total income and expenses.

[1451] 2. The server automatically generates financial statements such as trial balances and financial statements and saves them in the database.

[1452] 3. The server uses AI algorithms to analyze accounting data and identify unusual spending patterns and potential savings.

[1453] 4. The device displays the analysis results to the user in a visually easy-to-understand format (graphs and tables).

[1454] Collaboration with accountants

[1455] 1. The user enters and sends the details of what they want to discuss with the accountant via a communication terminal.

[1456] 2. The terminal sends the entered consultation details to the server.

[1457] 3. The server analyzes the received inquiry and generates an automated response using AI. If the issue cannot be resolved, it escalates to an accountant.

[1458] 4. The server sends the response received from the accountant to the communication terminal and notifies the user.

[1459] 5. The device presents the user with detailed answers and provides information to help the user decide on their next action.

[1460] Specific example

[1461] 1. Specific Examples of User Registration and Authentication

[1462] The user accesses the new registration page from their communication device, enters the required information, and clicks the registration button.

[1463] When the terminal sends input information to the server, the server uses the communication provider's API to perform authentication, generate a user ID, and send it back.

[1464] The device displays the authentication result to the user, notifying them of a success message and user ID.

[1465] 2. Specific examples of receipt entry

[1466] Users take a picture of their supermarket shopping receipt with a communication device and upload it.

[1467] The terminal sends the receipt image and input information to the server, which uses image recognition technology to extract and analyze the data and automatically classify the expenses.

[1468] The user reviews the analysis results and makes corrections as needed.

[1469] 3. Specific Examples of Financial Statement Preparation

[1470] The server aggregates monthly accounting data and automatically creates a trial balance.

[1471] The terminal displays the generated trial balance to the user and provides it in a downloadable format.

[1472] 4. Specific examples of conversations with accountants

[1473] The user enters and submits their inquiry regarding year-end financial settlement procedures.

[1474] The server performs a basic analysis, and if it cannot provide an automated answer, it forwards the request to an accountant.

[1475] The accountant enters the appropriate answer, and the server notifies the user of that answer.

[1476] As described above, the present invention provides a system that offers total support for accounting operations and significantly reduces the burden on users. This system enables users to perform accounting operations more efficiently and accurately.

[1477] The following describes the processing flow.

[1478] User registration and authentication

[1479] Step 1:

[1480] The user accesses the application from their communication device and enters registration information such as their name, phone number, and email address. They then click the registration button.

[1481] Step 2:

[1482] The terminal temporarily stores the entered information and sends the registration information to the server.

[1483] Step 3:

[1484] The server sends the received registration information to the communication provider's API for authentication.

[1485] Step 4:

[1486] The server generates a unique user ID based on the authentication result and sends it back to the terminal.

[1487] Step 5:

[1488] The terminal displays the authentication result and the generated user ID to the user, and provides login information for future logins.

[1489] Data entry and management

[1490] Step 1:

[1491] The user uses a communication device to take a picture of the receipt for the purchased items and uploads it to the system. They also enter relevant information (transaction date, amount, etc.) into the corresponding entry field.

[1492] Step 2:

[1493] The device sends the uploaded image and input information to the server.

[1494] Step 3:

[1495] The server uses image recognition technology to extract text data (such as transaction date, item, and amount) from the receipt image and saves it to a database.

[1496] Step 4:

[1497] The server analyzes the stored data and automatically categorizes it into income and expense categories.

[1498] Step 5:

[1499] The server sends the analysis results to the terminal.

[1500] Step 6:

[1501] The terminal displays the analysis results to the user and provides an interface for the user to review and make corrections.

[1502] Accounting and Analysis

[1503] Step 1:

[1504] The server aggregates daily and monthly accounting data and calculates total income and expenses.

[1505] Step 2:

[1506] The server automatically generates financial statements such as trial balances and financial statements and saves them in a database.

[1507] Step 3:

[1508] The server uses AI algorithms to analyze accounting data and identify unusual spending patterns and potential savings.

[1509] Step 4:

[1510] The server sends the analysis results to the terminal.

[1511] Step 5:

[1512] The device displays the analysis results to the user in a visually easy-to-understand format (graphs and tables).

[1513] Collaboration with accountants

[1514] Step 1:

[1515] The user inputs and sends the details of what they want to discuss with the accountant via a communication terminal.

[1516] Step 2:

[1517] The terminal sends the entered consultation details to the server.

[1518] Step 3:

[1519] The server analyzes the received inquiry and generates an automated response using AI. If the issue cannot be resolved, it is escalated to an accountant.

[1520] Step 4:

[1521] The server receives the response from the accountant and sends it to the communication terminal, notifying the user.

[1522] Step 5:

[1523] The device presents the user with detailed answers and provides information to help the user decide on their next action.

[1524] Through the above processing steps, this system can efficiently and accurately support accounting operations.

[1525] (Example 1)

[1526] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1527] For many small and medium-sized enterprises (SMEs) and sole proprietors, accounting is a time-consuming and difficult task to maintain accuracy in. Manual data entry and ledger management are time-consuming and costly, and there is a need for more efficient methods. Furthermore, advanced processing such as the automatic generation of necessary financial statements, data analysis, and identification of abnormal expenditure patterns are required, demanding specialized knowledge and skills. Against this backdrop, there is a challenge in building a system that can manage accounting tasks easily and accurately, and automatically generate necessary ledgers and financial statements.

[1528] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1529] In this invention, the server includes means for the user to input and transmit registration information using a communication terminal, means for authentication via a communication network interface, and means for generating a unique identification number based on the authentication result and returning it to the terminal. This enables the user to complete the registration process easily and accurately. Furthermore, the server includes means for the user to upload transaction-related images using a communication terminal, means for converting them into text data using image processing technology, means for analyzing and automatically classifying the text data, and means for transmitting the analysis results to the communication terminal for the user to confirm and correct. This enables the user to efficiently input and manage transaction information. In addition, the server includes means for automatically generating financial reports based on aggregated accounting data, means for transmitting the automatically generated financial reports to the communication terminal for the user to confirm, and means for analyzing accounting data using a machine learning algorithm and transmitting the analysis results to the communication terminal. This enables the user to easily create accurate financial reports and analyze accounting data.

[1530] A "user" is an individual or legal entity that uses a communication terminal to operate the system and input and verify necessary data.

[1531] A "communication terminal" is an electronic device that is connected to the internet and operated by the user, and specifically includes smartphones, tablets, and personal computers.

[1532] A "server" is a central computer system that receives data sent by users, processes it, and returns the results.

[1533] "Registration information" refers to personally identifiable information such as name, telephone number, and email address that the user provides to the system.

[1534] A "communication network interface" is a means of connection that allows a server to exchange data with external communication providers or other systems.

[1535] "Authentication" is the process of verifying the validity of registration information entered by a user via a communication network interface.

[1536] A "unique identification number" is a unique ID generated by the server to individually identify a user.

[1537] "Transaction-related images" refer to image files that users upload to the system, such as receipts and invoices, that represent transaction records.

[1538] "Image processing technology" refers to techniques for extracting text data from images, and specifically includes optical character recognition (OCR).

[1539] "Text data" refers to character information extracted from images related to a transaction using image processing technology.

[1540] "Automatic classification" is the process by which a server categorizes the text data it has analyzed.

[1541] "Accounting data" refers to transaction information entered by users and income and expense data compiled by the server.

[1542] A "financial report" is a document that shows the financial status, such as an income statement and balance sheet, which is automatically generated by a server based on accounting data.

[1543] A "machine learning algorithm" is an AI technology used by servers to analyze accounting data, identifying data patterns and performing predictions and classifications.

[1544] This invention relates to a system in which a server receives data entered by a user using a communication terminal, thereby improving the efficiency and accuracy of accounting operations. This system is implemented according to the following procedure.

[1545] User registration and authentication

[1546] The user accesses the application using a communication device (e.g., smartphone, tablet, PC) and enters registration information such as name, phone number, and email address. Once the user submits the information, the device temporarily stores this information and sends it to the server via an HTTPS request. The server receives this registration information through the communication network interface and uses the communication provider's API to perform authentication to verify the legitimacy of the information. If authentication is successful, the server generates a unique identification number and sends it back to the device. The device then displays the authentication result and the generated identification number to the user.

[1547] Data entry and management

[1548] The user takes a picture of the receipt for their purchased items using a communication terminal and uploads it to the system. The terminal sends this receipt image and related information such as the transaction date and amount entered by the user to the server. The server uses image processing technology (specifically, optical character recognition technology - OCR) to extract text data (transaction date, items, amount, etc.) from the receipt image. The extracted text data is analyzed by the server and automatically categorized into income and expense categories. The terminal displays the analysis results to the user and provides an interface that allows the user to review the results and make corrections as needed.

[1549] Accounting and Analysis

[1550] The server aggregates daily and monthly accounting data and calculates total income and expenses. Furthermore, the server automatically generates financial reports such as trial balances and financial statements, and stores them in a database. These generated financial reports are displayed to the user via a terminal. The server also uses machine learning algorithms to analyze accounting data, identifying unusual spending patterns and potential savings. The analysis results are also displayed to the user via the terminal in a visually easy-to-understand format (graphs and tables).

[1551] Collaboration with accountants

[1552] The user uses a communication terminal to input the details of their inquiry to an accountant and sends them to the system. The terminal sends this inquiry to a server, which analyzes the received information and generates an automated response using AI. Inquiries that cannot be resolved are escalated to an accountant. The accountant's response is sent back to the terminal via the server and notified to the user. The terminal presents the user with detailed information about the response and provides information to help them decide on their next course of action.

[1553] Examples and prompts for generative AI models

[1554] Next, we will show specific examples of operation and prompt messages.

[1555] 1. Specific Examples of User Registration and Authentication

[1556] The user accesses the new registration page from their communication device, enters the required information, and clicks the registration button.

[1557] The terminal sends input information to the server, the server uses the communication provider's API to perform authentication, generates a user ID, and sends it back.

[1558] The device displays the authentication result to the user, notifying them of a success message and user ID.

[1559] Prompts for Generative AI Models

[1560] Please explain the procedure for new user registration in the following format.

[1561] 1. The user opens the application and enters the required information.

[1562] 2. Send the input information to the server.

[1563] 3. The server authenticates the information and generates a user ID.

[1564] 4. Notify the user of the authentication result and user ID.

[1565] 2. Specific examples of receipt entry

[1566] Users take a picture of their supermarket shopping receipt with a communication device and upload it.

[1567] The terminal sends receipt images and input information to the server, which uses image recognition technology to extract and analyze the data and automatically classify the expenses.

[1568] The user reviews the analysis results and makes corrections as needed.

[1569] Prompts for Generative AI Models

[1570] Please explain the procedure for entering receipts at a supermarket in the following format.

[1571] 1. The user takes a picture of the receipt and uploads it.

[1572] 2. Send the image and information to the server.

[1573] 3. The server extracts, analyzes, and classifies the data.

[1574] 4. Display the results to the user for review and correction.

[1575] 3. Specific Examples of Financial Report Preparation

[1576] The server aggregates monthly accounting data and automatically creates a trial balance.

[1577] The terminal displays the generated trial balance to the user and provides it in a downloadable format.

[1578] Prompts for Generative AI Models

[1579] Please describe the procedure for creating a financial report from monthly accounting data in the following format.

[1580] 1. The server aggregates the data.

[1581] 2. Generate financial reports such as trial balances.

[1582] 3. Display the results to the user and provide them in a downloadable format.

[1583] 4. Specific examples of conversations with accountants

[1584] The user enters and submits their inquiry regarding year-end financial settlement procedures.

[1585] The server performs a basic analysis, and if it cannot provide an automated answer, it forwards the request to an accountant.

[1586] The accountant enters the appropriate answer, and the server notifies the user of that answer.

[1587] Prompts for Generative AI Models

[1588] Please describe the procedure for consulting with an accountant in the following format.

[1589] 1. The user enters their inquiry and submits it.

[1590] 2. The server analyzes the content and uses AI to automatically generate a response.

[1591] 3. If an automated response is not possible, the request will be forwarded to an accountant.

[1592] 4. The accountant enters the answer and notifies the user.

[1593] In this way, the present invention provides a system that allows users to perform accounting tasks easily and efficiently via a communication terminal. This improves the accuracy and efficiency of accounting tasks and significantly reduces the burden on users.

[1594] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1595] Step 1:

[1596] The user accesses the application from their communication device and enters registration information such as their name, phone number, and email address. Once the user has finished entering the information, they click the "Register" button. The entered data consists of personal information such as name, phone number, and email address.

[1597] Step 2:

[1598] The terminal temporarily stores the entered registration information and sends it to the server using an HTTPS request. The input here is the registration information provided by the user, and the output is an HTTPS request containing this information.

[1599] Step 3:

[1600] The server authenticates the received registration information via the provider API using the communication network interface. The input is the registration information from the terminal, which the server sends to the external API and receives the authentication result. The output is the authentication result.

[1601] Step 4:

[1602] The server generates a unique identification number (user ID) based on the authentication result and sends that ID back to the terminal. The input is the authentication result, and the output is the unique identification number (user ID).

[1603] Step 5:

[1604] The terminal displays the authentication result and the generated user ID to the user. The input is the identification number and authentication result from the server, and the output is the information displayed to the user. For example, a message such as "Registration complete. Your user ID is 123456."

[1605] Step 6:

[1606] The user takes a picture of the receipt for the purchased item using a communication device and enters relevant information such as the transaction date and amount. They then click the "Upload" button to confirm the image data and entered information. The input data consists of the receipt image and transaction information.

[1607] Step 7:

[1608] The device sends the uploaded receipt image and input information to the server via an HTTPS request. The input consists of the image and transaction information provided by the user to the device, and the output is the request sent to the server.

[1609] Step 8:

[1610] The server uses image recognition technology to extract text data (such as transaction date, item, and amount) from a receipt image. The input is a receipt image sent from the terminal, and the output is the extracted text data.

[1611] Step 9:

[1612] The server analyzes the extracted text data, automatically categorizes it into income and expense categories, and stores it in a database. The input is text data after OCR processing, and the output is categorized database entries.

[1613] Step 10:

[1614] The terminal displays the analysis results to the user, providing an interface that allows the user to review the results and make corrections as needed. The input is the analysis results sent from the server, and the output is the display information and correction interface for the user.

[1615] Step 11:

[1616] The server aggregates daily and monthly accounting data and calculates total income and expenses. The input is transaction information stored in the database, and the output is aggregated income and expense data.

[1617] Step 12:

[1618] The server automatically generates financial reports such as trial balances and financial statements and saves them to the database. The input is aggregated income and expenditure data, and the output is the generated financial reports.

[1619] Step 13:

[1620] The terminal displays the generated financial report to the user and provides it in a downloadable format as needed. The input is the financial report from the server, and the output is a display and download link for the user.

[1621] Step 14:

[1622] The server uses machine learning algorithms to analyze accounting data and identify unusual spending patterns and potential savings. The input is historical accounting data in the database, and the output is the analysis results and their insights.

[1623] Step 15:

[1624] The terminal displays the analysis results to the user in a visually easy-to-understand format (graphs and tables). The input is the analysis results from the server, and the output is displayed in graph and table format.

[1625] Step 16:

[1626] The user uses a communication terminal to input the details of their consultation with an accountant and sends them to the system. The input is the user's consultation content, and the output is the request sent to the server.

[1627] Step 17:

[1628] The terminal sends the entered consultation details to the server. The input is the user's consultation details, and the output is an HTTPS request.

[1629] Step 18:

[1630] The server analyzes the received inquiry and generates an automated response using AI. If the issue cannot be resolved, it escalates it to an accountant. The input is the user's inquiry, and the output is either an AI-generated response or an escalation request to an accountant.

[1631] Step 19:

[1632] The accountant enters the appropriate answer, the server sends that answer to the communication terminal, and the user is notified. The input is the accountant's answer, and the output is the notification to the user.

[1633] Step 20:

[1634] The terminal presents the user with detailed answers and provides information to help the user decide on their next action. The input is the detailed answers from the server, and the output is the information presented to the user.

[1635] (Application Example 1)

[1636] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1637] Modern brick-and-mortar stores demand efficient and accurate accounting and inventory management. However, traditional methods often involve manual processes, leading to frequent input errors and delays in data updates, thus hindering improvements in efficiency and accuracy. Furthermore, users spend a significant amount of time managing accounting data, and real-time inventory management remains challenging.

[1638] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1639] In this invention, the server includes means for a user to input and transmit registration information using a communication terminal, means for the server to receive the registration information and perform authentication via a request from a communication provider, means for the server to generate a unique user ID based on the authentication result and send it back to the terminal, means for the user to input transaction information at a physical store using a communication terminal, means for the user to take a picture of a receipt using smart glasses and send it to the server, means for the server to convert the receipt image into text data using image recognition technology and automatically classify it, and means for the server to update inventory data in real time based on the text data. This makes it possible to improve the efficiency and accuracy of accounting operations and inventory management at physical stores.

[1640] A "communication terminal" is a device used by a user to input and transmit information, and includes smartphones, tablets, and personal computers.

[1641] A "server" is a central computer system that receives data sent by users and performs authentication, data analysis, and storage.

[1642] "Authentication" is the process of verifying a user's registration information through a request from the communication provider to confirm that they are a legitimate user.

[1643] A "User ID" is a unique string or number generated by the server, used to uniquely identify a user.

[1644] A "physical store" is a commercial facility that provides goods and services in a physical location.

[1645] "Transaction information" refers to detailed data about the purchase of goods or services, including the transaction date, item, and amount.

[1646] "Smart glasses" are devices that allow the wearer to collect information about their environment and utilize display and camera functions.

[1647] A "receipt image" is an image of a paper receipt that shows the details of the purchased items.

[1648] "Image recognition technology" is a computer vision technology that extracts text and objects from images and treats them as data.

[1649] "Text data" refers to character information extracted using image recognition technology and stored in digital format.

[1650] "Automatic classification" is the process of automatically sorting analyzed data into specific categories.

[1651] "Inventory data" refers to the quantity and detailed information of products held by a physical store.

[1652] "Real-time" refers to data processing and updates occurring immediately at the moment an event takes place.

[1653] "Accounting work" refers to all tasks involving the recording, aggregation, and analysis of transactions, including the preparation of financial statements.

[1654] This invention relates to a system that streamlines and improves the accuracy of accounting and inventory management in physical stores. Specific embodiments are described below.

[1655] System Configuration

[1656] 1. Communication terminal

[1657] A device used by users to input and transmit information. This includes smartphones, tablets, and personal computers.

[1658] 2. Server

[1659] This is a central computer system that receives data sent from users and performs authentication, data analysis, and storage. The server is responsible for authenticating registration information, generating unique user IDs, aggregating accounting data, and performing AI-based data analysis.

[1660] 3. Smart Glasses

[1661] This device allows the wearer to collect information about their environment and utilize its display and camera functions. It is used for taking pictures of receipts and scanning barcodes.

[1662] Specific examples of hardware and software

[1663] Communication terminal:

[1664] Smartphone: iPhone or Android device

[1665] Computers and Tablets: Windows PCs, Macs, iPads, etc.

[1666] server:

[1667] Cloud servers: AWS (Amazon Web Services), Google Cloud Platform, Microsoft Azure, etc.

[1668] Software: Python, TensorFlow (used for implementing AI algorithms), OpenCV (for image recognition)

[1669] Smart glasses:

[1670] Devices: Google Glass, Vuzix, etc.

[1671] Contents of data processing and data calculations

[1672] User authentication:

[1673] 1. The user enters and submits registration information (name, phone number, email address, etc.) from their communication terminal.

[1674] 2. The server authenticates the received information via the communication provider's API, generates a unique user ID, and sends it back.

[1675] Analysis and classification of receipt images:

[1676] 1. The user uses smart glasses to take a picture of the receipt as transaction information and sends it to the server via a communication terminal.

[1677] 2. The server uses image recognition technology (using OpenCV) to extract text data from the receipt image.

[1678] 3. The extracted data is analyzed and automatically classified.

[1679] Real-time updates of inventory data:

[1680] 1. The server updates inventory data in real time based on text data obtained from the scanned receipts.

[1681] Specific example

[1682] A store employee wears smart glasses and scans the barcode of the items purchased by the customer. After issuing a receipt, the employee takes a picture of the receipt with the smart glasses' camera, and the data is sent to a server. The server uses image recognition technology to analyze the receipt information and updates inventory information in real time.

[1683] Example of a prompt

[1684] "Create an application that uses smart glasses to scan barcodes of items purchased by customers and take photos of receipts. This application sends the scanned product data and the captured receipt images to a server, where the server analyzes the data and updates inventory information in real time."

[1685] This invention improves the efficiency and accuracy of accounting operations and inventory management in physical stores, significantly reducing the burden on users.

[1686] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1687] Step 1:

[1688] The user enters and submits registration information (name, phone number, email address) using a communication terminal. The information entered by the user is temporarily stored on the communication terminal and then sent to the server. The registration information is used as input data, and as a result, the registration information is sent to the server.

[1689] Step 2:

[1690] The server uses the communication provider's API to perform authentication based on the registration information it receives. The server sends a request to the API and receives the authentication result from the communication provider. The registration information is used as input data, and the authentication result is obtained as output data.

[1691] Step 3:

[1692] The server generates a unique user ID based on the authentication result and sends that user ID back to the terminal. The server verifies the authentication result and uses an algorithm to generate the user ID. The authentication result is used as input data, and the user ID is generated as output data.

[1693] Step 4:

[1694] The user takes a picture of the receipt for the purchased items using smart glasses and sends it to the server via a communication terminal. The user takes a picture of the receipt using the camera on the smart glasses and saves it to the communication terminal. The receipt image is used as input data, and as a result, the receipt image is sent to the server.

[1695] Step 5:

[1696] The server converts the received receipt image into text data using image recognition technology. The server uses image recognition software such as OpenCV to extract text information from the image. A receipt image is used as input data, and text data is generated as output data.

[1697] Step 6:

[1698] The server analyzes the converted text data and automatically classifies the purchase information. The server uses an analysis algorithm to categorize the text data into income and expense categories. Text data is used as input, and the classified data is obtained as output.

[1699] Step 7:

[1700] The server updates inventory data in real time based on the analysis results. The server accesses the database and updates the inventory by subtracting the number of sold items. Classified purchase information is used as input data, and updated inventory data is obtained as output data.

[1701] Step 8:

[1702] The server automatically generates financial statements based on the aggregated accounting data and sends them to the user's communication terminal. The server uses the collected data to generate trial balances and financial statements and sends them to the communication terminal. The classified accounting information is used as input data, and the generated financial statements are obtained as output data.

[1703] Step 9:

[1704] The server uses an AI algorithm to analyze accounting data and sends the analysis results to a communication terminal. The server uses a generative AI model to identify unusual spending patterns and potential savings. Accounting data is used as input data, and the analysis results are obtained as output data.

[1705] The above processing steps improve the efficiency and accuracy of accounting operations and inventory management in physical stores.

[1706] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1707] This invention provides a system that improves the efficiency and accuracy of accounting operations by having a server receive and process data entered by a user using a communication terminal. Furthermore, by combining it with an emotion engine, it provides a function that allows the system to respond dynamically based on the user's emotions.

[1708] Program processing and its explanation

[1709] User registration and authentication

[1710] 1. The user accesses the application from their communication device and enters registration information such as their name, phone number, and email address. After entering the information, they click the registration button.

[1711] 2. The terminal temporarily stores the entered information and sends it to the server.

[1712] 3. The server sends the received registration information to the communication provider's API for authentication.

[1713] 4. The server generates a unique user ID based on the authentication result and sends it back to the terminal.

[1714] 5. The terminal displays the authentication result and the generated user ID to the user and provides login information for future logins.

[1715] Data entry and management

[1716] 1. The user uses a communication device to take a picture of the receipt for the purchased items and uploads it to the system. They also enter relevant information (transaction date, amount, etc.) into the corresponding entry field.

[1717] 2. The device sends the uploaded image and input information to the server.

[1718] 3. The server uses image recognition technology to extract text data (such as transaction date, item, and amount) from the receipt image and saves it to the database.

[1719] 4. The server analyzes the stored data and automatically categorizes it into income and expense categories.

[1720] 5. The server sends the analysis results to the terminal.

[1721] 6. The terminal displays the analysis results to the user and provides an interface for the user to review and correct them.

[1722] Accounting and Analysis

[1723] 1. The server aggregates daily and monthly accounting data and calculates total income and expenses.

[1724] 2. The server automatically generates financial statements such as trial balances and financial statements and saves them in the database.

[1725] 3. The server uses AI algorithms to analyze accounting data and identify unusual spending patterns and potential savings.

[1726] 4. The server sends the analysis results to the terminal.

[1727] 5. The device displays the analysis results to the user in a visually easy-to-understand format (graphs and tables).

[1728] Collaboration with accountants

[1729] 1. The user enters and sends the details of what they want to discuss with the accountant via a communication terminal.

[1730] 2. The terminal sends the entered consultation details to the server.

[1731] 3. The server analyzes the received inquiry and generates an automated response using AI. If it cannot resolve the issue, it escalates it to an accountant.

[1732] 4. The server sends the response received from the accountant to the communication terminal and notifies the user.

[1733] 5. The device presents the user with detailed answers and provides information to help the user decide on their next action.

[1734] Emotional engine integration

[1735] 1. The system collects facial expressions and voice data using the camera and microphone built into the user's communication terminal.

[1736] 2. The device sends the collected data to the emotion engine, which analyzes the user's emotional state.

[1737] 3. The server receives emotion data recognized by the emotion engine and dynamically adjusts the UI / UX based on it.

[1738] 4. The server optimizes the content and timing of accounting advice and alerts based on the analyzed sentiment data.

[1739] 5. The server accumulates emotional data, analyzes long-term emotional patterns, and provides personalized plans and advice tailored to the user's stress level.

[1740] Specific example

[1741] 1. Specific Examples of User Registration and Authentication

[1742] The user accesses the new registration page from their communication device, enters the required information, and clicks the registration button.

[1743] When the terminal sends input information to the server, the server uses the communication provider's API to perform authentication, generate a user ID, and send it back.

[1744] The device displays the authentication result to the user, notifying them of a success message and user ID.

[1745] 2. Specific examples of receipt entry

[1746] Users take a picture of their supermarket shopping receipt with a communication device and upload it.

[1747] The terminal sends the receipt image and input information to the server, which uses image recognition technology to extract and analyze the data and automatically classify the expenses.

[1748] The user reviews the analysis results and makes corrections as needed.

[1749] 3. Specific Examples of Financial Statement Preparation

[1750] The server aggregates monthly accounting data and automatically creates a trial balance.

[1751] The terminal displays the generated trial balance to the user and provides it in a downloadable format.

[1752] 4. Specific examples of consultations with an accountant

[1753] The user enters and submits their inquiry regarding year-end financial settlement procedures.

[1754] The server performs a basic analysis, and if it cannot provide an automated answer, it forwards the request to an accountant.

[1755] The accountant enters the appropriate answer, and the server notifies the user of that answer.

[1756] 5. Specific Examples of Emotional Engines

[1757] While the user is using the accounting system, the camera and microphone on the communication terminal capture the user's facial expressions and voice.

[1758] The device analyzes the captured data using an emotion engine to detect when the user is experiencing stress.

[1759] Based on the emotion data detected by the server, measures such as simplifying the system's UI will be taken.

[1760] The server will attempt to reduce user stress by providing accounting advice in a gentler tone.

[1761] Through the processing steps described above, this system not only efficiently and accurately supports accounting operations but also enables flexible responses that take into account the user's emotional state.

[1762] The following describes the processing flow.

[1763] User registration and authentication

[1764] Step 1:

[1765] The user accesses the application from their communication device and enters registration information such as their name, phone number, and email address. After entering the information, they click the registration button.

[1766] Step 2:

[1767] The terminal temporarily stores the entered information and sends it to the server.

[1768] Step 3:

[1769] The server sends the received registration information to the communication provider's API for authentication.

[1770] Step 4:

[1771] The server generates a unique user ID based on the authentication result and sends it back to the terminal.

[1772] Step 5:

[1773] The terminal displays the authentication result and the generated user ID to the user, and provides login information for future logins.

[1774] Data entry and management

[1775] Step 1:

[1776] The user uses a communication device to take a picture of the receipt for the purchased items and uploads it to the system. They also enter relevant information (transaction date, amount, etc.) into the corresponding entry field.

[1777] Step 2:

[1778] The device sends the uploaded image and input information to the server.

[1779] Step 3:

[1780] The server uses image recognition technology to extract text data (such as transaction date, item, and amount) from the receipt image and saves it to a database.

[1781] Step 4:

[1782] The server analyzes the stored data and automatically categorizes it into income and expense categories.

[1783] Step 5:

[1784] The server sends the analysis results to the terminal.

[1785] Step 6:

[1786] The terminal displays the analysis results to the user and provides an interface for the user to review and make corrections.

[1787] Accounting and Analysis

[1788] Step 1:

[1789] The server aggregates daily and monthly accounting data and calculates total income and expenses.

[1790] Step 2:

[1791] The server automatically generates financial statements such as trial balances and financial statements and saves them in a database.

[1792] Step 3:

[1793] The server uses AI algorithms to analyze accounting data and identify unusual spending patterns and potential savings.

[1794] Step 4:

[1795] The server sends the analysis results to the terminal.

[1796] Step 5:

[1797] The device displays the analysis results to the user in a visually easy-to-understand format (graphs and tables).

[1798] Collaboration with accountants

[1799] Step 1:

[1800] The user inputs and sends the details of what they want to discuss with the accountant via a communication terminal.

[1801] Step 2:

[1802] The terminal sends the entered consultation details to the server.

[1803] Step 3:

[1804] The server analyzes the received inquiry and generates an automated response using AI. If the issue cannot be resolved, it is escalated to an accountant.

[1805] Step 4:

[1806] The accountant enters their answers to the user's questions using a dedicated accountant terminal and sends them to the server.

[1807] Step 5:

[1808] The server receives the response from the accountant and sends it to the communication terminal, notifying the user.

[1809] Step 6:

[1810] The device presents the user with detailed answers and provides information to help the user decide on their next action.

[1811] Emotional engine integration

[1812] Step 1:

[1813] The system collects the user's facial expressions and voice data using the camera and microphone built into the communication terminal. If the user does not perform any specific actions, the system periodically captures data.

[1814] Step 2:

[1815] The device collects facial and voice data and transmits it to an emotion engine in real time to analyze the user's emotional state.

[1816] Step 3:

[1817] The server receives emotional data generated by the emotion engine and dynamically adjusts the UI / UX based on it.

[1818] Step 4:

[1819] The server optimizes the content and timing of accounting advice and alerts based on the user's emotional state. For example, if the user is feeling stressed, it will provide more helpful messages and advice.

[1820] Step 5:

[1821] The server tracks the user's emotional data over a long period and provides special plans and advice based on stress levels and emotional patterns. For example, if a user is consistently experiencing high stress levels, it will provide advice on relaxation techniques and efficient financial management methods.

[1822] Specific example

[1823] 1. Specific Examples of User Registration and Authentication

[1824] The user accesses the new registration page from their communication device, enters the required information, and clicks the registration button.

[1825] When the terminal sends input information to the server, the server uses the communication provider's API to perform authentication, generate a user ID, and send it back.

[1826] The device displays the authentication result to the user, notifying them of a success message and user ID.

[1827] 2. Specific examples of receipt entry

[1828] Users take a picture of their supermarket shopping receipt with a communication device and upload it.

[1829] The terminal sends receipt images and input information to the server, which uses image recognition technology to extract and analyze the data and automatically classify the expenses.

[1830] The user reviews the analysis results and makes corrections as needed.

[1831] 3. Specific Examples of Financial Statement Preparation

[1832] The server aggregates monthly accounting data and automatically creates a trial balance.

[1833] The terminal displays the generated trial balance to the user and provides it in a downloadable format.

[1834] 4. Specific examples of consultations with an accountant

[1835] The user enters and submits their inquiry regarding year-end financial settlement procedures.

[1836] The server performs a basic analysis, and if it cannot provide an automated answer, it forwards the request to an accountant.

[1837] The accountant enters the appropriate answer, and the server notifies the user of that answer.

[1838] 5. Specific Examples of Emotional Engines

[1839] While the user is using the accounting system, the camera and microphone on the communication terminal capture the user's facial expressions and voice.

[1840] The device analyzes the captured data using an emotion engine to detect when the user is experiencing stress.

[1841] Based on the emotion data detected by the server, measures such as simplifying the system's UI will be taken.

[1842] The server will attempt to reduce user stress by providing accounting advice in a gentler tone.

[1843] Through the above processes, this system not only efficiently and accurately supports accounting operations but also enables flexible responses that take into account the user's feelings.

[1844] (Example 2)

[1845] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1846] Traditional accounting systems require users to manually input and aggregate data, which is time-consuming, labor-intensive, and prone to human error. Furthermore, few systems consider the user's emotional state, leading to stress accumulation due to prolonged work. Additionally, collaboration with experts is often difficult, making timely consultation challenging. There is a need for a flexible system that addresses these issues, supports efficient and accurate accounting operations, and is considerate of the user's emotional state.

[1847] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1848] In this invention, the server includes means for a user to input and transmit registration information using a communication terminal; means for the server to receive the registration information and perform authentication via a request from a communication provider; means for the server to generate a unique user ID based on the authentication result and send it back to the terminal; means for a user to upload transaction-related images using a communication terminal; means for the server to receive the images and convert them into text data using image recognition technology; means for the server to analyze and automatically classify the text data; means for the server to send the analysis results to the communication terminal for the user to confirm and correct; means for the server to automatically generate financial statements based on aggregated accounting data; means for the server to send the automatically generated financial statements to the communication terminal for the user to confirm; means for the server to analyze accounting data using an artificial intelligence algorithm and send the analysis results to the communication terminal; means for the communication terminal to collect facial and voice data; means for the server to analyze the user's emotional state and dynamically adjust the UI / UX; and means for the server to optimize advice and alert content based on emotional data. This will enable more efficient user data entry, improved data accuracy, and accounting operations that take emotional states into consideration.

[1849] A "user" is a person who uses the system to input data, register information, and perform sentiment analysis.

[1850] A "communication terminal" is a device operated by a user that can input, transmit, and receive data, and collect data using a camera or microphone.

[1851] A "server" is a central computer system that receives data transmitted from communication terminals and performs processing such as storage, analysis, authentication, classification, aggregation, and sentiment analysis.

[1852] "Registration information" refers to personal information such as name, phone number, and email address that users enter using their communication devices.

[1853] A "communication provider" is an external service provider that authenticates users based on registration information and communicates with the system via an API.

[1854] A "unique user ID" is an identifier generated by the server based on the user's authentication result, used to uniquely identify each user.

[1855] "Transaction-related images" refer to image data necessary for accounting processing, including receipts and invoices for purchased goods.

[1856] "Image recognition technology" refers to technologies used by servers to extract text data from images related to transactions, and includes technologies such as OCR (Optical Character Recognition).

[1857] "Text data" refers to character information extracted using image recognition technology, and includes data such as transaction date, item, and amount.

[1858] A "database" is a system for systematically storing and managing data acquired by a server.

[1859] "Automatic classification" is the process by which a server classifies text data it has acquired into specific categories, and this includes the classification of income and expenses.

[1860] "Financial statements" are documents that show financial information, such as trial balances and financial statements, generated based on accounting data compiled by a server.

[1861] An "artificial intelligence algorithm" is a computational method that uses technologies such as machine learning and deep learning, which a server uses to analyze accounting data.

[1862] "Facial and voice data" refers to data that shows the user's facial expressions and voice characteristics, collected using the camera and microphone of the communication device.

[1863] "Emotional state" refers to the user's current psychological state, obtained by analyzing collected facial and voice data, and can include states such as stress or joy.

[1864] "UI / UX" refers to the user interface and user experience, and is a set of design elements that optimize the interaction between a system and its user.

[1865] "Advice and alerts" refer to the advice and warnings that the server provides to the user based on sentiment data and financial data.

[1866] A "specialist" is someone who possesses the knowledge and experience to provide consultation services through the system, and this primarily refers to individuals such as accountants.

[1867] A "prompt" is an input sentence or instruction sent to a generative AI model to obtain a specific output.

[1868] This invention is a system that improves the efficiency and accuracy of accounting operations by having a server receive and process data entered by a user using a communication terminal. Furthermore, by incorporating an emotion engine, it provides a function that allows the system to respond dynamically based on the user's emotions. The following hardware and software are used to implement this invention.

[1869] Hardware and software configuration

[1870] Communication devices (smartphones, tablets, personal computers)

[1871] server

[1872] Camera and microphone (built into the communication terminal)

[1873] Database systems (e.g., MySQL, MongoDB)

[1874] API services (e.g., Twilio API, Google Cloud Vision API, Microsoft Azure Emotion API)

[1875] Programming languages ​​and frameworks (e.g., Node.js, Flask, Pandas, TensorFlow)

[1876] User registration and authentication

[1877] Users access the application using a communication device and enter registration information such as their name, phone number, and email address. The entered information is sent from the device to the server. The server authenticates the user via a request from a communication provider such as the Twilio API and generates a unique user ID based on the authentication result. The generated user ID is sent back to the device, and the user can use it to log in from the next time onward.

[1878] Data entry and management

[1879] The user uses a communication device to take a picture of the receipt for their purchased items and upload it to the system. They also enter information such as the transaction date and amount into the corresponding entry field. The device sends the uploaded image and entered information to the server. The server uses the Google Cloud Vision API to extract text data from the receipt image and saves it to a database. The server then analyzes the saved data and automatically categorizes it into income and expense categories. The analysis results are sent to the device, where the user can review the results and make corrections as needed.

[1880] Accounting and Analysis

[1881] The server aggregates daily and monthly accounting data and calculates total income and expenses. It also automatically generates financial statements such as trial balances and financial statements and stores them in a database. Furthermore, the server analyzes the accounting data using AI algorithms such as TensorFlow to identify unusual spending patterns and potential savings. The analysis results are sent to the terminal, and the user can view the results in a visually easy-to-understand format.

[1882] Collaboration with experts

[1883] When a user enters a specific inquiry and sends it to the system using a communication terminal, the server analyzes the input and generates an automatic response based on a generated AI model. If the server cannot resolve the issue, it escalates it to an expert. After receiving the response from the expert, the server sends the content to the terminal and notifies the user.

[1884] Emotional engine integration

[1885] While a user is using the accounting system, the camera and microphone on the communication terminal collect the user's facial expressions and voice data. The collected data is sent to a server, where the Microsoft Azure Emotion API analyzes the user's emotional state. Based on the emotional data, the server dynamically adjusts the UI / UX and optimizes accounting advice and alerts to reduce user stress. Furthermore, the server accumulates emotional data over the long term and provides special plans and advice tailored to the user's stress level.

[1886] Examples of specific cases and prompt statements

[1887] 1. Specific examples of user registration and authentication:

[1888] The user accesses the new registration page from their communication device, enters the required information, and clicks the registration button. The device then sends the information to the server, which uses the Twilio API to perform authentication, generates a user ID, and sends it back. The user receives a "Registration Successful" message and the user ID.

[1889] 2. Specific examples of receipt entry:

[1890] The user takes a picture of their supermarket shopping receipt and uploads it. The device sends the receipt image and input information to the server, which uses the Google Cloud Vision API to extract and analyze the data and automatically categorize expenses. The user reviews the analysis results and makes corrections as needed.

[1891] 3. Specific examples of financial statement preparation:

[1892] The server aggregates monthly accounting data and automatically generates a trial balance. The generated trial balance is displayed to the user via their terminal and provided in a downloadable format.

[1893] 4. Specific examples of consultations with experts:

[1894] The user enters and submits a question such as, "Why did this month's expenses exceed the budget?" The server generates a prompt and inputs it into a generating AI model (e.g., GPT-3) to automatically generate an answer. If it cannot resolve the issue, it forwards the question to an expert and notifies the user of the answer.

[1895] 5. Specific examples of the emotional engine:

[1896] While a user is using the accounting system, the camera and microphone on the communication terminal capture the user's facial expressions and voice. The server analyzes this data using the Microsoft Azure Emotion API to detect if the user is experiencing stress. As a result, the server simplifies the UI and provides accounting advice in a more friendly tone.

[1897] Through the processing steps described above, this system not only efficiently and accurately supports accounting operations but also enables flexible responses that take into account the user's emotional state.

[1898] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1899] Step 1:

[1900] The user accesses the application from their communication device and enters registration information such as their name, phone number, and email address. After entering the information, they click the registration button.

[1901] Input: Registration information such as name, phone number, and email address.

[1902] Output: Request sent to the server by the terminal

[1903] Specific operation: The user enters the required information into the input form and presses the "Register" button in the browser or application on their communication device.

[1904] Step 2:

[1905] The terminal temporarily stores the entered information and sends it to the server using the secure HTTP protocol.

[1906] Input: User-entered registration information

[1907] Output: HTTP POST request to the server

[1908] Specific operation: The terminal temporarily stores the input information in memory and then sends the data to the server via secure HTTPS communication.

[1909] Step 3:

[1910] The server verifies the received information and sends an authentication request to perform phone number authentication using the Twilio API.

[1911] Input: Registration information sent from the device

[1912] Output: Authentication request and authentication result to the Twilio API

[1913] Specific operation: A Node.js script is executed on the server side, formatting the received registration information and sending a request to the Twilio API. This performs phone number verification.

[1914] Step 4:

[1915] The server receives the authentication result, generates a unique user ID, saves it to the database, and simultaneously sends it back to the terminal.

[1916] Input: Authentication result from Twilio API

[1917] Output: Unique user ID, saved to database, and returned to terminal.

[1918] Specific operation: The server confirms successful authentication, generates a unique user ID and saves it to MongoDB, and then sends an authentication success response containing that user ID back to the terminal.

[1919] Step 5:

[1920] The device displays a "Registration successful" message and the generated user ID to the user.

[1921] Input: User ID and authentication result returned from the server.

[1922] Output: "Registration successful" message and user ID displayed.

[1923] Specific operation: The application on the terminal receives a response from the server and displays a "registration successful" message and user ID to the user in the form of a pop-up or notification.

[1924] Step 6:

[1925] The user uses a communication device to take a picture of the receipt for the purchased items and uploads it to the system. They also enter information such as the transaction date and amount into the corresponding entry field.

[1926] Input: Receipt image, transaction date, amount, and other input data.

[1927] Output: Request sent to the server by the terminal

[1928] Specific operation: The user takes a picture of the receipt using the camera on their communication device, enters the transaction date and amount into the application's form, and then clicks the "Upload" button.

[1929] Step 7:

[1930] The terminal temporarily stores the image file and input data, and then sends an HTTP POST request to the server.

[1931] Input: User-submitted receipt image and associated transaction information

[1932] Output: HTTP POST request to the server

[1933] Specific operation: The terminal temporarily stores the receipt image and input information in memory, and then sends the image and data to the server using secure HTTPS communication.

[1934] Step 8:

[1935] The server uses the Google Cloud Vision API to extract text data from the receipt image and saves it to a database.

[1936] Input: Receipt image and transaction information sent from the device.

[1937] Output: Extraction results of character data and saving to the database.

[1938] Specific operation: A Python script is executed on the server side, sending an image to the Google Cloud Vision API, converting the extracted text data into a Pandas DataFrame, and saving it to a MySQL database.

[1939] Step 9:

[1940] The server analyzes the stored data and automatically categorizes it into income and expense categories.

[1941] Input: Text data extracted from receipts

[1942] Output: Accounting data categorized by type

[1943] Specific operation: An analysis script is executed on the server side, automatically classifying the data based on income and expense categories, and updating the appropriate fields in the database with the results.

[1944] Step 10:

[1945] The server returns the analysis results to the terminal in JSON format.

[1946] Input: Accounting data categorized by type

[1947] Output: JSON response of the analysis results

[1948] Specific operation: The server formats the analysis results in JSON format and sends the response to the terminal using secure HTTPS communication.

[1949] Step 11:

[1950] The terminal displays the analysis results it has received and provides an interface for the user to review and correct them.

[1951] Input: Analysis results sent from the server

[1952] Output: Provides an interface for displaying and modifying analysis results.

[1953] Specific operation: The application on the terminal side parses the JSON response, displays the parsing results to the user, and provides an interface for correcting the data as needed.

[1954] Step 12:

[1955] The server aggregates daily and monthly accounting data and calculates total income and expenses.

[1956] Input: Accounting data in the database

[1957] Output: Total income and expenses

[1958] Specific operation: A scheduled script is executed on the server side, using Pandas to aggregate accounting data in the database and calculate daily and monthly totals of income and expenses.

[1959] Step 13:

[1960] The server automatically generates financial statements such as trial balances and financial statements based on aggregated data and saves them in a database.

[1961] Input: Daily and monthly aggregated data

[1962] Output: Auto-generated financial statements

[1963] Specific operation: The generation script is executed on the server side, generating financial statements using Pandas and Matplotlib, and saving them in PDF format.

[1964] Step 14:

[1965] The server uses an AI algorithm based on TensorFlow to analyze accounting data and identify unusual spending patterns and potential savings.

[1966] Input: Automatically generated financial statements and accounting data

[1967] Output: Analysis results

[1968] Specific operation: The server runs an AI model, uses accounting data as input to detect anomalies and suggest cost-saving measures, analyzes the results, and saves them to a database.

[1969] Step 15:

[1970] The server returns the analysis results to the terminal in JSON format.

[1971] Input: Analysis results by AI algorithm

[1972] Output: JSON response

[1973] Specific operation: The server formats the analysis results in JSON format and sends the response to the terminal using secure HTTPS communication.

[1974] Step 16:

[1975] The device displays the analysis results to the user in graph or table format.

[1976] Input: Analysis results sent from the server

[1977] Output: Display in graph or tabular format

[1978] Specific operation: The application on the terminal receives the analysis results and displays them visually using a graph library.

[1979] Step 17:

[1980] The user inputs and sends the details of what they want to discuss with the accountant via a communication terminal.

[1981] Input: Consultation details

[1982] Output: Request sent to the server by the terminal

[1983] Specific action: The user enters their inquiry into the consultation form on their communication device and presses the "Send" button.

[1984] Step 18:

[1985] The terminal temporarily saves the entered consultation details and sends them to the server.

[1986] Input: User-entered consultation details

[1987] Output: HTTP POST request to the server

[1988] Specific operation: The terminal temporarily stores the consultation details in memory and then sends the data to the server via secure HTTPS communication.

[1989] Step 19:

[1990] The server analyzes the received inquiry and generates an automated response based on an AI model. If it cannot resolve the issue, it escalates it to an expert.

[1991] Input: Consultation content sent from the device

[1992] Output: Automated response and escalation results

[1993] Specific operation: The server uses NLP technology to analyze the consultation content and inputs it as a prompt into a generating AI model (e.g., GPT-3) to automatically generate an answer. If a solution cannot be found, the issue is escalated to an expert.

[1994] Step 20:

[1995] The server receives the answer from the expert and sends it to the communication terminal, notifying the user.

[1996] Input: Answer from an expert

[1997] Output: Response to terminal

[1998] Specific operation: After receiving a response from an expert, the server formats the content in JSON format and sends the response to the terminal using secure HTTPS communication.

[1999] Step 21:

[2000] The device presents the user with detailed answers and provides information to help them decide on their next action.

[2001] Input: Response received from the server

[2002] Output: Display to the user

[2003] Specific operation: The application on the device receives the response and displays it to the user in the form of a pop-up or notification, providing information for the next action.

[2004] Step 22:

[2005] While the user is using the accounting system, the camera and microphone on the communication terminal collect facial expressions and voice data.

[2006] Input: User's facial expressions and voice data

[2007] Output: Request sent to the server by the terminal

[2008] Specific operation: The camera and microphone built into the communication terminal capture the user's facial expressions and voice in real time, and temporarily store this data.

[2009] Step 23:

[2010] The device sends the collected data to the emotion engine for analysis.

[2011] Input: Facial expressions and voice data

[2012] Output: Emotion analysis results

[2013] Specific operation: The device sends the collected data to the Microsoft Azure Emotion API to analyze the user's emotional state in real time.

[2014] Step 24:

[2015] The server analyzes the user's emotional state and dynamically adjusts the UI / UX.

[2016] Input: Analysis results from the emotion engine

[2017] Output: UI / UX adjustment results

[2018] Specific operation: The server dynamically adjusts UI elements and tone based on emotional data to optimize the user experience.

[2019] Step 25:

[2020] The server optimizes accounting advice and alerts based on sentiment data.

[2021] Input: Sentimental data and financial data

[2022] Output: Optimized accounting advice and alerts

[2023] Specific operation: The server analyzes emotional data and generates specific accounting advice and alerts tailored to the user's emotional state.

[2024] Step 26:

[2025] The server accumulates emotional data, analyzes long-term emotional patterns, and provides personalized plans and advice.

[2026] Input: Accumulated emotional data

[2027] Output: Special plans or advice

[2028] Specific operation: The server analyzes emotional data accumulated over a long period and provides special countermeasures and advice tailored to the user's stress level.

[2029] (Application Example 2)

[2030] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[2031] Currently, many accounting systems do not take into account the emotional state of the user, leading to high levels of user stress. Furthermore, users often encounter complex operations and take considerable time to understand when performing accounting tasks. Therefore, cumbersome user interfaces contribute to increased stress. This situation highlights the need for efficient and stress-free accounting operations.

[2032] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[2033] In this invention, the server includes means for the user to input and transmit registration information using a communication terminal; means for the server to receive the registration information and perform authentication via a request from a communication provider; means for the server to generate a unique user ID based on the authentication result and send it back to the terminal; means for collecting the user's facial and voice data from the communication terminal and identifying emotions; means for dynamically adjusting the user interface based on the identified emotions; and means for analyzing the identified emotion data and evaluating the user's stress level. As a result, the user can perform accounting tasks through a simplified user interface, improving work efficiency while reducing stress.

[2034] A "communication terminal" is an electronic device used for inputting, transmitting, and receiving information, and includes smartphones and head-mounted displays.

[2035] A "server" is a computing system used for storing, processing, and managing data, and it performs user registration information and sentiment data analysis.

[2036] "Registration information" refers to information necessary to identify a user, such as their name, phone number, and email address, which they enter using a communication device.

[2037] "Authentication" is the process by which a server verifies the authenticity of a user's registration information through a request from a communication provider.

[2038] A "User ID" is a unique identifier generated by the server based on the authentication result and is used to identify the user.

[2039] "Facial expression data" refers to data obtained by capturing a user's facial expressions with the camera on a communication device and analyzing it to determine the user's emotions.

[2040] "Voice data" refers to data obtained by capturing the user's voice with the microphone of a communication terminal and analyzing it to determine the user's emotions.

[2041] "Emotion recognition" is the process of analyzing a user's facial expression and voice data to identify their current emotional state.

[2042] "Dynamic adjustment" is a feature that changes elements of the user interface in real time based on the user's identified emotions.

[2043] A "user interface" is a means of display and operation for a user to interact with a system, and includes screen design and operation buttons.

[2044] "Emotional data" refers to data that indicates the user's emotional state, obtained through emotion recognition.

[2045] "Stress level" refers to the degree of stress a user is experiencing, evaluated based on emotional data.

[2046] "Image recognition technology" is a technology that analyzes information such as characters from uploaded images and converts it into text data.

[2047] This invention is a system that streamlines and improves the accuracy of accounting operations by allowing users to input and transmit registration information using a communication terminal, which is then received by a server. Furthermore, this system provides a function to analyze the user's emotional state and dynamically adjust the user interface based on that analysis.

[2048] Hardware and software to be used

[2049] Hardware: Smartphones, head-mounted displays (HMDs)

[2050] Software: Flask (Python-based web framework), SQLite (database system), external authentication, image recognition, and sentiment analysis service APIs.

[2051] Program Processing Overview

[2052] User-performed processes

[2053] Users first access the system using a communication terminal and enter registration information such as their name, phone number, and email address. This information is sent to the server, where authentication is performed via the communication provider. If authentication is successful, the server generates a unique user ID and sends it back to the terminal. From then on, users can log in to the system using this user ID.

[2054] Uploading and managing receipts

[2055] Users take a picture of their purchase receipt with a communication device and upload it to the system. The server receives this receipt image and uses image recognition technology to convert it into text data (e.g., transaction date, item, amount). This data is stored in a database and automatically categorized. The analysis results are displayed on the device, allowing the user to review and modify them.

[2056] Aggregation and analysis of accounting data

[2057] The server aggregates daily and monthly accounting data and calculates total income and expenses. It automatically generates financial documents such as trial balances and financial statements and stores them in a database. Furthermore, the server uses AI algorithms to analyze the accounting data, identifying unusual spending patterns and potential savings. These analysis results are transmitted to the communication terminal and displayed visually to the user.

[2058] Emotional engine integration

[2059] When a user uses the camera and microphone built into their communication device to collect facial and voice data, it is sent to a server. The server analyzes the user's emotions using an emotion analysis algorithm and dynamically adjusts the user interface based on the results. For example, if the user is feeling stressed, the UI is simplified and accounting advice is provided in a gentler tone.

[2060] Examples of usage and prompt statements

[2061] User registration usage example

[2062] The user accesses the new registration page from their communication device, enters the required information, and clicks the registration button. The server performs authentication, generates a user ID, and sends it back to the device.

[2063] Prompt example: "A new user is attempting to register with the name 'User A', phone number '090-1234-5678', and email address 'user@example.com'."

[2064] Example of using receipt entry

[2065] Users take photos of their supermarket shopping receipts and upload them. The server uses image recognition technology to extract and analyze the data, automatically classifying expenses.

[2066] Example prompt: "The user has taken a picture of a receipt and uploaded it. Extract the date, item, and amount from the image."

[2067] Examples of using an emotion engine

[2068] While a user is using the accounting system, the camera and microphone on the communication terminal capture the user's facial expressions and voice. The server analyzes this data to detect stress and simplifies the UI to make it easier to use.

[2069] Example prompt: "Analyze data from the user's camera and microphone to assess their stress level. If stress is high, simplify the user interface."

[2070] Through the processing steps described above, this system not only efficiently and accurately supports accounting operations but also enables flexible responses that take into account the user's emotional state.

[2071] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[2072] Step 1:

[2073] The user accesses the new registration page from their communication device, enters registration information such as their name, phone number, and email address, and submits it. This entered data is temporarily stored by the device and then sent to the server.

[2074] Input: Name, phone number, email address

[2075] Output: Registration information sent to the server

[2076] Step 2:

[2077] The server authenticates the received registration information via the communication provider's API. It then retrieves the authentication result from the communication provider and generates a unique user ID based on it.

[2078] Input: Registration information (name, phone number, email address)

[2079] Data processing: Authentication using the communication provider's API

[2080] Output: Authentication result, unique user ID

[2081] Step 3:

[2082] The server sends a generated user ID back to the terminal, and the terminal displays the authentication result and user ID to the user. The user uses this user ID for future logins.

[2083] Input: Unique User ID

[2084] Output: Authentication result and user ID displayed on the terminal

[2085] Step 4:

[2086] The user uses a communication device to take a picture of the receipt related to the transaction, enters relevant information (transaction date, amount, etc.) into the corresponding entry field, and uploads it to the system.

[2087] Input: Receipt image, transaction date, amount

[2088] Output: Receipt image and related information sent to the server

[2089] Step 5:

[2090] The server converts the received receipt image into text data using image recognition technology. The converted data is then stored in a database.

[2091] Input: Receipt image

[2092] Data processing: Text conversion using image recognition technology

[2093] Output: Converted text data

[2094] Step 6:

[2095] The server analyzes the stored data and automatically categorizes it into income and expense categories. The analysis results are sent to the terminal, where the user can review and modify them.

[2096] Input: Text data

[2097] Data processing: Automated classification algorithms

[2098] Output: Automated classification of analysis results

[2099] Step 7:

[2100] The user collects facial and audio data using the camera and microphone built into their communication terminal. This data is then transmitted from the terminal to a server.

[2101] Input: Facial expression data, audio data

[2102] Output: Facial expression and audio data sent to the server

[2103] Step 8:

[2104] The server uses an emotion analysis algorithm to analyze the user's emotional state and dynamically adjusts the user interface based on the analysis results.

[2105] Input: Facial expression data, audio data

[2106] Data processing: Sentiment analysis algorithm

[2107] Output: Adjusted user interface

[2108] Step 9:

[2109] The server accumulates long-term emotional data and evaluates user stress levels. Based on the evaluation, it generates data to provide special plans and advice.

[2110] Input: Analyzed sentiment data

[2111] Data processing: Stress level evaluation algorithm

[2112] Output: Special plans and advice provided to the user.

[2113] Through these processing steps, this system not only efficiently and accurately supports accounting operations but also enables flexible responses that take into account the user's emotional state.

[2114] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[2115] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include those described above. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions shown by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2116] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[2117] [Fourth Embodiment]

[2118] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[2119] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[2120] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[2121] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[2122] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[2123] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[2124] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[2125] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[2126] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[2127] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[2128] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[2129] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[2130] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[2131] This invention is a system that improves the efficiency and accuracy of accounting operations by having a server receive and process data entered by a user using a communication terminal. The details of this system program and specific examples of its processing are shown below.

[2132] Program processing and its explanation

[2133] User registration and authentication

[2134] 1. The user accesses the application from their communication device and enters registration information such as their name, phone number, and email address. After entering the information, they click the registration button.

[2135] 2. The terminal temporarily stores the entered information and sends it to the server.

[2136] 3. The server verifies the received registration information via the communication provider's API and performs authentication.

[2137] 4. The server generates a unique user ID based on the authentication result and sends it back to the terminal.

[2138] 5. The terminal displays the authentication result and the generated user ID to the user and provides login information for future logins.

[2139] Data entry and management

[2140] 1. The user uses a communication device to take a picture of the receipt for the purchased items and uploads it to the system. They also enter relevant information (transaction date, amount, etc.) into the corresponding entry field.

[2141] 2. The device sends the uploaded image and input information to the server.

[2142] 3. The server uses image recognition technology to extract text data (such as transaction date, item, and amount) from the receipt image and saves it to the database.

[2143] 4. The server analyzes the stored data and automatically categorizes it into income and expense categories.

[2144] 5. The terminal displays the analysis results to the user and provides an interface for the user to review and correct them.

[2145] Accounting and Analysis

[2146] 1. The server aggregates daily and monthly accounting data and calculates total income and expenses.

[2147] 2. The server automatically generates financial statements such as trial balances and financial statements and saves them in the database.

[2148] 3. The server uses AI algorithms to analyze accounting data and identify unusual spending patterns and potential savings.

[2149] 4. The device displays the analysis results to the user in a visually easy-to-understand format (graphs and tables).

[2150] Collaboration with accountants

[2151] 1. The user enters and sends the details of what they want to discuss with the accountant via a communication terminal.

[2152] 2. The terminal sends the entered consultation details to the server.

[2153] 3. The server analyzes the received inquiry and generates an automated response using AI. If the issue cannot be resolved, it escalates to an accountant.

[2154] 4. The server sends the response received from the accountant to the communication terminal and notifies the user.

[2155] 5. The device presents the user with detailed answers and provides information to help the user decide on their next action.

[2156] Specific example

[2157] 1. Specific Examples of User Registration and Authentication

[2158] The user accesses the new registration page from their communication device, enters the required information, and clicks the registration button.

[2159] When the terminal sends input information to the server, the server uses the communication provider's API to perform authentication, generate a user ID, and send it back.

[2160] The device displays the authentication result to the user, notifying them of a success message and user ID.

[2161] 2. Specific examples of receipt entry

[2162] Users take a picture of their supermarket shopping receipt with a communication device and upload it.

[2163] The terminal sends the receipt image and input information to the server, which uses image recognition technology to extract and analyze the data and automatically classify the expenses.

[2164] The user reviews the analysis results and makes corrections as needed.

[2165] 3. Specific Examples of Financial Statement Preparation

[2166] The server aggregates monthly accounting data and automatically creates a trial balance.

[2167] The terminal displays the generated trial balance to the user and provides it in a downloadable format.

[2168] 4. Specific examples of conversations with accountants

[2169] The user enters and submits their inquiry regarding year-end financial settlement procedures.

[2170] The server performs a basic analysis, and if it cannot provide an automated answer, it forwards the request to an accountant.

[2171] The accountant enters the appropriate answer, and the server notifies the user of that answer.

[2172] As described above, the present invention provides a system that offers total support for accounting operations and significantly reduces the burden on users. This system enables users to perform accounting operations more efficiently and accurately.

[2173] The following describes the processing flow.

[2174] User registrat...

Claims

1. A means by which the user enters and transmits registration information using a communication terminal, A means by which the server receives the registration information and performs authentication via a request from the communication provider, A system that includes a means for the server to generate a unique user ID based on the authentication result and send it back to the terminal.

2. A means for users to upload images related to transactions using a communication terminal, A server receives the image and converts it into text data using image recognition technology, The server provides means for analyzing and automatically classifying the aforementioned text data, The system according to claim 1, which includes means for the server to transmit analysis results to a communication terminal and for the user to confirm and correct them.

3. A means for the server to automatically generate financial statements based on the aggregated accounting data, A means by which the server automatically generates financial statements and sends them to a communication terminal for the user to review, The system according to claim 1, comprising means for a server to analyze accounting data using an AI algorithm and transmit the analysis results to a communication terminal.

4. A means by which a user sends the details of their consultation to an accountant using a communication terminal, A server receives the aforementioned consultation content, performs a basic analysis, and generates an automated response. If the server cannot automatically provide a response, the issue will be escalated to an accountant, and a means of receiving the registered response from the accountant will be provided. The system according to claim 1, which includes means by which a server transmits a response from an accountant to a communication terminal and a user confirms the content of the response.

5. A means by which a communication terminal temporarily stores the user's personal information and accounting data and transmits it to a server when necessary, A means for the server to store the data it receives in a database and perform the necessary processing, The system according to claim 1, comprising means for displaying data generated by the server on a communication terminal in a visually understandable format.

Images