System

Abstract

A system is provided.SOLUTION: A system including means for analyzing a bill using an artificial intelligence model and automatically transcribing transaction data extracted from the bill to a final tax return material, means for converting voice data into text and generating minutes and a summary of a meeting, means for inputting a travel schedule and accommodation conditions and proposing and arranging optimal transportation means and accommodation facilities, and means for analyzing sales data and cost data and making a strategic improvement proposal.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Modern freelancers are required to efficiently handle numerous miscellaneous tasks so they can focus on the core of their business. However, a wide range of tasks, such as managing invoices, preparing tax return documents, taking meeting minutes, arranging transportation and accommodation, and managing sales and cost data, consume time and effort. These miscellaneous tasks are a factor in reducing the productivity and efficiency of freelancers. Therefore, there is a strong demand for a method to efficiently handle these numerous tasks so that freelancers can focus on the core of their business. [Means for solving the problem]

[0005] The present invention solves the above problems by providing a system that combines an artificial intelligence model and optical character recognition technology. Specifically, it provides a means for analyzing invoices and automatically transcribing transaction data into tax return documents. It also includes a means for converting voice data into text and generating meeting minutes and summaries. It also provides a means for proposing and arranging optimal transportation and accommodations based on travel itineraries and accommodation requirements entered by the user. Additionally, the system includes a means for analyzing sales and cost data and making strategic improvement proposals. This enables efficient business support, allowing freelancers to focus on their business.

[0006] An "artificial intelligence model" is a collection of computer programs that automatically analyze data and make decisions, and includes algorithms for efficiently performing specific tasks.

[0007] An "invoice" is a written or electronic document that describes the details of a transaction fee claim, and includes information such as the transaction date, the business partner, the transaction content, and the amount.

[0008] "Tax return materials" refers to paper or electronic files containing income, deductions, and other information required for submission to tax authorities, specifically reporting an individual's or corporation's annual income.

[0009] Optical character recognition (OCR) is a technology that recognizes characters in an image as electronic data and extracts them as text information. It is used to read handwritten or printed characters.

[0010] "Audio data" refers to digital data including recordings of audio such as conferences and telephone calls, and refers to information stored as acoustic signals.

[0011] "Convert to text" refers to the process of extracting character information from audio data or image data and converting it into a string of characters, using voice recognition technology or OCR technology.

[0012] "Minutes" are documents that record the progress and statements made at a meeting in written form, and are used for decision-making at meetings and to confirm important matters.

[0013] A "summary" is a concise summary of the contents of a meeting or report, conveying the main points and conclusions in a short form.

[0014] "Transportation" refers to the means or method of transportation used to travel to a destination, including airplanes, trains, automobiles, etc.

[0015] "Accommodation" refers to a place provided for people to stay for a certain period of time, and includes hotels, inns, guest houses, etc.

[0016] "Sales data" refers to data that indicates information on revenues earned from the sale of products or the provision of services over a certain period of time.

[0017] "Cost data" refers to data that indicates information on expenses and expenditures related to business operations, product production, and service provision.

[0018] A "strategic proposal" is a presentation of specific action plans and improvement measures aimed at improving business efficiency and profitability. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5]FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0020] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0021] First, the terms used in the following description will be explained.

[0022] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0023] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0024] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0025] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0026] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0027] [First embodiment]

[0028] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0029] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0030] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0031] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0032] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0033] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0034] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0035] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0036] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0037] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0038] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0039] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0040] The system of the present invention provides a means for freelancers to efficiently handle the many miscellaneous tasks they must perform on a daily basis, thereby helping them to focus on their core business. Below, we will explain each function of the system of the present invention and its specific embodiment.

[0041] Support for automatic creation of tax return documents

[0042] 1. Upload your invoice:

[0043] The user prepares a scanner or PDF file of the invoice on the terminal.

[0044] The user uploads the invoice file from the terminal to the server.

[0045] 2. Invoice analysis:

[0046] The server receives the uploaded file and uses optical character recognition (OCR) technology to scan the invoice and extract the text information.

[0047] The server automatically determines necessary data such as transaction date, amount, and trading partner from the extracted information.

[0048] 3. Data transcription and tax return generation:

[0049] The server stores the identified data in an internal Excel template.

[0050] The server arranges the data according to the tax return format and automatically fills in the necessary sections.

[0051] The server generates the completed Excel file and tax return, which the user can download from their device or the server can email to the user.

[0052] Examples:

[0053] When a user uploads a PDF of an invoice to their device, the server uses OCR technology to extract the necessary information from the PDF (e.g., amount 100,000 yen, transaction date October 3, 2023, business partner X), automatically reflects this information in an Excel spreadsheet and tax return, and provides it to the user.

[0054] Creating meeting minutes and summaries

[0055] 1. Uploading audio data:

[0056] A user saves an audio file of an online conference on a terminal and uploads the audio file from the terminal to a server.

[0057] 2. Audio data conversion and transcription:

[0058] The server converts the received audio file into text using a speech recognition engine.

[0059] Based on the converted text, minutes and summaries are automatically generated using a generative artificial intelligence model.

[0060] 3. Providing minutes and summaries:

[0061] The server stores the completed minutes and summary as files, which the user can download from their terminal or send to them by email.

[0062] Examples:

[0063] When a user uploads an audio recording file of an online meeting to their terminal, the server converts the audio into text, generates minutes and a summary summarizing the contents of the meeting, and provides them to the user.

[0064] Transportation and hotel arrangements

[0065] 1. Enter your travel dates and desired conditions:

[0066] The user inputs the business trip schedule, transportation means, and accommodation requirements into the terminal, and transmits the information from the terminal to the server.

[0067] 2. Travel and accommodation search and suggestions:

[0068] Based on the received information, the server performs an online search, lists the best transportation options and accommodations, and makes suggestions to the user.

[0069] 3. Proposal and Arrangement:

[0070] The server makes reservations for transportation and accommodation based on the options selected by the user on the terminal.

[0071] The server will send a confirmation email to the user confirming the reservation.

[0072] Examples:

[0073] When a user inputs the dates of a business trip and specifies the desired means of transportation and accommodation, the server proposes the optimal flight and hotel plan, makes a reservation for the plan selected by the user, and sends a confirmation email.

[0074] Sales and cost information management and strategy proposals

[0075] 1. Upload sales and cost data:

[0076] The user prepares data on sales and costs on the terminal and uploads it to the server from the terminal.

[0077] 2. Data analysis and strategy proposal:

[0078] The server analyzes the received data and extracts sales and cost trends.

[0079] The server uses a generative artificial intelligence model to automatically generate data-based strategic proposals and improvement measures.

[0080] 3. Proposal report provided:

[0081] The server compiles the generated proposals in the form of a report, which the user can download from their terminal or the server can send to the user by email.

[0082] Examples:

[0083] When a user uploads an Excel sheet of recent sales and costs to their device, the server analyzes the data and generates a proposal for the user, such as "reducing advertising costs by 20% will increase profits by 15%."

[0084] As described above, the system of the present invention efficiently handles and supports a wide range of miscellaneous tasks, allowing freelancers to focus their time and efforts on their core business.

[0085] The processing flow will be explained below.

[0086] Support for automatic creation of tax return documents

[0087] Step 1:

[0088] The user prepares a PDF file of the invoice on the terminal.

[0089] Step 2:

[0090] The user uploads the invoice file from the terminal to the server.

[0091] Step 3:

[0092] The server receives the uploaded invoice file.

[0093] Step 4:

[0094] The server uses optical character recognition (OCR) technology to extract text information from the invoice.

[0095] Step 5:

[0096] The server automatically determines necessary data such as transaction date, amount, and trading partner from the extracted information.

[0097] Step 6:

[0098] The server stores the identified data in an internal Excel template.

[0099] Step 7:

[0100] The server arranges the data according to the tax return format and automatically fills in the necessary sections.

[0101] Step 8:

[0102] The server generates the completed Excel file and tax return.

[0103] Step 9:

[0104] The user downloads the generated file from the device, or the server sends it to the user by email.

[0105] Creating meeting minutes and summaries

[0106] Step 1:

[0107] The user saves the audio file of the online conference on the device.

[0108] Step 2:

[0109] The user uploads an audio file from the device to the server.

[0110] Step 3:

[0111] The server converts the received audio file into text using a speech recognition engine.

[0112] Step 4:

[0113] The server inputs the converted text into a generative AI model.

[0114] Step 5:

[0115] The server uses generative AI models to create minutes and summaries.

[0116] Step 6:

[0117] The server saves the created minutes and summaries as files.

[0118] Step 7:

[0119] The user downloads the minutes and summary from their terminal, or the server sends them to the user by email.

[0120] Transportation and hotel arrangements

[0121] Step 1:

[0122] The user inputs the business trip schedule, transportation method, and accommodation requirements into the terminal.

[0123] Step 2:

[0124] The user sends input information from the terminal to the server.

[0125] Step 3:

[0126] The server performs an online search based on the received information.

[0127] Step 4:

[0128] The server will list the best transportation and accommodation options.

[0129] Step 5:

[0130] The server presents the listed candidates to the user.

[0131] Step 6:

[0132] The user selects from the candidates via the terminal and sends it to the server.

[0133] Step 7:

[0134] The server makes transportation and accommodation reservations based on the user's selections.

[0135] Step 8:

[0136] The server will send a confirmation email to the user confirming the reservation.

[0137] Sales and cost information management and strategy proposals

[0138] Step 1:

[0139] The user provides data on sales and costs on the terminal.

[0140] Step 2:

[0141] The user uploads data from the device to the server.

[0142] Step 3:

[0143] The server receives the uploaded data.

[0144] Step 4:

[0145] The server analyzes sales and cost data to extract trends and patterns.

[0146] Step 5:

[0147] The server uses a generative AI model based on the extracted data to automatically generate strategic proposals and improvement measures.

[0148] Step 6:

[0149] The server compiles the generated proposals in the form of a report.

[0150] Step 7:

[0151] The user downloads the proposal report from the terminal, or the server sends it to the user by email.

[0152] Example 1

[0153] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0154] Freelancers and sole proprietors are often overwhelmed with miscellaneous tasks, making it difficult to devote sufficient time to their core work. In particular, they face challenges in efficiently preparing tax return documents, taking meeting minutes, arranging business trips, and managing sales and cost data. There is a need for an environment that efficiently handles these miscellaneous tasks and allows freelancers to concentrate on their core work.

[0155] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0156] In this invention, the server includes means for analyzing invoices using an AI model and automatically transcribing transaction data extracted from the invoices into tax return documents, means for converting voice data into text and generating meeting minutes and summaries, means for inputting travel itineraries and accommodation requirements and proposing and arranging optimal transportation and accommodations, means for analyzing sales data and cost data and making strategic improvement proposals, means for providing an interface for users to upload files from their terminals, storage means for temporarily storing and analyzing files received by the server, and means for automatically generating proposal reports using a generative AI model. This allows freelancers to efficiently handle necessary miscellaneous tasks while focusing on their core business.

[0157] An "artificial intelligence model" is a collection of machine learning algorithms used to analyze data and perform a specific task.

[0158] An "invoice" is a document issued to bill for goods or services.

[0159] "Transaction data" refers to data that includes information related to a transaction, such as the transaction date, amount, and counterparty.

[0160] "Tax return documents" are documents to be submitted to tax authorities and contain information such as income and expenses.

[0161] "Audio data" means a recording of audio or an audio signal stored in digital form.

[0162] A "minutes" is a document that records the contents of a meeting, including the participants, what was said, and the main points of the meeting.

[0163] A "summary" is a summary that briefly summarizes the detailed content.

[0164] A "travel itinerary" is a schedule for business trips or travel.

[0165] "Accommodation conditions" refers to desired conditions and requirements regarding accommodation facilities.

[0166] "Transportation" refers to the means used for transportation, including cars, trains, airplanes, etc.

[0167] "Accommodation facilities" are facilities for staying overnight, including hotels, inns, and guesthouses.

[0168] "Sales data" refers to data that records revenues earned through the purchase and sale of goods and the provision of services.

[0169] "Cost data" is data that records the costs involved in manufacturing a product or providing a service.

[0170] "Improvement proposals" are specific suggestions or advice to solve current problems.

[0171] "File upload" refers to a user sending a file from their own terminal to a server.

[0172] "Storage" refers to a storage device or mechanism for saving data.

[0173] An "interface" refers to the means or screen that a user uses to operate a system.

[0174] A "prompt" refers to an instruction or question that is input into a generative AI model.

[0175] The system of the present invention provides a means for freelancers to efficiently handle the many miscellaneous tasks they must perform on a daily basis, thereby helping them to focus on their core business. Below, we will explain each function of the system of the present invention and its specific embodiment.

[0176] Support for automatic creation of tax return documents

[0177] The user prepares a scanner or PDF file of the invoice on their device and uploads the invoice file from their device to the server. The server receives the uploaded file and scans it using optical character recognition (OCR) technology such as Google Cloud Vision API to extract text information. The server determines necessary data from the extracted text information, such as the transaction date, amount, and client, and saves it in an internal Excel template using a library such as openpyxl. The server then arranges the data according to the tax return format and automatically fills it in. The completed tax return is provided via an HTTP response so that the user can download it from their device, or it is sent by email.

[0178] Examples:

[0179] When a user uploads a PDF of an invoice to their device, the server uses OCR technology to extract the necessary information from the PDF (e.g., 100,000 yen, October 3, 2023, business partner X), automatically reflects this information in an Excel spreadsheet and tax return, and provides it to the user.

[0180] Example prompt sentence:

[0181] "How can I extract transaction dates, amounts, and business partners from uploaded PDF invoices and reflect them in Excel templates and tax returns?"

[0182] Creating meeting minutes and summaries

[0183] Users save audio files of online meetings on their devices and then upload them from their devices to the server. The server converts the received audio files into text using the Amazon Transcribe API. Based on the converted text data, minutes and summaries are automatically generated using OpenAI's generative AI model. The completed minutes and summaries are saved in file format for users to download, provided as an HTTP response, or sent via email.

[0184] Examples:

[0185] When a user uploads an audio file of an online conference to their terminal, the server converts the audio into text, generates minutes and a summary summarizing the contents of the conference, and provides them to the user.

[0186] Example prompt sentence:

[0187] "How can I automatically generate minutes and summaries from audio files of online meetings?"

[0188] Transportation and hotel arrangements

[0189] The user enters their business trip dates, transportation method, and accommodation requirements into their device and sends them from the device to the server. Based on the received information, the server uses the Google Maps API or Expedia API to search for the optimal transportation method and accommodation. The server then lists the optimal plans and presents them to the user. The user selects the desired plan from their device and completes the reservation procedure. The server then reserves transportation and accommodation based on the selected plan and sends a confirmation email to the user.

[0190] Examples:

[0191] When a user inputs the dates of a business trip and specifies the desired means of transportation and accommodation conditions, the server proposes the optimal flight and hotel plan, makes a reservation for the plan selected by the user, and sends a confirmation email.

[0192] Example prompt sentence:

[0193] "How can I suggest and book the best travel and accommodation options based on the travel dates and mode of travel entered by the user?"

[0194] Sales and cost information management and strategy proposals

[0195] The user prepares sales and cost data as an Excel file and uploads it from their device to the server. The server reads the received Excel file using the Pandas library and analyzes the data. The server uses Matplotlib and Seaborn to visualize sales and cost trends, and uses a generative AI model (e.g., OpenAI's GPT-4) to generate strategic proposals and improvement measures based on the data. The proposal report is saved in PDF format and made available for the user to download from their device or sent via email.

[0196] Examples:

[0197] When a user uploads an Excel sheet containing recent sales and cost data to their device, the server analyzes the data, creates graphs that visualize sales and cost trends, and generates strategic suggestions based on the data (e.g., "reducing advertising costs by 20% will increase profits by 15%), which are then provided to the user.

[0198] Example prompt sentence:

[0199] "How can I analyze sales and cost data and automatically generate strategic proposals?"

[0200] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0201] Support for automatic creation of tax return documents

[0202] Step 1:

[0203] The user prepares a scanner or PDF file of the invoice on the device.

[0204] Specifically, users scan a paper invoice or save an existing digital invoice to their PC or smartphone.

[0205] Step 2:

[0206] The user uploads the invoice file from the terminal to the server.

[0207] Input: Invoice PDF or scanned file

[0208] Specifically, the user opens a file selection dialog of the browser, selects a file, and clicks the upload button.

[0209] Output: The invoice file is sent to the server.

[0210] Step 3:

[0211] The server receives the uploaded file.

[0212] Input: Uploaded invoice file

[0213] Specifically, the server receives the HTTP POST request and saves the file in temporary storage.

[0214] Output: Invoice file saved in storage

[0215] Step 4:

[0216] The server calls the Google Cloud Vision API or similar, scans the file using OCR technology, and extracts text information.

[0217] Input: Saved invoice file

[0218] Specifically, the server uses an OCR engine to extract text and obtain character information in JSON format.

[0219] Output: Extracted text information

[0220] Step 5:

[0221] The server automatically determines necessary data such as transaction date, amount, and trading partner from the extracted text information.

[0222] Input: Extracted text information

[0223] Specifically, the server uses a natural language processing library (e.g., spaCy) to analyze the required data and extract specific fields.

[0224] Output: Transaction date, amount, counterparty, etc.

[0225] Step 6:

[0226] The server saves the identified data in an internal Excel template.

[0227] Input: Transaction date, amount, counterparty, etc.

[0228] Specifically, the server uses a library such as openpyxl to embed data into the relevant cells of the Excel template.

[0229] Output: Excel file with completed tax return

[0230] Step 7:

[0231] The server provides the completed Excel file for the user to download from their device or sends it by email.

[0232] Input: Completed Excel file

[0233] Specifically, the server returns the file as an HTTP response, or sends the file to the user using an email sending API.

[0234] Output: Tax return provided to user

[0235] Creating meeting minutes and summaries

[0236] Step 1:

[0237] The user saves the audio file of the online conference on the device.

[0238] Specifically, the user uses the recording function of the conference app to save the audio file.

[0239] Step 2:

[0240] The user uploads an audio file from the device to the server.

[0241] Input: Online meeting audio file

[0242] Specifically, the user selects a file from a dedicated form and clicks the upload button.

[0243] Output: The audio file is sent to the server.

[0244] Step 3:

[0245] The server saves the received audio file.

[0246] Input: Uploaded audio file

[0247] Specifically, the server receives the HTTP POST request and temporarily stores it in storage.

[0248] Output: Audio file saved in storage

[0249] Step 4:

[0250] The server converts the speech to text using the Amazon Transcribe API.

[0251] Input: Saved audio file

[0252] Specifically, the server sends the audio file to the API and receives the converted text.

[0253] Output: Text data

[0254] Step 5:

[0255] The server uses OpenAI's generative AI model to analyze the text data and automatically generate minutes and summaries.

[0256] Input: Text data

[0257] Specifically, the server calls the generative AI model and generates minutes and summaries based on the text data.

[0258] Output: Generated minutes and summary

[0259] Step 6:

[0260] The server saves the generated minutes and summaries in a file format and provides them for users to download or sends them by email.

[0261] Input: Generated minutes and summaries

[0262] Specifically, the server returns the file as an HTTP response or sends it using an email sending API.

[0263] Output: Minutes and summary provided to user

[0264] Transportation and hotel arrangements

[0265] Step 1:

[0266] The user inputs the business trip schedule, transportation method, and accommodation conditions into the terminal and transmits them to the server.

[0267] Input: business trip schedule, transportation, accommodation conditions

[0268] Specifically, the user enters the required information into the input form and clicks the submit button.

[0269] Output: Information sent to the server

[0270] Step 2:

[0271] Based on the information received by the server, the server uses the Google Maps API and Expedia API to search for the best transportation and accommodation options.

[0272] Input: business trip schedule, transportation, accommodation conditions

[0273] Specifically, the server calls the API and retrieves the plan that meets the conditions.

[0274] Output: A list of the best transportation and accommodation options

[0275] Step 3:

[0276] The server organizes the search results and generates an HTML page to present to the user.

[0277] Input: A list of the best transportation and accommodation options

[0278] Specifically, the server embeds the data in an HTML template and provides it to the user as a response.

[0279] Output: The list of plans presented to the user

[0280] Step 4:

[0281] The user selects the desired plan and completes the reservation procedure.

[0282] Input: Select your desired plan

[0283] As a specific operation, the user selects a plan from the list and clicks the reservation button.

[0284] Output: Selected plan information

[0285] Step 5:

[0286] The server makes reservations for transportation and accommodation based on the selected plan.

[0287] Input: Selected plan information

[0288] Specifically, the server uses an API to send a request to the transportation or hotel reservation system and complete the reservation.

[0289] Output: Reservation confirmation information

[0290] Step 6:

[0291] The server will send a confirmation email to the user confirming the reservation.

[0292] Input: Reservation confirmation information

[0293] Specifically, the server sends the confirmation information to the email sending API and sends an email to the user.

[0294] Output: Confirmation email sent to the user

[0295] Sales and cost information management and strategy proposals

[0296] Step 1:

[0297] The user prepares sales and cost data as an Excel file and uploads it to the server from the terminal.

[0298] Input: Excel file containing sales data and cost data

[0299] Specifically, the user selects an Excel file from the file selection form and clicks the upload button.

[0300] Output: Excel file sent to the server

[0301] Step 2:

[0302] Read the Excel file received by the server.

[0303] Input: Uploaded Excel file

[0304] Specifically, the server uses the Pandas library to read the Excel file and import it as a data frame.

[0305] Output: Sales and cost data in data frame format

[0306] Step 3:

[0307] The server analyzes the data and visualizes sales and cost trends.

[0308] Input: Sales and cost data in data frame format

[0309] Specifically, the server generates graphs using Matplotlib and Seaborn.

[0310] Output: Visualized sales and cost data (graphs)

[0311] Step 4:

[0312] The server uses the generative AI model to generate data-based strategy proposals and improvement measures.

[0313] Input: Visualized sales and cost data

[0314] Specifically, the server inputs data into the generative AI model and receives automatically generated suggestions.

[0315] Output: Report with strategic proposals and improvement measures

[0316] Step 5:

[0317] The server saves the generated proposal report in PDF format and provides it for the user to download or sends it by email.

[0318] Input: Report with strategic proposals and improvement measures

[0319] Specifically, the server converts the report into PDF format and returns the file as an HTTP response or sends it using an API for sending emails.

[0320] Output: Proposal report provided to the user

[0321] (Application example 1)

[0322] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0323] Freelancers must spend a large amount of time on miscellaneous tasks in the course of their work. This includes a wide range of tasks, including processing transaction data such as invoices and receipts, tedious administrative tasks such as filing tax returns, creating meeting minutes, and even arranging transportation and accommodation. Not only do these tasks significantly reduce the efficiency of their work, but if not managed properly, they also increase the risk of financial loss and wasted time. To solve this problem, a system that automates and efficiently processes these miscellaneous tasks is needed.

[0324] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0325] In this invention, the server includes means for analyzing invoices using an artificial intelligence model and automatically transcribing transaction data extracted from the invoices into tax return documents, means for converting voice data into text and generating meeting minutes and summaries, means for inputting travel itineraries and accommodation requirements and proposing and arranging optimal transportation and accommodations, means for analyzing sales and cost data and making strategic improvement proposals, means for inputting and automatically analyzing payment information, extracting necessary transaction information using optical character recognition technology, organizing the data, and generating tax return data, and means for tracking expenses and sales and displaying real-time financial status on a dashboard. This allows freelancers to significantly reduce the time they spend on miscellaneous tasks and focus on their core business.

[0326] An "artificial intelligence model" is a system that uses machine learning algorithms to analyze data, recognize patterns, and make predictions and classifications.

[0327] An "invoice" is a document that lists the details and amount of a transaction and requests payment from the other party.

[0328] "Transaction data" is a collection of information about economic activities such as sales and expenditures.

[0329] "Tax return documents" are documents submitted by individuals and corporations to the tax office for tax returns.

[0330] "Audio data" refers to data in which audio is recorded in digital format.

[0331] "Minutes" are documents that record the contents of a meeting and the decisions made.

[0332] A "summary" is a document that briefly summarizes the main points of a meeting or other event.

[0333] A "travel itinerary" is a specific schedule for a business trip or trip.

[0334] "Accommodation conditions" refer to the conditions and requirements required of accommodation facilities.

[0335] "Transportation" refers to the means or methods of transportation used for travel.

[0336] "Accommodation" refers to a place where you stay, such as a hotel or lodging, during a trip or business trip.

[0337] "Sales data" is information relating to income obtained through a transaction.

[0338] "Cost data" refers to information relating to the costs incurred in carrying out a transaction or business.

[0339] "Strategic improvement proposals" are specific proposals for improving efficiency and profits based on the results obtained from data analysis.

[0340] "Payment information" refers to information regarding the payment method and amount used in a transaction.

[0341] "Optical character recognition technology" is a technology for extracting character information from images.

[0342] "Organizing data" refers to classifying information and organizing it systematically.

[0343] A "dashboard" is a screen or tool for visually displaying data and information.

[0344] This invention is a system that supports freelancers in efficiently handling daily chores, allowing them to concentrate on their primary work. The system of the present invention is composed of multiple means with various functions.

[0345] First, the server uses an artificial intelligence model to analyze the invoice uploaded by the user and extract transaction data from the invoice. Optical character recognition (OCR) technology is used to extract text information from PDF files or scanned images, automatically determining the necessary transaction information (e.g., amount, transaction date, and customer). Software used includes pytesseract (OCR) and pdf2image (PDF to image generation).

[0346] For example, when a user uploads a PDF of an invoice to their device, it is sent from the device to the server, which then uses OCR technology to scan the invoice and extract the necessary information, which is then stored in a database on the server and automatically transcribed into the format for tax return documents.

[0347] Next, it includes a function to generate meeting minutes and summaries using audio data. After the user saves the audio file of an online meeting on their device, they upload it from their device to the server. The server uses a speech recognition engine to convert the audio data into text, and then uses a generative AI model to automatically generate minutes and summaries. During this process, the Google Speech-to-Text API and generative AI models are used to create highly accurate minutes.

[0348] On the other hand, the server inputs the user's travel schedule and accommodation requirements, and suggests the optimal means of transportation and accommodation, and also provides the function of making arrangements if necessary. When the user inputs the business trip schedule and desired means of transportation and accommodation requirements on the terminal, the server performs an online search and lists the optimal plans. The reservation procedure is then carried out based on the options selected by the user.

[0349] It also includes a means for management, analysis, and improvement proposals for sales and cost data. When users upload sales and cost data from their devices to the server, the server analyzes the data and extracts sales and cost trends. It uses a generative artificial intelligence model to automatically generate strategic improvement proposals based on the data and creates reports to provide to users.

[0350] It also uses payment information input and automatic analysis functions, extracts necessary transaction information using optical character recognition technology, and automatically organizes the data, automatically generating data for tax returns. It also provides a dashboard that tracks expenses and sales in real time and displays financial status.

[0351] For example, when a user makes a payment through a terminal, the transaction information is automatically entered into the server, and optical character recognition technology is used to extract the necessary data from receipts and invoices, automatically organizing the data and enabling the generation of tax return data and real-time tracking of expenses and sales.

[0352] Example prompt sentence:

[0353] "Parse the uploaded invoice PDF file and extract the invoice date, amount, and account."

[0354] "Convert audio data from online meetings to text and generate minutes and summaries."

[0355] "Search and suggest the best transportation and accommodation options based on the travel dates and transportation preferences entered by the user."

[0356] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0357] Step 1:

[0358] The user prepares a scanned invoice or a PDF file on the terminal and uploads the invoice file from the terminal to the server. The input is the scanned invoice or PDF file, and the output is sending the file to the server. Specifically, the user takes a photo of the invoice using a smartphone or PC, or selects the scanned PDF file and uploads it to the server.

[0359] Step 2:

[0360] The server receives the uploaded file and uses optical character recognition (OCR) technology to scan the file and extract text information. The input is the uploaded invoice file, and the output is the extracted text information. Specifically, the server uses pytesseract to extract text data from PDFs and images, and extracts necessary data such as transaction date, amount, and client.

[0361] Step 3:

[0362] The server stores the extracted data in an internal database and automatically transcribes it into the format of tax return documents. The input is the extracted transaction data, and the output is the generation of tax return documents. Specifically, the server organizes the data using pandas, transcribes it into an Excel template, and generates the final tax return documents.

[0363] Step 4:

[0364] A user saves an audio file of an online conference on their device and uploads it to a server from their device. The input is the audio file of the online conference, and the output is the transfer of the audio file to the server. Specifically, the user uses the conference recording function to obtain the audio data and uploads it to the server.

[0365] Step 5:

[0366] The server converts the received audio file into text using a speech recognition engine, and automatically generates minutes and summaries using a generative AI model. The input is an audio file, and the output is textual minutes and summaries. Specifically, the server converts audio into text using the Google Speech-to-Text API, and generates minutes and summaries using a generative AI model.

[0367] Step 6:

[0368] The user inputs the dates of a business trip, specifies the desired means of transportation and accommodation conditions, and sends the data to the server. The input is the business trip dates, means of transportation, and accommodation conditions, and the output is sending the information to the server. Specifically, the user uses a terminal to enter the necessary information into an input form and sends it to the server.

[0369] Step 7:

[0370] Based on the information received, the server performs an online search, lists the optimal means of transportation and accommodation, and proposes it to the user. The input is the user's travel schedule and accommodation requirements, and the output is a list of proposals. Specifically, the server uses APIs and web scraping to collect information on means of transportation and accommodation, and then lists the optimal plans.

[0371] Step 8:

[0372] The server makes reservations for transportation and accommodation based on the options selected by the user. The input is the plan selected by the user, and the output is a confirmed reservation. Specifically, the server uses the reservation site or API to process the transportation and accommodation reservations.

[0373] Step 9:

[0374] The user uploads data related to sales and costs from the terminal to the server. The input is sales data and cost data, and the output is sending the data to the server. Specifically, the user prepares the data in a format such as an Excel spreadsheet and uploads it from the terminal to the server.

[0375] Step 10:

[0376] The server analyzes the data and automatically generates strategic improvement proposals using a generative AI model. The input is sales data and cost data, and the output is strategic improvement proposals. Specifically, the server analyzes the data using data analysis tools and generative AI models, and generates improvement proposals.

[0377] Step 11:

[0378] When a user makes a payment, they enter their payment information through their device, and the server receives and analyzes the payment data. The input is payment information, and the output is analyzed payment data. Specifically, when a user makes a purchase or uses a service, they enter their payment information into the app, and the server receives and analyzes that information.

[0379] Step 12:

[0380] The server uses optical character recognition technology to extract the necessary transaction information and automatically organize the data. The input is payment information or scanned receipts, and the output is organized transaction information. Specifically, the server uses pytesseract to extract the necessary transaction information from receipts or invoices and saves it in an internal database.

[0381] Step 13:

[0382] The server generates tax return data based on the organized transaction information and provides a dashboard for tracking expenses and sales in real time. The input is transaction information, and the output is tax return data and a dashboard. Specifically, the server uses pandas to organize the data and perform tracking and display.

[0383] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0384] The system of the present invention provides a means for freelancers to efficiently handle the many miscellaneous tasks they must perform on a daily basis, helping them to focus on their core business. In particular, by combining it with an emotion engine that recognizes the user's emotions, a function is added to provide services that are more suited to the user. Each function of the system of the present invention and its specific embodiments are described below.

[0385] Support for automatic creation of tax return documents

[0386] 1. Upload your invoice:

[0387] The user prepares a scanner or PDF file of the invoice on the terminal.

[0388] The user uploads the invoice file from the terminal to the server.

[0389] 2. Invoice analysis:

[0390] The server receives the uploaded invoice file and uses optical character recognition (OCR) technology to scan the invoice and extract text information.

[0391] The server automatically determines necessary data such as transaction date, amount, and trading partner from the extracted information.

[0392] 3. Data transcription and tax return generation:

[0393] The server stores the identified data in an internal Excel template.

[0394] The server arranges the data according to the tax return format and automatically fills in the necessary sections.

[0395] The server generates the completed Excel file and tax return, which the user can download from their device or the server can email to the user.

[0396] Examples:

[0397] When a user uploads a PDF of an invoice to their device, the server uses OCR technology to extract the necessary information from the PDF (e.g., amount 100,000 yen, transaction date October 3, 2023, business partner X), automatically reflects this information in an Excel spreadsheet and tax return, and provides it to the user.

[0398] Creating meeting minutes and summaries

[0399] 1. Uploading audio data:

[0400] A user saves an audio file of an online conference on a terminal and uploads the audio file from the terminal to a server.

[0401] 2. Audio data conversion and transcription:

[0402] The server converts the received audio file into text using a speech recognition engine.

[0403] Based on the converted text, minutes and summaries are automatically generated using a generative artificial intelligence model.

[0404] 3. Add sentiment analysis:

[0405] The server uses an emotion engine based on the voice data and text data to analyze the emotions of the conference participants.

[0406] Based on the analyzed emotional data, the content of the minutes and summaries is adjusted to generate documents that incorporate emotional nuances.

[0407] 4. Provision of minutes and summaries:

[0408] The server stores the completed minutes and summary as files, which the user can download from their terminal or send to them by email.

[0409] Examples:

[0410] When a user uploads an audio recording of an online meeting to their device, the server converts the audio into text and uses an emotion engine to analyze the emotions of the meeting participants based on the text and audio data. The minutes and summaries generated reflect the emotional nuances of the participants and are provided to the user.

[0411] Transportation and hotel arrangements

[0412] 1. Enter your travel dates and desired conditions:

[0413] The user inputs the business trip schedule, transportation means, and accommodation requirements into the terminal, and transmits the information from the terminal to the server.

[0414] 2. Travel and accommodation search and suggestions:

[0415] The server performs an online search based on the received information and lists the best transportation and accommodation options.

[0416] It uses an emotion engine to recognize the user's emotional state and generate optimal suggestions based on that state.

[0417] 3. Proposal and Arrangement:

[0418] The server makes reservations for transportation and accommodation based on the options selected by the user on the terminal.

[0419] The server will send a confirmation email to the user confirming the reservation.

[0420] Examples:

[0421] When a user inputs the dates of their business trip and specifies their preferred means of transportation and accommodation, the server uses an emotion engine to understand the user's emotional state. For example, if the user is feeling stressed, the server will suggest comfortable means of transportation and accommodations that will allow them to relax. The server then makes a reservation for the plan selected by the user and sends a confirmation email.

[0422] Sales and cost information management and strategy proposals

[0423] 1. Upload sales and cost data:

[0424] The user prepares data on sales and costs on the terminal and uploads it to the server from the terminal.

[0425] 2. Data analysis and strategy proposal:

[0426] The server analyzes the received data and extracts sales and cost trends.

[0427] The server uses a generative artificial intelligence model to automatically generate data-based strategic proposals and improvement measures.

[0428] 3. Adding emotional support:

[0429] The server uses an emotion engine to analyze the user's emotional state and adjust the content of the strategy proposal.

[0430] For example, if a user is feeling stressed, we suggest solutions that require less effort.

[0431] 4. Proposal report provided:

[0432] The server compiles the generated proposals in the form of a report, which the user can download from their terminal or the server can send to the user by email.

[0433] Examples:

[0434] When a user uploads sales and cost data to their device, the server analyzes it and uses a generative AI model to generate a suggestion such as "reducing advertising costs by 20% will increase profits by 15%." At the same time, an emotion engine analyzes the user's emotional state and suggests solutions that require less effort to implement if the user is feeling stressed. The generated suggestion report is then provided to the user.

[0435] As described above, the system of the present invention efficiently handles a wide range of miscellaneous tasks, allowing freelancers to focus their time and efforts on their core business. The addition of an emotion engine provides services that are more suited to users, improving their work efficiency and satisfaction.

[0436] The processing flow will be explained below.

[0437] Support for automatic creation of tax return documents

[0438] Step 1:

[0439] The user prepares a PDF file of the invoice on the terminal.

[0440] Step 2:

[0441] The user uploads the invoice file from the terminal to the server.

[0442] Step 3:

[0443] The server receives the uploaded invoice file.

[0444] Step 4:

[0445] The server uses optical character recognition (OCR) technology to extract text information from the invoice.

[0446] Step 5:

[0447] The server automatically determines necessary data such as transaction date, amount, and trading partner from the extracted information.

[0448] Step 6:

[0449] The server stores the identified data in an internal Excel template.

[0450] Step 7:

[0451] The server arranges the data according to the tax return format and automatically fills in the necessary sections.

[0452] Step 8:

[0453] The server generates the completed Excel file and tax return.

[0454] Step 9:

[0455] The server analyzes the user's emotions using an emotion engine and adds a message to the proposal that gives the user a sense of security.

[0456] Step 10:

[0457] The user downloads the generated file from the device, or the server sends it to the user by email.

[0458] Creating meeting minutes and summaries

[0459] Step 1:

[0460] The user saves the audio file of the online conference on the device.

[0461] Step 2:

[0462] The user uploads an audio file from the device to the server.

[0463] Step 3:

[0464] The server converts the received audio file into text using a speech recognition engine.

[0465] Step 4:

[0466] The server inputs the converted text into a generative AI model.

[0467] Step 5:

[0468] The server uses generative AI models to create minutes and summaries.

[0469] Step 6:

[0470] The server uses an emotion engine based on the voice data and text data to analyze the emotions of the conference participants.

[0471] Step 7:

[0472] Based on the results of the sentiment analysis, the server adjusts the content of the minutes and summary to generate documents that reflect emotional nuances.

[0473] Step 8:

[0474] The server saves the created minutes and summaries as files.

[0475] Step 9:

[0476] The user downloads the minutes and summary from their terminal, or the server sends them to the user by email.

[0477] Transportation and hotel arrangements

[0478] Step 1:

[0479] The user inputs the business trip schedule, transportation method, and accommodation requirements into the terminal.

[0480] Step 2:

[0481] The user sends input information from the terminal to the server.

[0482] Step 3:

[0483] The server performs an online search based on the received information.

[0484] Step 4:

[0485] The server will list the best transportation and accommodation options.

[0486] Step 5:

[0487] The server analyzes the listed candidates using an emotion engine to highlight the best candidates based on the user's emotional state.

[0488] Step 6:

[0489] The server presents the list to the user with comments that match the emotions.

[0490] Step 7:

[0491] The user selects from the candidates via the terminal and contacts the server.

[0492] Step 8:

[0493] The server makes transportation and accommodation reservations based on the user's selections.

[0494] Step 9:

[0495] The server will send a confirmation email to the user confirming the reservation.

[0496] Sales and cost information management and strategy proposals

[0497] Step 1:

[0498] The user provides data on sales and costs on the terminal.

[0499] Step 2:

[0500] The user uploads data from the device to the server.

[0501] Step 3:

[0502] The server receives the uploaded data.

[0503] Step 4:

[0504] The server analyzes sales and cost data to extract trends and patterns.

[0505] Step 5:

[0506] The server uses a generative AI model based on the extracted data to automatically generate strategic proposals and improvement measures.

[0507] Step 6:

[0508] The server uses an emotion engine to analyze the user's emotional state and adjust the content of the strategy proposal.

[0509] Step 7:

[0510] The server compiles the suggestions, including sentiment-enabled comments, in the form of a report.

[0511] Step 8:

[0512] The user downloads the proposal report from the terminal, or the server sends it to the user by email.

[0513] Example 2

[0514] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0515] Freelancers have to handle a wide variety of chores on a daily basis, which can prevent them from concentrating on their primary work. This often leads to emotional stress, which can reduce work efficiency and satisfaction. There is a need for a system that can solve the above problems, improve the work efficiency of freelancers, and provide appropriate support based on their emotions.

[0516] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for analyzing invoices using an artificial intelligence model and automatically transcribing transaction data extracted from the invoices into tax return documents; means for scanning invoices using optical character recognition technology and extracting text information; means for converting voice data to text and generating meeting minutes and summaries; means for analyzing the emotions of meeting participants based on the voice data and text data using an emotion engine and adjusting the content of the minutes and summaries; means for inputting travel itineraries and accommodation requirements and proposing and arranging optimal transportation and accommodations; means for recognizing the user's emotional state using the emotion engine and generating optimal proposals based on that state; means for analyzing sales data and cost data and making strategic improvement proposals; and means for analyzing the user's emotional state using the emotion engine and adjusting the content of strategic proposals. This frees freelancers from mundane tasks, allowing them to focus on their core business and receive optimal support tailored to their emotions.

[0517] An "artificial intelligence model" is an algorithm that uses machine learning and deep learning to analyze data and make predictions and classifications.

[0518] "Optical character recognition technology" is a technology that recognizes character information contained in images and PDFs and extracts it as text data.

[0519] A "voice recognition engine" is a technology that analyzes voice data and converts it into text data.

[0520] An "emotion engine" is a technology that analyzes emotions based on voice and text data and uses the results to provide appropriate feedback and adjustments.

[0521] "Tax return documents" are income and expenditure reports to be submitted to tax authorities, and are documents into which transaction data is transcribed.

[0522] "Minutes" are a written record of the contents of a meeting.

[0523] A "summary" is an overview that summarizes the main points of detailed data or documents.

[0524] "Transportation" refers to the vehicles and methods of travel used by people to get around.

[0525] "Accommodation" refers to a place where travelers and business people stay, and includes hotels, inns, etc.

[0526] "Sales data" is a record of income earned through business activities.

[0527] "Cost data" is a record of the costs incurred in business activities.

[0528] "Strategic improvement proposals" involve proposing specific measures to improve business efficiency and profitability based on data analysis.

[0529] The system of the present invention provides a means for freelancers to efficiently handle the many miscellaneous tasks they must perform on a daily basis, helping them to focus on their core business. In particular, by combining it with an emotion engine that recognizes the user's emotions, a function is added to provide services that are more suited to the user. Each function of the system of the present invention and its specific embodiments are described below.

[0530] Support for automatic creation of tax return documents

[0531] 1. The user prepares a scanned image or PDF file of the invoice on the terminal and uploads it to the server through the system's web interface.

[0532] 2. The server receives the uploaded invoice file and uses an OCR engine (e.g., Tesseract or Google Cloud Vision API) to scan the image of the file and extract text information.

[0533] 3. The server analyzes the extracted text information, automatically identifies important data such as transaction date, amount, and trading partner, and saves it in an internal Excel template.

[0534] 4. The server places the data in the appropriate location according to the tax return format and automatically fills in the necessary sections.

[0535] 5. The server generates the completed Excel file and tax return, which the user can download from their device or the server will email to the user.

[0536] As a specific example, when a user uploads a PDF of an invoice to a terminal, the server uses OCR technology to extract information such as "amount 100,000 yen," "transaction date October 3, 2023," and "client XX company" from the PDF, and automatically reflects this information in an Excel spreadsheet and tax return.

[0537] Creating meeting minutes and summaries

[0538] 1. The user saves the audio file of the online meeting on their device and uploads it to the server through the system's web interface.

[0539] 2. The server converts the received audio file into text using a speech recognition engine (e.g., Google Speech-to-Text API or IBM Watson).

[0540] 3. The server automatically generates minutes and summaries based on the converted text using a generative AI model (e.g., OpenAI's GPT or BERT).

[0541] 4. The server uses an emotion engine (e.g., Affectiva or IBM Tone Analyzer) to analyze the emotions of the meeting participants based on the voice and text data, and adjusts the content of the minutes and summary based on the analyzed emotion data.

[0542] 5. The server stores the completed minutes and summary, and the user can download them from their device, or the server will send them to the user by email.

[0543] For example, when a user uploads an audio recording of an online meeting, the server converts the audio into text and analyzes the emotions using an emotion engine, so the generated minutes and summaries reflect the emotional nuances of the participants.

[0544] Transportation and hotel arrangements

[0545] 1. The user enters the business trip schedule, transportation method, and accommodation requirements on the terminal and sends the information to the server through the system's web interface.

[0546] 2. The server performs an online search based on the received information and lists the best transportation and accommodation options. It uses an emotion engine to recognize the user's emotional state and generate optimal suggestions based on that state.

[0547] 3. Based on the user's selection, the server will process the reservation for transportation and accommodation, and automatically send the user a confirmation email.

[0548] As a concrete example, when a user inputs the dates of a business trip, the server uses an emotion engine to understand the user's emotional state, and if the user is feeling stressed, it will suggest and make reservations for comfortable transportation and accommodations where they can relax.

[0549] Sales and cost information management and strategy proposals

[0550] 1. The user prepares sales and cost data as a spreadsheet or CSV file and uploads it to the server through the system's web interface.

[0551] 2. The server analyzes the received data, extracts sales and cost trends, and automatically generates strategic improvement proposals based on the data using a generative AI model.

[0552] 3. The server uses an emotion engine to analyze the user's emotional state and adjust the proposed strategy accordingly. If the user is feeling stressed, it will suggest a solution that requires less effort.

[0553] 4. The server saves the completed proposal report and the user can download it from their device, or the server will send it to the user by email.

[0554] As a concrete example, when a user uploads sales and cost data, the server generates a specific suggestion such as "reducing advertising costs by 20% will increase profits by 15%," and provides a low-stress solution based on the results of analysis by the emotion engine.

[0555] As described above, the system of the present invention efficiently handles a wide range of miscellaneous tasks, allowing freelancers to focus their time and efforts on their core work, and by adding an emotion engine, it provides a service that is more suited to users, improving work efficiency and satisfaction.

[0556] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0557] Support for automatic creation of tax return documents

[0558] Processing Steps

[0559] Step 1: Prepare and upload your invoice

[0560] Users scan invoices and save them as image or PDF files on their devices.

[0561] The user opens a browser on their device, accesses the system's web interface, and logs in.

[0562] The user clicks the "Upload Invoice" button, selects the invoice file from the file selection dialog, and presses the "Upload" button.

[0563] Input: Scanned invoice image / PDF file

[0564] Output: Invoice file is sent to the server

[0565] Step 2: Parse the invoice

[0566] The server receives the uploaded invoice file.

[0567] The server calls an OCR engine (e.g., Tesseract or Google Cloud Vision API) to extract text data from the invoice image / PDF.

[0568] Input: Uploaded invoice file

[0569] Output: Extracted text data

[0570] Step 3: Automatically transcribe data

[0571] The server analyzes the extracted text data and automatically determines important information such as the transaction date, amount, and trading partner.

[0572] The server stores the determined data in an internal Excel template, for example, entering the transaction date in cell A1 and the amount in cell B1.

[0573] Input: Extracted text data

[0574] Output: Transcribed Excel data

[0575] Step 4: Generate your tax return documents

[0576] The server places the data in the appropriate location according to the tax return format and automatically fills in the necessary sections.

[0577] The server generates the completed Excel file and tax return.

[0578] Input: Transcribed Excel data

[0579] Output: Generated tax return documents

[0580] Step 5: Submit your tax return documents

[0581] The server temporarily stores the generated tax return documents and provides a link for the user to download or sends them by email.

[0582] Users can download it from their device or open the email attachment.

[0583] Input: Generated tax return documents

[0584] Output: User retrieves tax return documents

[0585] ---

[0586] Creating meeting minutes and summaries

[0587] Processing Steps

[0588] Step 1: Save and upload your audio data

[0589] Users save the audio files of online meetings to their devices.

[0590] The user accesses the system's web interface, logs in, clicks the "Upload Audio Data" button, selects an audio file, and presses the "Upload" button.

[0591] Input: Meeting audio file

[0592] Output: The audio file is uploaded to the server.

[0593] Step 2: Convert the audio data

[0594] The server converts the received audio file into text using a speech recognition engine (e.g., Google Speech-to-Text API or IBM Watson).

[0595] Input: Audio file

[0596] Output: Converted text data

[0597] Step 3: Generate minutes and summaries

[0598] The server automatically generates minutes and summaries based on the converted text using a generative AI model (e.g., OpenAI's GPT or BERT).

[0599] Input: Converted text data

[0600] Output: Generated minutes and summary

[0601] Step 4: Add sentiment analysis

[0602] The server uses an emotion engine (e.g., Affectiva or IBM Tone Analyzer) based on voice and text data to analyze the emotions of conference participants.

[0603] Based on the analyzed emotional data, the server adjusts the content of the minutes and summary, creating documents that incorporate emotion.

[0604] Input: Audio data, text data

[0605] Output: Minutes and summaries reflecting sentiment analysis data

[0606] Step 5: Provide minutes and summaries

[0607] The server temporarily stores the generated minutes and summaries and provides a link for users to download them or sends them by email.

[0608] Users can download it from their device or open the email attachment.

[0609] Input: Generated minutes and summary

[0610] Output: User gets minutes and summary

[0611] ---

[0612] Transportation and hotel arrangements

[0613] Processing Steps

[0614] Step 1: Enter travel dates and accommodation requirements

[0615] The user inputs the business trip schedule, transportation method, and accommodation requirements into the terminal.

[0616] The user logs into the system's web interface, enters information in the "Travel and Accommodation Input" form, and presses the "Submit" button.

[0617] Input: business trip schedule, transportation, accommodation conditions

[0618] Output: The entered business trip information is sent to the server.

[0619] Step 2: Find transportation and accommodation

[0620] Based on the information received, the server uses online search engines (e.g. Google Flights or Booking.com API) to list the best transportation and accommodation options.

[0621] The server uses an emotion engine to recognize the user's emotional state and generate optimal suggestions according to that state.

[0622] Input: Business trip information, emotional state

[0623] Output: A list of the best transportation options and accommodations

[0624] Step 3: View and select suggestions

[0625] The user can check the suggested options through the device's browser and select the desired transportation and accommodation.

[0626] Input: Suggested candidates

[0627] Output: User selected candidate

[0628] Step 4: Arrange your booking

[0629] The server processes the reservations for the selected transportation and accommodation.

[0630] The server will automatically send a confirmation email to the user confirming the reservation.

[0631] Input: User selected candidate

[0632] Output: Reservation confirmation email

[0633] ---

[0634] Sales and cost information management and strategy proposals

[0635] Processing Steps

[0636] Step 1: Upload your sales and cost data

[0637] Users prepare sales and cost data on their devices as spreadsheets or CSV files.

[0638] The user accesses the system's web interface, clicks the "Upload Cost of Sales Data" button, selects the file, and presses the "Upload" button.

[0639] Input: Sales and cost data spreadsheet / CSV file

[0640] Output: Sales and cost data is uploaded to the server

[0641] Step 2: Analyze the data

[0642] The server analyzes the received data and extracts sales and cost trends.

[0643] Input: Uploaded sales and cost data

[0644] Output: Analyzed sales and cost trends

[0645] Step 3: Generate strategic proposals

[0646] The server uses generative AI models to automatically generate data-based strategic proposals and improvement measures.

[0647] Input: Analyzed sales and cost trends

[0648] Output: Generated strategy proposals

[0649] Step 4: Adding emotional support

[0650] The server uses an emotion engine to analyze the user's emotional state and adjust the content of the strategy proposal.

[0651] For example, if a user is feeling stressed, we suggest solutions that require less effort.

[0652] Input: Strategy proposal, user's emotional state

[0653] Output: Strategy suggestions tailored to emotional state

[0654] Step 5: Providing a proposal report

[0655] The server will temporarily store the generated proposal report and provide a link for the user to download or send it via email.

[0656] Users can download it from their device or open the email attachment.

[0657] Input: Generated proposal report

[0658] Output: User gets proposal report

[0659] (Application example 2)

[0660] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0661] The purpose of this invention is to solve the inefficiencies in the various tasks and work processes that freelancers and factory workers face on a daily basis. In particular, in addition to processing invoices, taking meeting minutes, arranging travel and accommodation, and managing sales and costs, the invention aims to improve work efficiency and reduce worker stress by analyzing the emotional state of workers in real time and providing appropriate work instructions.

[0662] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0663] In this invention, the server includes means for analyzing invoices using an artificial intelligence model and automatically transcribing transaction data extracted from the invoices into tax return documents, means for converting voice data into text and generating meeting minutes and summaries, means for inputting travel itineraries and accommodation requirements and proposing and arranging optimal transportation and accommodations, and means for analyzing the emotional state of workers using an emotion recognition model and generating optimal work instructions. This makes it possible to centrally streamline multiple tasks and further optimize the work environment through emotion analysis.

[0664] An "artificial intelligence model" is a computer program or algorithm that automatically learns from data to perform specific tasks or analyses.

[0665] An "invoice" is a document that indicates payment obligations for a transaction and includes details such as the transaction amount and date.

[0666] "Transaction data" refers to information related to a transaction, including, for example, the transaction date, amount, and counterparty.

[0667] "Tax return documents" refers to a set of documents regarding income and taxable items that an individual or corporation submits to the tax office.

[0668] "Audio data" means data in digital or analog form that is a recording of the human voice.

[0669] "Meeting minutes" are documents that record the contents and decisions of a meeting.

[0670] A "summary" is a short document that summarizes detailed information.

[0671] A "travel itinerary" is a schedule that describes how to travel within a specific period of time.

[0672] "Accommodation conditions" is information indicating the conditions and wishes required when using accommodation facilities such as hotels.

[0673] "Transportation" is the means used to travel from one place to another, examples being trains, buses, and airplanes.

[0674] "Accommodation" refers to a facility that provides a place for people to stay temporarily, and examples include hotels and guesthouses.

[0675] "Sales data" refers to data that represents information about the revenue earned by a company or individual over a certain period of time.

[0676] "Cost data" refers to data that represents information about expenses and costs incurred within a particular period of time.

[0677] An "emotion recognition model" is an algorithm or program for determining a person's emotional state based on audio or image data.

[0678] "Work instructions" are specific instructions or commands given to perform a specific task.

[0679] A "server" is a computer system that provides data and services over a network.

[0680] The system of the present invention is designed to improve the work and work efficiency of freelancers and factory workers. The system includes the following means:

[0681] 1. Automated invoice processing methods:

[0682] The server receives invoice files uploaded by users via their terminals. It uses optical character recognition (OCR) technology to extract transaction data from the invoices and automatically transcribes it into tax return documents. This reduces the burden of invoice processing for users. The software used includes OpenCV and Tesseract for OCR technology.

[0683] 2. Meeting minutes and summary generators:

[0684] The server converts user-provided voice data into text using a speech recognition engine. It then uses a generative artificial intelligence model to automatically generate minutes and summaries from the text. It also uses an emotion engine to analyze the emotions of meeting participants and reflect them in the minutes and summaries. Software used includes TensorFlow and Transformers.

[0685] 3. Travel and accommodation arrangements:

[0686] When a user inputs their travel itinerary and accommodation requirements, the server searches for, proposes, and arranges suitable transportation and accommodation options. Based on emotion analysis, the server generates optimal proposals tailored to the user's emotional state. The technologies used include emotion recognition models and a database search engine.

[0687] 4. Revenue and cost control measures:

[0688] The server analyzes sales and cost data uploaded by users from their devices and uses a generative AI model to make strategic improvement proposals. It also generates proposals that take into account the user's emotional state based on emotion recognition. The software used includes data analysis tools and generative AI models.

[0689] 5. Emotion recognition and work instruction generation method:

[0690] The server analyzes the worker's emotional state in real time and generates optimal work instructions. To do this, it monitors the worker's face using smart glasses and a camera and determines their emotions using an emotion recognition model. Based on the results of the determination, it uses a generative AI model to provide optimal instructions. For example, if a worker is feeling stressed, the instruction to "take a short break" is automatically generated and displayed on the smart glasses' display.

[0691] Example prompt for a generative AI model:

[0692] "Generate short messages to workers who show signs of stress or fatigue, directing them to appropriate actions."

[0693] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0694] Step 1:

[0695] The user prepares the bill file on the terminal and uploads it to the server.

[0696] Input: PDF or image file of invoice

[0697] Output: Invoice file sent to the server

[0698] Specific operation: The user selects an invoice file from the terminal and uploads it to the server via a web form or a dedicated application.

[0699] Step 2:

[0700] The server receives the invoice file and scans it using optical character recognition (OCR) technology to extract text information.

[0701] Input: Submitted invoice file

[0702] Output: Extracted text information (transaction date, amount, trading partner, etc.)

[0703] What it does: The server uses OCR software (e.g., Tesseract) to extract text from the image, then filters out the necessary data (transaction date, amount, counterparty).

[0704] Step 3:

[0705] The server automatically transcribes the extracted transaction data into tax return documents.

[0706] Input: Extracted text information

[0707] Output: Excel or PDF file of tax return documents

[0708] Specific operation: The server embeds the extracted data into a template in the database and formats it as tax return documents.

[0709] Step 4:

[0710] The user saves the audio data on the device and uploads it to the server.

[0711] Input: Audio file

[0712] Output: Audio file sent to the server

[0713] Specific operation: After the meeting ends, the user uploads the recorded audio file to the server via a web form or a dedicated application.

[0714] Step 5:

[0715] The server converts the received audio file into text using a speech recognition engine.

[0716] Input: Submitted audio file

[0717] Output: Text data

[0718] Specific operation: The server uses a speech recognition engine (e.g., Google Speech-to-Text) to convert the voice data into text.

[0719] Step 6:

[0720] The server uses the converted text data to automatically generate minutes and summaries using a generative AI model.

[0721] Input: Text data

[0722] Output: Text data of minutes and summary

[0723] Specific operation: The server uses a generative AI model (e.g., GPT-3.5) to summarize the text data and create minutes and summaries.

[0724] Step 7:

[0725] The server uses an emotion recognition model based on the text data to analyze the emotions of the conference participants.

[0726] Input: Converted text and audio data

[0727] Output: Emotional state of meeting participants

[0728] What it does: The server uses an emotion recognition model (e.g., Emotion Recognition API) to analyze the emotional state from the text and audio data and adds that information to the minutes and summary.

[0729] Step 8:

[0730] The user inputs the travel schedule and accommodation requirements into the terminal and transmits them to the server.

[0731] Input: Travel schedule and accommodation conditions

[0732] Output: Travel and accommodation information sent to the server

[0733] Specific operation: The user uses the terminal to enter the business trip dates and accommodation requirements, and sends them to the server via a special form.

[0734] Step 9:

[0735] The server receives the travel schedule and accommodation information, and searches for and suggests suitable transportation and accommodation.

[0736] Input: Travel schedule and accommodation information

[0737] Output: A list of transportation and accommodation suggestions

[0738] Specific operation: The server uses online search engines and APIs to list and suggest optimal transportation and accommodation options.

[0739] Step 10:

[0740] The server analyzes the user's emotional state and adjusts the suggestions accordingly.

[0741] Input: User travel schedule and accommodation information, user emotion data

[0742] Output: A list of travel and accommodation plan suggestions based on emotions

[0743] Specific operation: The server uses an emotion recognition model to analyze the user's emotional state and generate optimal suggestions based on that state.

[0744] Step 11:

[0745] The user inputs sales data and cost data into the terminal and uploads it to the server.

[0746] Input: Sales and cost data files

[0747] Output: Sales and cost data sent to the server

[0748] Specific operation: The user uploads sales and cost data in file format from the terminal to the server.

[0749] Step 12:

[0750] The server analyzes the received sales and cost data and automatically generates strategic improvement proposals using a generative AI model.

[0751] Input: Sales data and cost data

[0752] Output: Report of strategic improvement recommendations

[0753] How it works: The server uses data analysis tools (e.g., Pandas and NumPy) to analyze trends in sales and cost data and uses generative AI models to create improvement suggestions.

[0754] Step 13:

[0755] The server uses an emotion recognition model to analyze the user's emotional state and adjust the suggestions.

[0756] Input: User sales data, cost data, and emotion data

[0757] Output: A report of improvement suggestions based on sentiment

[0758] Specific operation: The server analyzes the user's emotional state and suggests solutions that require less effort if the user is feeling stressed.

[0759] Step 14:

[0760] The server analyzes the worker's emotional state in real time and generates optimal work instructions.

[0761] Input: Facial image and voice data of the worker

[0762] Output: Real-time work instructions

[0763] Specific operation: The server analyzes the worker's facial image captured by the smart glasses' camera using an emotion recognition model (e.g., TensorFlow), and then uses a generative AI model to generate work instructions based on the worker's emotional state, which are then displayed on the smart glasses' display.

[0764] As a specific example of operation, if it is analyzed that a worker is feeling stressed, the instruction to "take a short break" will be displayed on the smart glasses' display.

[0765] Example prompt for the generative AI model: "Generate a short message to guide workers who show signs of stress or fatigue on appropriate actions."

[0766] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0767] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0768] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0769] [Second embodiment]

[0770] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0771] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0772] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0773] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0774] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0775] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0776] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0777] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0778] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0779] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0780] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0781] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0782] The system of the present invention provides a means for freelancers to efficiently handle the many miscellaneous tasks they must perform on a daily basis, thereby helping them to focus on their core business. Below, we will explain each function of the system of the present invention and its specific embodiment.

[0783] Support for automatic creation of tax return documents

[0784] 1. Upload your invoice:

[0785] The user prepares a scanner or PDF file of the invoice on the terminal.

[0786] The user uploads the invoice file from the terminal to the server.

[0787] 2. Invoice analysis:

[0788] The server receives the uploaded file and uses optical character recognition (OCR) technology to scan the invoice and extract the text information.

[0789] The server automatically determines necessary data such as transaction date, amount, and trading partner from the extracted information.

[0790] 3. Data transcription and tax return generation:

[0791] The server stores the identified data in an internal Excel template.

[0792] The server arranges the data according to the tax return format and automatically fills in the necessary sections.

[0793] The server generates the completed Excel file and tax return, which the user can download from their device or the server can email to the user.

[0794] Examples:

[0795] When a user uploads a PDF of an invoice to their device, the server uses OCR technology to extract the necessary information from the PDF (e.g., amount 100,000 yen, transaction date October 3, 2023, business partner X), automatically reflects this information in an Excel spreadsheet and tax return, and provides it to the user.

[0796] Creating meeting minutes and summaries

[0797] 1. Uploading audio data:

[0798] A user saves an audio file of an online conference on a terminal and uploads the audio file from the terminal to a server.

[0799] 2. Audio data conversion and transcription:

[0800] The server converts the received audio file into text using a speech recognition engine.

[0801] Based on the converted text, minutes and summaries are automatically generated using a generative artificial intelligence model.

[0802] 3. Providing minutes and summaries:

[0803] The server stores the completed minutes and summary as files, which the user can download from their terminal or send to them by email.

[0804] Examples:

[0805] When a user uploads an audio recording file of an online meeting to their terminal, the server converts the audio into text, generates minutes and a summary summarizing the contents of the meeting, and provides them to the user.

[0806] Transportation and hotel arrangements

[0807] 1. Enter your travel dates and desired conditions:

[0808] The user inputs the business trip schedule, transportation means, and accommodation requirements into the terminal, and transmits the information from the terminal to the server.

[0809] 2. Travel and accommodation search and suggestions:

[0810] Based on the received information, the server performs an online search, lists the best transportation options and accommodations, and makes suggestions to the user.

[0811] 3. Proposal and Arrangement:

[0812] The server makes reservations for transportation and accommodation based on the options selected by the user on the terminal.

[0813] The server will send a confirmation email to the user confirming the reservation.

[0814] Examples:

[0815] When a user inputs the dates of a business trip and specifies the desired means of transportation and accommodation, the server proposes the optimal flight and hotel plan, makes a reservation for the plan selected by the user, and sends a confirmation email.

[0816] Sales and cost information management and strategy proposals

[0817] 1. Upload sales and cost data:

[0818] The user prepares data on sales and costs on the terminal and uploads it to the server from the terminal.

[0819] 2. Data analysis and strategy proposal:

[0820] The server analyzes the received data and extracts sales and cost trends.

[0821] The server uses a generative artificial intelligence model to automatically generate data-based strategic proposals and improvement measures.

[0822] 3. Proposal report provided:

[0823] The server compiles the generated proposals in the form of a report, which the user can download from their terminal or the server can send to the user by email.

[0824] Examples:

[0825] When a user uploads an Excel sheet of recent sales and costs to their device, the server analyzes the data and generates a proposal for the user, such as "reducing advertising costs by 20% will increase profits by 15%."

[0826] As described above, the system of the present invention efficiently handles and supports a wide range of miscellaneous tasks, allowing freelancers to focus their time and efforts on their core business.

[0827] The processing flow will be explained below.

[0828] Support for automatic creation of tax return documents

[0829] Step 1:

[0830] The user prepares a PDF file of the invoice on the terminal.

[0831] Step 2:

[0832] The user uploads the invoice file from the terminal to the server.

[0833] Step 3:

[0834] The server receives the uploaded invoice file.

[0835] Step 4:

[0836] The server uses optical character recognition (OCR) technology to extract text information from the invoice.

[0837] Step 5:

[0838] The server automatically determines necessary data such as transaction date, amount, and trading partner from the extracted information.

[0839] Step 6:

[0840] The server stores the identified data in an internal Excel template.

[0841] Step 7:

[0842] The server arranges the data according to the tax return format and automatically fills in the necessary sections.

[0843] Step 8:

[0844] The server generates the completed Excel file and tax return.

[0845] Step 9:

[0846] The user downloads the generated file from the device, or the server sends it to the user by email.

[0847] Creating meeting minutes and summaries

[0848] Step 1:

[0849] The user saves the audio file of the online conference on the device.

[0850] Step 2:

[0851] The user uploads an audio file from the device to the server.

[0852] Step 3:

[0853] The server converts the received audio file into text using a speech recognition engine.

[0854] Step 4:

[0855] The server inputs the converted text into a generative AI model.

[0856] Step 5:

[0857] The server uses generative AI models to create minutes and summaries.

[0858] Step 6:

[0859] The server saves the created minutes and summaries as files.

[0860] Step 7:

[0861] The user downloads the minutes and summary from their terminal, or the server sends them to the user by email.

[0862] Transportation and hotel arrangements

[0863] Step 1:

[0864] The user inputs the business trip schedule, transportation method, and accommodation requirements into the terminal.

[0865] Step 2:

[0866] The user sends input information from the terminal to the server.

[0867] Step 3:

[0868] The server performs an online search based on the received information.

[0869] Step 4:

[0870] The server will list the best transportation and accommodation options.

[0871] Step 5:

[0872] The server presents the listed candidates to the user.

[0873] Step 6:

[0874] The user selects from the candidates via the terminal and sends it to the server.

[0875] Step 7:

[0876] The server makes transportation and accommodation reservations based on the user's selections.

[0877] Step 8:

[0878] The server will send a confirmation email to the user confirming the reservation.

[0879] Sales and cost information management and strategy proposals

[0880] Step 1:

[0881] The user provides data on sales and costs on the terminal.

[0882] Step 2:

[0883] The user uploads data from the device to the server.

[0884] Step 3:

[0885] The server receives the uploaded data.

[0886] Step 4:

[0887] The server analyzes sales and cost data to extract trends and patterns.

[0888] Step 5:

[0889] The server uses a generative AI model based on the extracted data to automatically generate strategic proposals and improvement measures.

[0890] Step 6:

[0891] The server compiles the generated proposals in the form of a report.

[0892] Step 7:

[0893] The user downloads the proposal report from the terminal, or the server sends it to the user by email.

[0894] Example 1

[0895] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0896] Freelancers and sole proprietors are often overwhelmed with miscellaneous tasks, making it difficult to devote sufficient time to their core work. In particular, they face challenges in efficiently preparing tax return documents, taking meeting minutes, arranging business trips, and managing sales and cost data. There is a need for an environment that efficiently handles these miscellaneous tasks and allows freelancers to concentrate on their core work.

[0897] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0898] In this invention, the server includes means for analyzing invoices using an AI model and automatically transcribing transaction data extracted from the invoices into tax return documents, means for converting voice data into text and generating meeting minutes and summaries, means for inputting travel itineraries and accommodation requirements and proposing and arranging optimal transportation and accommodations, means for analyzing sales data and cost data and making strategic improvement proposals, means for providing an interface for users to upload files from their terminals, storage means for temporarily storing and analyzing files received by the server, and means for automatically generating proposal reports using a generative AI model. This allows freelancers to efficiently handle necessary miscellaneous tasks while focusing on their core business.

[0899] An "artificial intelligence model" is a collection of machine learning algorithms used to analyze data and perform a specific task.

[0900] An "invoice" is a document issued to bill for goods or services.

[0901] "Transaction data" refers to data that includes information related to a transaction, such as the transaction date, amount, and counterparty.

[0902] "Tax return documents" are documents to be submitted to tax authorities and contain information such as income and expenses.

[0903] "Audio data" means a recording of audio or an audio signal stored in digital form.

[0904] A "minutes" is a document that records the contents of a meeting, including the participants, what was said, and the main points of the meeting.

[0905] A "summary" is a summary that briefly summarizes the detailed content.

[0906] A "travel itinerary" is a schedule for business trips or travel.

[0907] "Accommodation conditions" refers to desired conditions and requirements regarding accommodation facilities.

[0908] "Transportation" refers to the means used for transportation, including cars, trains, airplanes, etc.

[0909] "Accommodation facilities" are facilities for staying overnight, including hotels, inns, and guesthouses.

[0910] "Sales data" refers to data that records revenues earned through the purchase and sale of goods and the provision of services.

[0911] "Cost data" is data that records the costs involved in manufacturing a product or providing a service.

[0912] "Improvement proposals" are specific suggestions or advice to solve current problems.

[0913] "File upload" refers to a user sending a file from their own terminal to a server.

[0914] "Storage" refers to a storage device or mechanism for saving data.

[0915] An "interface" refers to the means or screen that a user uses to operate a system.

[0916] A "prompt" refers to an instruction or question that is input into a generative AI model.

[0917] The system of the present invention provides a means for freelancers to efficiently handle the many miscellaneous tasks they must perform on a daily basis, thereby helping them to focus on their core business. Below, we will explain each function of the system of the present invention and its specific embodiment.

[0918] Support for automatic creation of tax return documents

[0919] The user prepares a scanner or PDF file of the invoice on their device and uploads the invoice file from their device to the server. The server receives the uploaded file and scans it using optical character recognition (OCR) technology such as Google Cloud Vision API to extract text information. The server determines necessary data from the extracted text information, such as the transaction date, amount, and client, and saves it in an internal Excel template using a library such as openpyxl. The server then arranges the data according to the tax return format and automatically fills it in. The completed tax return is provided via an HTTP response so that the user can download it from their device, or it is sent by email.

[0920] Examples:

[0921] When a user uploads a PDF of an invoice to their device, the server uses OCR technology to extract the necessary information from the PDF (e.g., 100,000 yen, October 3, 2023, business partner X), automatically reflects this information in an Excel spreadsheet and tax return, and provides it to the user.

[0922] Example prompt sentence:

[0923] "How can I extract transaction dates, amounts, and business partners from uploaded PDF invoices and reflect them in Excel templates and tax returns?"

[0924] Creating meeting minutes and summaries

[0925] Users save audio files of online meetings on their devices and then upload them from their devices to the server. The server converts the received audio files into text using the Amazon Transcribe API. Based on the converted text data, minutes and summaries are automatically generated using OpenAI's generative AI model. The completed minutes and summaries are saved in file format for users to download, provided as an HTTP response, or sent via email.

[0926] Examples:

[0927] When a user uploads an audio file of an online conference to their terminal, the server converts the audio into text, generates minutes and a summary summarizing the contents of the conference, and provides them to the user.

[0928] Example prompt sentence:

[0929] "How can I automatically generate minutes and summaries from audio files of online meetings?"

[0930] Transportation and hotel arrangements

[0931] The user enters their business trip dates, transportation method, and accommodation requirements into their device and sends them from the device to the server. Based on the received information, the server uses the Google Maps API or Expedia API to search for the optimal transportation method and accommodation. The server then lists the optimal plans and presents them to the user. The user selects the desired plan from their device and completes the reservation procedure. The server then reserves transportation and accommodation based on the selected plan and sends a confirmation email to the user.

[0932] Examples:

[0933] When a user inputs the dates of a business trip and specifies the desired means of transportation and accommodation conditions, the server proposes the optimal flight and hotel plan, makes a reservation for the plan selected by the user, and sends a confirmation email.

[0934] Example prompt sentence:

[0935] "How can I suggest and book the best travel and accommodation options based on the travel dates and mode of travel entered by the user?"

[0936] Sales and cost information management and strategy proposals

[0937] The user prepares sales and cost data as an Excel file and uploads it from their device to the server. The server reads the received Excel file using the Pandas library and analyzes the data. The server uses Matplotlib and Seaborn to visualize sales and cost trends, and uses a generative AI model (e.g., OpenAI's GPT-4) to generate strategic proposals and improvement measures based on the data. The proposal report is saved in PDF format and made available for the user to download from their device or sent via email.

[0938] Examples:

[0939] When a user uploads an Excel sheet containing recent sales and cost data to their device, the server analyzes the data, creates graphs that visualize sales and cost trends, and generates strategic suggestions based on the data (e.g., "reducing advertising costs by 20% will increase profits by 15%), which are then provided to the user.

[0940] Example prompt sentence:

[0941] "How can I analyze sales and cost data and automatically generate strategic proposals?"

[0942] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0943] Support for automatic creation of tax return documents

[0944] Step 1:

[0945] The user prepares a scanner or PDF file of the invoice on the device.

[0946] Specifically, users scan a paper invoice or save an existing digital invoice to their PC or smartphone.

[0947] Step 2:

[0948] The user uploads the invoice file from the terminal to the server.

[0949] Input: Invoice PDF or scanned file

[0950] Specifically, the user opens a file selection dialog of the browser, selects a file, and clicks the upload button.

[0951] Output: The invoice file is sent to the server.

[0952] Step 3:

[0953] The server receives the uploaded file.

[0954] Input: Uploaded invoice file

[0955] Specifically, the server receives the HTTP POST request and saves the file in temporary storage.

[0956] Output: Invoice file saved in storage

[0957] Step 4:

[0958] The server calls the Google Cloud Vision API or similar, scans the file using OCR technology, and extracts text information.

[0959] Input: Saved invoice file

[0960] Specifically, the server uses an OCR engine to extract text and obtain character information in JSON format.

[0961] Output: Extracted text information

[0962] Step 5:

[0963] The server automatically determines necessary data such as transaction date, amount, and trading partner from the extracted text information.

[0964] Input: Extracted text information

[0965] Specifically, the server uses a natural language processing library (e.g., spaCy) to analyze the required data and extract specific fields.

[0966] Output: Transaction date, amount, counterparty, etc.

[0967] Step 6:

[0968] The server saves the identified data in an internal Excel template.

[0969] Input: Transaction date, amount, counterparty, etc.

[0970] Specifically, the server uses a library such as openpyxl to embed data into the relevant cells of the Excel template.

[0971] Output: Excel file with completed tax return

[0972] Step 7:

[0973] The server provides the completed Excel file for the user to download from their device or sends it by email.

[0974] Input: Completed Excel file

[0975] Specifically, the server returns the file as an HTTP response, or sends the file to the user using an email sending API.

[0976] Output: Tax return provided to user

[0977] Creating meeting minutes and summaries

[0978] Step 1:

[0979] The user saves the audio file of the online conference on the device.

[0980] Specifically, the user uses the recording function of the conference app to save the audio file.

[0981] Step 2:

[0982] The user uploads an audio file from the device to the server.

[0983] Input: Online meeting audio file

[0984] Specifically, the user selects a file from a dedicated form and clicks the upload button.

[0985] Output: The audio file is sent to the server.

[0986] Step 3:

[0987] The server saves the received audio file.

[0988] Input: Uploaded audio file

[0989] Specifically, the server receives the HTTP POST request and temporarily stores it in storage.

[0990] Output: Audio file saved in storage

[0991] Step 4:

[0992] The server converts the speech to text using the Amazon Transcribe API.

[0993] Input: Saved audio file

[0994] Specifically, the server sends the audio file to the API and receives the converted text.

[0995] Output: Text data

[0996] Step 5:

[0997] The server uses OpenAI's generative AI model to analyze the text data and automatically generate minutes and summaries.

[0998] Input: Text data

[0999] Specifically, the server calls the generative AI model and generates minutes and summaries based on the text data.

[1000] Output: Generated minutes and summary

[1001] Step 6:

[1002] The server saves the generated minutes and summaries in a file format and provides them for users to download or sends them by email.

[1003] Input: Generated minutes and summaries

[1004] Specifically, the server returns the file as an HTTP response or sends it using an email sending API.

[1005] Output: Minutes and summary provided to user

[1006] Transportation and hotel arrangements

[1007] Step 1:

[1008] The user inputs the business trip schedule, transportation method, and accommodation conditions into the terminal and transmits them to the server.

[1009] Input: business trip schedule, transportation, accommodation conditions

[1010] Specifically, the user enters the required information into the input form and clicks the submit button.

[1011] Output: Information sent to the server

[1012] Step 2:

[1013] Based on the information received by the server, the server uses the Google Maps API and Expedia API to search for the best transportation and accommodation options.

[1014] Input: business trip schedule, transportation, accommodation conditions

[1015] Specifically, the server calls the API and retrieves the plan that meets the conditions.

[1016] Output: A list of the best transportation and accommodation options

[1017] Step 3:

[1018] The server organizes the search results and generates an HTML page to present to the user.

[1019] Input: A list of the best transportation and accommodation options

[1020] Specifically, the server embeds the data in an HTML template and provides it to the user as a response.

[1021] Output: The list of plans presented to the user

[1022] Step 4:

[1023] The user selects the desired plan and completes the reservation procedure.

[1024] Input: Select your desired plan

[1025] As a specific operation, the user selects a plan from the list and clicks the reservation button.

[1026] Output: Selected plan information

[1027] Step 5:

[1028] The server makes reservations for transportation and accommodation based on the selected plan.

[1029] Input: Selected plan information

[1030] Specifically, the server uses an API to send a request to the transportation or hotel reservation system and complete the reservation.

[1031] Output: Reservation confirmation information

[1032] Step 6:

[1033] The server will send a confirmation email to the user confirming the reservation.

[1034] Input: Reservation confirmation information

[1035] Specifically, the server sends the confirmation information to the email sending API and sends an email to the user.

[1036] Output: Confirmation email sent to the user

[1037] Sales and cost information management and strategy proposals

[1038] Step 1:

[1039] The user prepares sales and cost data as an Excel file and uploads it to the server from the terminal.

[1040] Input: Excel file containing sales data and cost data

[1041] Specifically, the user selects an Excel file from the file selection form and clicks the upload button.

[1042] Output: Excel file sent to the server

[1043] Step 2:

[1044] Read the Excel file received by the server.

[1045] Input: Uploaded Excel file

[1046] Specifically, the server uses the Pandas library to read the Excel file and import it as a data frame.

[1047] Output: Sales and cost data in data frame format

[1048] Step 3:

[1049] The server analyzes the data and visualizes sales and cost trends.

[1050] Input: Sales and cost data in data frame format

[1051] Specifically, the server generates graphs using Matplotlib and Seaborn.

[1052] Output: Visualized sales and cost data (graphs)

[1053] Step 4:

[1054] The server uses the generative AI model to generate data-based strategy proposals and improvement measures.

[1055] Input: Visualized sales and cost data

[1056] Specifically, the server inputs data into the generative AI model and receives automatically generated suggestions.

[1057] Output: Report with strategic proposals and improvement measures

[1058] Step 5:

[1059] The server saves the generated proposal report in PDF format and provides it for the user to download or sends it by email.

[1060] Input: Report with strategic proposals and improvement measures

[1061] Specifically, the server converts the report into PDF format and returns the file as an HTTP response or sends it using an API for sending emails.

[1062] Output: Proposal report provided to the user

[1063] (Application example 1)

[1064] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1065] Freelancers must spend a large amount of time on miscellaneous tasks in the course of their work. This includes a wide range of tasks, including processing transaction data such as invoices and receipts, tedious administrative tasks such as filing tax returns, creating meeting minutes, and even arranging transportation and accommodation. Not only do these tasks significantly reduce the efficiency of their work, but if not managed properly, they also increase the risk of financial loss and wasted time. To solve this problem, a system that automates and efficiently processes these miscellaneous tasks is needed.

[1066] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1067] In this invention, the server includes means for analyzing invoices using an artificial intelligence model and automatically transcribing transaction data extracted from the invoices into tax return documents, means for converting voice data into text and generating meeting minutes and summaries, means for inputting travel itineraries and accommodation requirements and proposing and arranging optimal transportation and accommodations, means for analyzing sales and cost data and making strategic improvement proposals, means for inputting and automatically analyzing payment information, extracting necessary transaction information using optical character recognition technology, organizing the data, and generating tax return data, and means for tracking expenses and sales and displaying real-time financial status on a dashboard. This allows freelancers to significantly reduce the time they spend on miscellaneous tasks and focus on their core business.

[1068] An "artificial intelligence model" is a system that uses machine learning algorithms to analyze data, recognize patterns, and make predictions and classifications.

[1069] An "invoice" is a document that lists the details and amount of a transaction and requests payment from the other party.

[1070] "Transaction data" is a collection of information about economic activities such as sales and expenditures.

[1071] "Tax return documents" are documents submitted by individuals and corporations to the tax office for tax returns.

[1072] "Audio data" refers to data in which audio is recorded in digital format.

[1073] "Minutes" are documents that record the contents of a meeting and the decisions made.

[1074] A "summary" is a document that briefly summarizes the main points of a meeting or other event.

[1075] A "travel itinerary" is a specific schedule for a business trip or trip.

[1076] "Accommodation conditions" refer to the conditions and requirements required of accommodation facilities.

[1077] "Transportation" refers to the means or methods of transportation used for travel.

[1078] "Accommodation" refers to a place where you stay, such as a hotel or lodging, during a trip or business trip.

[1079] "Sales data" is information relating to income obtained through a transaction.

[1080] "Cost data" refers to information relating to the costs incurred in carrying out a transaction or business.

[1081] "Strategic improvement proposals" are specific proposals for improving efficiency and profits based on the results obtained from data analysis.

[1082] "Payment information" refers to information regarding the payment method and amount used in a transaction.

[1083] "Optical character recognition technology" is a technology for extracting character information from images.

[1084] "Organizing data" refers to classifying information and organizing it systematically.

[1085] A "dashboard" is a screen or tool for visually displaying data and information.

[1086] This invention is a system that supports freelancers in efficiently handling daily chores, allowing them to concentrate on their primary work. The system of the present invention is composed of multiple means with various functions.

[1087] First, the server uses an artificial intelligence model to analyze the invoice uploaded by the user and extract transaction data from the invoice. Optical character recognition (OCR) technology is used to extract text information from PDF files or scanned images, automatically determining the necessary transaction information (e.g., amount, transaction date, and customer). Software used includes pytesseract (OCR) and pdf2image (PDF to image generation).

[1088] For example, when a user uploads a PDF of an invoice to their device, it is sent from the device to the server, which then uses OCR technology to scan the invoice and extract the necessary information, which is then stored in a database on the server and automatically transcribed into the format for tax return documents.

[1089] Next, it includes a function to generate meeting minutes and summaries using audio data. After the user saves the audio file of an online meeting on their device, they upload it from their device to the server. The server uses a speech recognition engine to convert the audio data into text, and then uses a generative AI model to automatically generate minutes and summaries. During this process, the Google Speech-to-Text API and generative AI models are used to create highly accurate minutes.

[1090] On the other hand, the server inputs the user's travel schedule and accommodation requirements, and suggests the optimal means of transportation and accommodation, and also provides the function of making arrangements if necessary. When the user inputs the business trip schedule and desired means of transportation and accommodation requirements on the terminal, the server performs an online search and lists the optimal plans. The reservation procedure is then carried out based on the options selected by the user.

[1091] It also includes a means for management, analysis, and improvement proposals for sales and cost data. When users upload sales and cost data from their devices to the server, the server analyzes the data and extracts sales and cost trends. It uses a generative artificial intelligence model to automatically generate strategic improvement proposals based on the data and creates reports to provide to users.

[1092] It also uses payment information input and automatic analysis functions, extracts necessary transaction information using optical character recognition technology, and automatically organizes the data, automatically generating data for tax returns. It also provides a dashboard that tracks expenses and sales in real time and displays financial status.

[1093] For example, when a user makes a payment through a terminal, the transaction information is automatically entered into the server, and optical character recognition technology is used to extract the necessary data from receipts and invoices, automatically organizing the data and enabling the generation of tax return data and real-time tracking of expenses and sales.

[1094] Example prompt sentence:

[1095] "Parse the uploaded invoice PDF file and extract the invoice date, amount, and account."

[1096] "Convert audio data from online meetings to text and generate minutes and summaries."

[1097] "Search and suggest the best transportation and accommodation options based on the travel dates and transportation preferences entered by the user."

[1098] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1099] Step 1:

[1100] The user prepares a scanned invoice or a PDF file on the terminal and uploads the invoice file from the terminal to the server. The input is the scanned invoice or PDF file, and the output is sending the file to the server. Specifically, the user takes a photo of the invoice using a smartphone or PC, or selects the scanned PDF file and uploads it to the server.

[1101] Step 2:

[1102] The server receives the uploaded file and uses optical character recognition (OCR) technology to scan the file and extract text information. The input is the uploaded invoice file, and the output is the extracted text information. Specifically, the server uses pytesseract to extract text data from PDFs and images, and extracts necessary data such as transaction date, amount, and client.

[1103] Step 3:

[1104] The server stores the extracted data in an internal database and automatically transcribes it into the format of tax return documents. The input is the extracted transaction data, and the output is the generation of tax return documents. Specifically, the server organizes the data using pandas, transcribes it into an Excel template, and generates the final tax return documents.

[1105] Step 4:

[1106] A user saves an audio file of an online conference on their device and uploads it to a server from their device. The input is the audio file of the online conference, and the output is the transfer of the audio file to the server. Specifically, the user uses the conference recording function to obtain the audio data and uploads it to the server.

[1107] Step 5:

[1108] The server converts the received audio file into text using a speech recognition engine, and automatically generates minutes and summaries using a generative AI model. The input is an audio file, and the output is textual minutes and summaries. Specifically, the server converts audio into text using the Google Speech-to-Text API, and generates minutes and summaries using a generative AI model.

[1109] Step 6:

[1110] The user inputs the dates of a business trip, specifies the desired means of transportation and accommodation conditions, and sends the data to the server. The input is the business trip dates, means of transportation, and accommodation conditions, and the output is sending the information to the server. Specifically, the user uses a terminal to enter the necessary information into an input form and sends it to the server.

[1111] Step 7:

[1112] Based on the information received, the server performs an online search, lists the optimal means of transportation and accommodation, and proposes it to the user. The input is the user's travel schedule and accommodation requirements, and the output is a list of proposals. Specifically, the server uses APIs and web scraping to collect information on means of transportation and accommodation, and then lists the optimal plans.

[1113] Step 8:

[1114] The server makes reservations for transportation and accommodation based on the options selected by the user. The input is the plan selected by the user, and the output is a confirmed reservation. Specifically, the server uses the reservation site or API to process the transportation and accommodation reservations.

[1115] Step 9:

[1116] The user uploads data related to sales and costs from the terminal to the server. The input is sales data and cost data, and the output is sending the data to the server. Specifically, the user prepares the data in a format such as an Excel spreadsheet and uploads it from the terminal to the server.

[1117] Step 10:

[1118] The server analyzes the data and automatically generates strategic improvement proposals using a generative AI model. The input is sales data and cost data, and the output is strategic improvement proposals. Specifically, the server analyzes the data using data analysis tools and generative AI models, and generates improvement proposals.

[1119] Step 11:

[1120] When a user makes a payment, they enter their payment information through their device, and the server receives and analyzes the payment data. The input is payment information, and the output is analyzed payment data. Specifically, when a user makes a purchase or uses a service, they enter their payment information into the app, and the server receives and analyzes that information.

[1121] Step 12:

[1122] The server uses optical character recognition technology to extract the necessary transaction information and automatically organize the data. The input is payment information or scanned receipts, and the output is organized transaction information. Specifically, the server uses pytesseract to extract the necessary transaction information from receipts or invoices and saves it in an internal database.

[1123] Step 13:

[1124] The server generates tax return data based on the organized transaction information and provides a dashboard for tracking expenses and sales in real time. The input is transaction information, and the output is tax return data and a dashboard. Specifically, the server uses pandas to organize the data and perform tracking and display.

[1125] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1126] The system of the present invention provides a means for freelancers to efficiently handle the many miscellaneous tasks they must perform on a daily basis, helping them to focus on their core business. In particular, by combining it with an emotion engine that recognizes the user's emotions, a function is added to provide services that are more suited to the user. Each function of the system of the present invention and its specific embodiments are described below.

[1127] Support for automatic creation of tax return documents

[1128] 1. Upload your invoice:

[1129] The user prepares a scanner or PDF file of the invoice on the terminal.

[1130] The user uploads the invoice file from the terminal to the server.

[1131] 2. Invoice analysis:

[1132] The server receives the uploaded invoice file and uses optical character recognition (OCR) technology to scan the invoice and extract text information.

[1133] The server automatically determines necessary data such as transaction date, amount, and trading partner from the extracted information.

[1134] 3. Data transcription and tax return generation:

[1135] The server stores the identified data in an internal Excel template.

[1136] The server arranges the data according to the tax return format and automatically fills in the necessary sections.

[1137] The server generates the completed Excel file and tax return, which the user can download from their device or the server can email to the user.

[1138] Examples:

[1139] When a user uploads a PDF of an invoice to their device, the server uses OCR technology to extract the necessary information from the PDF (e.g., amount 100,000 yen, transaction date October 3, 2023, business partner X), automatically reflects this information in an Excel spreadsheet and tax return, and provides it to the user.

[1140] Creating meeting minutes and summaries

[1141] 1. Uploading audio data:

[1142] A user saves an audio file of an online conference on a terminal and uploads the audio file from the terminal to a server.

[1143] 2. Audio data conversion and transcription:

[1144] The server converts the received audio file into text using a speech recognition engine.

[1145] Based on the converted text, minutes and summaries are automatically generated using a generative artificial intelligence model.

[1146] 3. Add sentiment analysis:

[1147] The server uses an emotion engine based on the voice data and text data to analyze the emotions of the conference participants.

[1148] Based on the analyzed emotional data, the content of the minutes and summaries is adjusted to generate documents that incorporate emotional nuances.

[1149] 4. Provision of minutes and summaries:

[1150] The server stores the completed minutes and summary as files, which the user can download from their terminal or send to them by email.

[1151] Examples:

[1152] When a user uploads an audio recording of an online meeting to their device, the server converts the audio into text and uses an emotion engine to analyze the emotions of the meeting participants based on the text and audio data. The minutes and summaries generated reflect the emotional nuances of the participants and are provided to the user.

[1153] Transportation and hotel arrangements

[1154] 1. Enter your travel dates and desired conditions:

[1155] The user inputs the business trip schedule, transportation means, and accommodation requirements into the terminal, and transmits the information from the terminal to the server.

[1156] 2. Travel and accommodation search and suggestions:

[1157] The server performs an online search based on the received information and lists the best transportation and accommodation options.

[1158] It uses an emotion engine to recognize the user's emotional state and generate optimal suggestions based on that state.

[1159] 3. Proposal and Arrangement:

[1160] The server makes reservations for transportation and accommodation based on the options selected by the user on the terminal.

[1161] The server will send a confirmation email to the user confirming the reservation.

[1162] Examples:

[1163] When a user inputs the dates of their business trip and specifies their preferred means of transportation and accommodation, the server uses an emotion engine to understand the user's emotional state. For example, if the user is feeling stressed, the server will suggest comfortable means of transportation and accommodations that will allow them to relax. The server then makes a reservation for the plan selected by the user and sends a confirmation email.

[1164] Sales and cost information management and strategy proposals

[1165] 1. Upload sales and cost data:

[1166] The user prepares data on sales and costs on the terminal and uploads it to the server from the terminal.

[1167] 2. Data analysis and strategy proposal:

[1168] The server analyzes the received data and extracts sales and cost trends.

[1169] The server uses a generative artificial intelligence model to automatically generate data-based strategic proposals and improvement measures.

[1170] 3. Adding emotional support:

[1171] The server uses an emotion engine to analyze the user's emotional state and adjust the content of the strategy proposal.

[1172] For example, if a user is feeling stressed, we suggest solutions that require less effort.

[1173] 4. Proposal report provided:

[1174] The server compiles the generated proposals in the form of a report, which the user can download from their terminal or the server can send to the user by email.

[1175] Examples:

[1176] When a user uploads sales and cost data to their device, the server analyzes it and uses a generative AI model to generate a suggestion such as "reducing advertising costs by 20% will increase profits by 15%." At the same time, an emotion engine analyzes the user's emotional state and suggests solutions that require less effort to implement if the user is feeling stressed. The generated suggestion report is then provided to the user.

[1177] As described above, the system of the present invention efficiently handles a wide range of miscellaneous tasks, allowing freelancers to focus their time and efforts on their core business. The addition of an emotion engine provides services that are more suited to users, improving their work efficiency and satisfaction.

[1178] The processing flow will be explained below.

[1179] Support for automatic creation of tax return documents

[1180] Step 1:

[1181] The user prepares a PDF file of the invoice on the terminal.

[1182] Step 2:

[1183] The user uploads the invoice file from the terminal to the server.

[1184] Step 3:

[1185] The server receives the uploaded invoice file.

[1186] Step 4:

[1187] The server uses optical character recognition (OCR) technology to extract text information from the invoice.

[1188] Step 5:

[1189] The server automatically determines necessary data such as transaction date, amount, and trading partner from the extracted information.

[1190] Step 6:

[1191] The server stores the identified data in an internal Excel template.

[1192] Step 7:

[1193] The server arranges the data according to the tax return format and automatically fills in the necessary sections.

[1194] Step 8:

[1195] The server generates the completed Excel file and tax return.

[1196] Step 9:

[1197] The server analyzes the user's emotions using an emotion engine and adds a message to the proposal that gives the user a sense of security.

[1198] Step 10:

[1199] The user downloads the generated file from the device, or the server sends it to the user by email.

[1200] Creating meeting minutes and summaries

[1201] Step 1:

[1202] The user saves the audio file of the online conference on the device.

[1203] Step 2:

[1204] The user uploads an audio file from the device to the server.

[1205] Step 3:

[1206] The server converts the received audio file into text using a speech recognition engine.

[1207] Step 4:

[1208] The server inputs the converted text into a generative AI model.

[1209] Step 5:

[1210] The server uses generative AI models to create minutes and summaries.

[1211] Step 6:

[1212] The server uses an emotion engine based on the voice data and text data to analyze the emotions of the conference participants.

[1213] Step 7:

[1214] Based on the results of the sentiment analysis, the server adjusts the content of the minutes and summary to generate documents that reflect emotional nuances.

[1215] Step 8:

[1216] The server saves the created minutes and summaries as files.

[1217] Step 9:

[1218] The user downloads the minutes and summary from their terminal, or the server sends them to the user by email.

[1219] Transportation and hotel arrangements

[1220] Step 1:

[1221] The user inputs the business trip schedule, transportation method, and accommodation requirements into the terminal.

[1222] Step 2:

[1223] The user sends input information from the terminal to the server.

[1224] Step 3:

[1225] The server performs an online search based on the received information.

[1226] Step 4:

[1227] The server will list the best transportation and accommodation options.

[1228] Step 5:

[1229] The server analyzes the listed candidates using an emotion engine to highlight the best candidates based on the user's emotional state.

[1230] Step 6:

[1231] The server presents the list to the user with comments that match the emotions.

[1232] Step 7:

[1233] The user selects from the candidates via the terminal and contacts the server.

[1234] Step 8:

[1235] The server makes transportation and accommodation reservations based on the user's selections.

[1236] Step 9:

[1237] The server will send a confirmation email to the user confirming the reservation.

[1238] Sales and cost information management and strategy proposals

[1239] Step 1:

[1240] The user provides data on sales and costs on the terminal.

[1241] Step 2:

[1242] The user uploads data from the device to the server.

[1243] Step 3:

[1244] The server receives the uploaded data.

[1245] Step 4:

[1246] The server analyzes sales and cost data to extract trends and patterns.

[1247] Step 5:

[1248] The server uses a generative AI model based on the extracted data to automatically generate strategic proposals and improvement measures.

[1249] Step 6:

[1250] The server uses an emotion engine to analyze the user's emotional state and adjust the content of the strategy proposal.

[1251] Step 7:

[1252] The server compiles the suggestions, including sentiment-enabled comments, in the form of a report.

[1253] Step 8:

[1254] The user downloads the proposal report from the terminal, or the server sends it to the user by email.

[1255] Example 2

[1256] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1257] Freelancers have to handle a wide variety of chores on a daily basis, which can prevent them from concentrating on their primary work. This often leads to emotional stress, which can reduce work efficiency and satisfaction. There is a need for a system that can solve the above problems, improve the work efficiency of freelancers, and provide appropriate support based on their emotions.

[1258] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for analyzing invoices using an artificial intelligence model and automatically transcribing transaction data extracted from the invoices into tax return documents; means for scanning invoices using optical character recognition technology and extracting text information; means for converting voice data to text and generating meeting minutes and summaries; means for analyzing the emotions of meeting participants based on the voice data and text data using an emotion engine and adjusting the content of the minutes and summaries; means for inputting travel itineraries and accommodation requirements and proposing and arranging optimal transportation and accommodations; means for recognizing the user's emotional state using the emotion engine and generating optimal proposals based on that state; means for analyzing sales data and cost data and making strategic improvement proposals; and means for analyzing the user's emotional state using the emotion engine and adjusting the content of strategic proposals. This frees freelancers from mundane tasks, allowing them to focus on their core business and receive optimal support tailored to their emotions.

[1259] An "artificial intelligence model" is an algorithm that uses machine learning and deep learning to analyze data and make predictions and classifications.

[1260] "Optical character recognition technology" is a technology that recognizes character information contained in images and PDFs and extracts it as text data.

[1261] A "voice recognition engine" is a technology that analyzes voice data and converts it into text data.

[1262] An "emotion engine" is a technology that analyzes emotions based on voice and text data and uses the results to provide appropriate feedback and adjustments.

[1263] "Tax return documents" are income and expenditure reports to be submitted to tax authorities, and are documents into which transaction data is transcribed.

[1264] "Minutes" are a written record of the contents of a meeting.

[1265] A "summary" is an overview that summarizes the main points of detailed data or documents.

[1266] "Transportation" refers to the vehicles and methods of travel used by people to get around.

[1267] "Accommodation" refers to a place where travelers and business people stay, and includes hotels, inns, etc.

[1268] "Sales data" is a record of income earned through business activities.

[1269] "Cost data" is a record of the costs incurred in business activities.

[1270] "Strategic improvement proposals" involve proposing specific measures to improve business efficiency and profitability based on data analysis.

[1271] The system of the present invention provides a means for freelancers to efficiently handle the many miscellaneous tasks they must perform on a daily basis, helping them to focus on their core business. In particular, by combining it with an emotion engine that recognizes the user's emotions, a function is added to provide services that are more suited to the user. Each function of the system of the present invention and its specific embodiments are described below.

[1272] Support for automatic creation of tax return documents

[1273] 1. The user prepares a scanned image or PDF file of the invoice on the terminal and uploads it to the server through the system's web interface.

[1274] 2. The server receives the uploaded invoice file and uses an OCR engine (e.g., Tesseract or Google Cloud Vision API) to scan the image of the file and extract text information.

[1275] 3. The server analyzes the extracted text information, automatically identifies important data such as transaction date, amount, and trading partner, and saves it in an internal Excel template.

[1276] 4. The server places the data in the appropriate location according to the tax return format and automatically fills in the necessary sections.

[1277] 5. The server generates the completed Excel file and tax return, which the user can download from their device or the server will email to the user.

[1278] As a specific example, when a user uploads a PDF of an invoice to a terminal, the server uses OCR technology to extract information such as "amount 100,000 yen," "transaction date October 3, 2023," and "client XX company" from the PDF, and automatically reflects this information in an Excel spreadsheet and tax return.

[1279] Creating meeting minutes and summaries

[1280] 1. The user saves the audio file of the online meeting on their device and uploads it to the server through the system's web interface.

[1281] 2. The server converts the received audio file into text using a speech recognition engine (e.g., Google Speech-to-Text API or IBM Watson).

[1282] 3. The server automatically generates minutes and summaries based on the converted text using a generative AI model (e.g., OpenAI's GPT or BERT).

[1283] 4. The server uses an emotion engine (e.g., Affectiva or IBM Tone Analyzer) to analyze the emotions of the meeting participants based on the voice and text data, and adjusts the content of the minutes and summary based on the analyzed emotion data.

[1284] 5. The server stores the completed minutes and summary, and the user can download them from their device, or the server will send them to the user by email.

[1285] For example, when a user uploads an audio recording of an online meeting, the server converts the audio into text and analyzes the emotions using an emotion engine, so the generated minutes and summaries reflect the emotional nuances of the participants.

[1286] Transportation and hotel arrangements

[1287] 1. The user enters the business trip schedule, transportation method, and accommodation requirements on the terminal and sends the information to the server through the system's web interface.

[1288] 2. The server performs an online search based on the received information and lists the best transportation and accommodation options. It uses an emotion engine to recognize the user's emotional state and generate optimal suggestions based on that state.

[1289] 3. Based on the user's selection, the server will process the reservation for transportation and accommodation, and automatically send the user a confirmation email.

[1290] As a concrete example, when a user inputs the dates of a business trip, the server uses an emotion engine to understand the user's emotional state, and if the user is feeling stressed, it will suggest and make reservations for comfortable transportation and accommodations where they can relax.

[1291] Sales and cost information management and strategy proposals

[1292] 1. The user prepares sales and cost data as a spreadsheet or CSV file and uploads it to the server through the system's web interface.

[1293] 2. The server analyzes the received data, extracts sales and cost trends, and automatically generates strategic improvement proposals based on the data using a generative AI model.

[1294] 3. The server uses an emotion engine to analyze the user's emotional state and adjust the proposed strategy accordingly. If the user is feeling stressed, it will suggest a solution that requires less effort.

[1295] 4. The server saves the completed proposal report and the user can download it from their device, or the server will send it to the user by email.

[1296] As a concrete example, when a user uploads sales and cost data, the server generates a specific suggestion such as "reducing advertising costs by 20% will increase profits by 15%," and provides a low-stress solution based on the results of analysis by the emotion engine.

[1297] As described above, the system of the present invention efficiently handles a wide range of miscellaneous tasks, allowing freelancers to focus their time and efforts on their core work, and by adding an emotion engine, it provides a service that is more suited to users, improving work efficiency and satisfaction.

[1298] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1299] Support for automatic creation of tax return documents

[1300] Processing Steps

[1301] Step 1: Prepare and upload your invoice

[1302] Users scan invoices and save them as image or PDF files on their devices.

[1303] The user opens a browser on their device, accesses the system's web interface, and logs in.

[1304] The user clicks the "Upload Invoice" button, selects the invoice file from the file selection dialog, and presses the "Upload" button.

[1305] Input: Scanned invoice image / PDF file

[1306] Output: Invoice file is sent to the server

[1307] Step 2: Parse the invoice

[1308] The server receives the uploaded invoice file.

[1309] The server calls an OCR engine (e.g., Tesseract or Google Cloud Vision API) to extract text data from the invoice image / PDF.

[1310] Input: Uploaded invoice file

[1311] Output: Extracted text data

[1312] Step 3: Automatically transcribe data

[1313] The server analyzes the extracted text data and automatically determines important information such as the transaction date, amount, and trading partner.

[1314] The server stores the determined data in an internal Excel template, for example, entering the transaction date in cell A1 and the amount in cell B1.

[1315] Input: Extracted text data

[1316] Output: Transcribed Excel data

[1317] Step 4: Generate your tax return documents

[1318] The server places the data in the appropriate location according to the tax return format and automatically fills in the necessary sections.

[1319] The server generates the completed Excel file and tax return.

[1320] Input: Transcribed Excel data

[1321] Output: Generated tax return documents

[1322] Step 5: Submit your tax return documents

[1323] The server temporarily stores the generated tax return documents and provides a link for the user to download or sends them by email.

[1324] Users can download it from their device or open the email attachment.

[1325] Input: Generated tax return documents

[1326] Output: User retrieves tax return documents

[1327] ---

[1328] Creating meeting minutes and summaries

[1329] Processing Steps

[1330] Step 1: Save and upload your audio data

[1331] Users save the audio files of online meetings to their devices.

[1332] The user accesses the system's web interface, logs in, clicks the "Upload Audio Data" button, selects an audio file, and presses the "Upload" button.

[1333] Input: Meeting audio file

[1334] Output: The audio file is uploaded to the server.

[1335] Step 2: Convert the audio data

[1336] The server converts the received audio file into text using a speech recognition engine (e.g., Google Speech-to-Text API or IBM Watson).

[1337] Input: Audio file

[1338] Output: Converted text data

[1339] Step 3: Generate minutes and summaries

[1340] The server automatically generates minutes and summaries based on the converted text using a generative AI model (e.g., OpenAI's GPT or BERT).

[1341] Input: Converted text data

[1342] Output: Generated minutes and summary

[1343] Step 4: Add sentiment analysis

[1344] The server uses an emotion engine (e.g., Affectiva or IBM Tone Analyzer) based on voice and text data to analyze the emotions of conference participants.

[1345] Based on the analyzed emotional data, the server adjusts the content of the minutes and summary, creating documents that incorporate emotion.

[1346] Input: Audio data, text data

[1347] Output: Minutes and summaries reflecting sentiment analysis data

[1348] Step 5: Provide minutes and summaries

[1349] The server temporarily stores the generated minutes and summaries and provides a link for users to download them or sends them by email.

[1350] Users can download it from their device or open the email attachment.

[1351] Input: Generated minutes and summary

[1352] Output: User gets minutes and summary

[1353] ---

[1354] Transportation and hotel arrangements

[1355] Processing Steps

[1356] Step 1: Enter travel dates and accommodation requirements

[1357] The user inputs the business trip schedule, transportation method, and accommodation requirements into the terminal.

[1358] The user logs into the system's web interface, enters information in the "Travel and Accommodation Input" form, and presses the "Submit" button.

[1359] Input: business trip schedule, transportation, accommodation conditions

[1360] Output: The entered business trip information is sent to the server.

[1361] Step 2: Find transportation and accommodation

[1362] Based on the information received, the server uses online search engines (e.g. Google Flights or Booking.com API) to list the best transportation and accommodation options.

[1363] The server uses an emotion engine to recognize the user's emotional state and generate optimal suggestions according to that state.

[1364] Input: Business trip information, emotional state

[1365] Output: A list of the best transportation options and accommodations

[1366] Step 3: View and select suggestions

[1367] The user can check the suggested options through the device's browser and select the desired transportation and accommodation.

[1368] Input: Suggested candidates

[1369] Output: User selected candidate

[1370] Step 4: Arrange your booking

[1371] The server processes the reservations for the selected transportation and accommodation.

[1372] The server will automatically send a confirmation email to the user confirming the reservation.

[1373] Input: User selected candidate

[1374] Output: Reservation confirmation email

[1375] ---

[1376] Sales and cost information management and strategy proposals

[1377] Processing Steps

[1378] Step 1: Upload your sales and cost data

[1379] Users prepare sales and cost data on their devices as spreadsheets or CSV files.

[1380] The user accesses the system's web interface, clicks the "Upload Cost of Sales Data" button, selects the file, and presses the "Upload" button.

[1381] Input: Sales and cost data spreadsheet / CSV file

[1382] Output: Sales and cost data is uploaded to the server

[1383] Step 2: Analyze the data

[1384] The server analyzes the received data and extracts sales and cost trends.

[1385] Input: Uploaded sales and cost data

[1386] Output: Analyzed sales and cost trends

[1387] Step 3: Generate strategic proposals

[1388] The server uses generative AI models to automatically generate data-based strategic proposals and improvement measures.

[1389] Input: Analyzed sales and cost trends

[1390] Output: Generated strategy proposals

[1391] Step 4: Adding emotional support

[1392] The server uses an emotion engine to analyze the user's emotional state and adjust the content of the strategy proposal.

[1393] For example, if a user is feeling stressed, we suggest solutions that require less effort.

[1394] Input: Strategy proposal, user's emotional state

[1395] Output: Strategy suggestions tailored to emotional state

[1396] Step 5: Providing a proposal report

[1397] The server will temporarily store the generated proposal report and provide a link for the user to download or send it via email.

[1398] Users can download it from their device or open the email attachment.

[1399] Input: Generated proposal report

[1400] Output: User gets proposal report

[1401] (Application example 2)

[1402] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1403] The purpose of this invention is to solve the inefficiencies in the various tasks and work processes that freelancers and factory workers face on a daily basis. In particular, in addition to processing invoices, taking meeting minutes, arranging travel and accommodation, and managing sales and costs, the invention aims to improve work efficiency and reduce worker stress by analyzing the emotional state of workers in real time and providing appropriate work instructions.

[1404] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1405] In this invention, the server includes means for analyzing invoices using an artificial intelligence model and automatically transcribing transaction data extracted from the invoices into tax return documents, means for converting voice data into text and generating meeting minutes and summaries, means for inputting travel itineraries and accommodation requirements and proposing and arranging optimal transportation and accommodations, and means for analyzing the emotional state of workers using an emotion recognition model and generating optimal work instructions. This makes it possible to centrally streamline multiple tasks and further optimize the work environment through emotion analysis.

[1406] An "artificial intelligence model" is a computer program or algorithm that automatically learns from data to perform specific tasks or analyses.

[1407] An "invoice" is a document that indicates payment obligations for a transaction and includes details such as the transaction amount and date.

[1408] "Transaction data" refers to information related to a transaction, including, for example, the transaction date, amount, and counterparty.

[1409] "Tax return documents" refers to a set of documents regarding income and taxable items that an individual or corporation submits to the tax office.

[1410] "Audio data" means data in digital or analog form that is a recording of the human voice.

[1411] "Meeting minutes" are documents that record the contents and decisions of a meeting.

[1412] A "summary" is a short document that summarizes detailed information.

[1413] A "travel itinerary" is a schedule that describes how to travel within a specific period of time.

[1414] "Accommodation conditions" is information indicating the conditions and wishes required when using accommodation facilities such as hotels.

[1415] "Transportation" is the means used to travel from one place to another, examples being trains, buses, and airplanes.

[1416] "Accommodation" refers to a facility that provides a place for people to stay temporarily, and examples include hotels and guesthouses.

[1417] "Sales data" refers to data that represents information about the revenue earned by a company or individual over a certain period of time.

[1418] "Cost data" refers to data that represents information about expenses and costs incurred within a particular period of time.

[1419] An "emotion recognition model" is an algorithm or program for determining a person's emotional state based on audio or image data.

[1420] "Work instructions" are specific instructions or commands given to perform a specific task.

[1421] A "server" is a computer system that provides data and services over a network.

[1422] The system of the present invention is designed to improve the work and work efficiency of freelancers and factory workers. The system includes the following means:

[1423] 1. Automated invoice processing methods:

[1424] The server receives invoice files uploaded by users via their terminals. It uses optical character recognition (OCR) technology to extract transaction data from the invoices and automatically transcribes it into tax return documents. This reduces the burden of invoice processing for users. The software used includes OpenCV and Tesseract for OCR technology.

[1425] 2. Meeting minutes and summary generators:

[1426] The server converts user-provided voice data into text using a speech recognition engine. It then uses a generative artificial intelligence model to automatically generate minutes and summaries from the text. It also uses an emotion engine to analyze the emotions of meeting participants and reflect them in the minutes and summaries. Software used includes TensorFlow and Transformers.

[1427] 3. Travel and accommodation arrangements:

[1428] When a user inputs their travel itinerary and accommodation requirements, the server searches for, proposes, and arranges suitable transportation and accommodation options. Based on emotion analysis, the server generates optimal proposals tailored to the user's emotional state. The technologies used include emotion recognition models and a database search engine.

[1429] 4. Revenue and cost control measures:

[1430] The server analyzes sales and cost data uploaded by users from their devices and uses a generative AI model to make strategic improvement proposals. It also generates proposals that take into account the user's emotional state based on emotion recognition. The software used includes data analysis tools and generative AI models.

[1431] 5. Emotion recognition and work instruction generation method:

[1432] The server analyzes the worker's emotional state in real time and generates optimal work instructions. To do this, it monitors the worker's face using smart glasses and a camera and determines their emotions using an emotion recognition model. Based on the results of the determination, it uses a generative AI model to provide optimal instructions. For example, if a worker is feeling stressed, the instruction to "take a short break" is automatically generated and displayed on the smart glasses' display.

[1433] Example prompt for a generative AI model:

[1434] "Generate short messages to workers who show signs of stress or fatigue, directing them to appropriate actions."

[1435] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1436] Step 1:

[1437] The user prepares the bill file on the terminal and uploads it to the server.

[1438] Input: PDF or image file of invoice

[1439] Output: Invoice file sent to the server

[1440] Specific operation: The user selects an invoice file from the terminal and uploads it to the server via a web form or a dedicated application.

[1441] Step 2:

[1442] The server receives the invoice file and scans it using optical character recognition (OCR) technology to extract text information.

[1443] Input: Submitted invoice file

[1444] Output: Extracted text information (transaction date, amount, trading partner, etc.)

[1445] What it does: The server uses OCR software (e.g., Tesseract) to extract text from the image, then filters out the necessary data (transaction date, amount, counterparty).

[1446] Step 3:

[1447] The server automatically transcribes the extracted transaction data into tax return documents.

[1448] Input: Extracted text information

[1449] Output: Excel or PDF file of tax return documents

[1450] Specific operation: The server embeds the extracted data into a template in the database and formats it as tax return documents.

[1451] Step 4:

[1452] The user saves the audio data on the device and uploads it to the server.

[1453] Input: Audio file

[1454] Output: Audio file sent to the server

[1455] Specific operation: After the meeting ends, the user uploads the recorded audio file to the server via a web form or a dedicated application.

[1456] Step 5:

[1457] The server converts the received audio file into text using a speech recognition engine.

[1458] Input: Submitted audio file

[1459] Output: Text data

[1460] Specific operation: The server uses a speech recognition engine (e.g., Google Speech-to-Text) to convert the voice data into text.

[1461] Step 6:

[1462] The server uses the converted text data to automatically generate minutes and summaries using a generative AI model.

[1463] Input: Text data

[1464] Output: Text data of minutes and summary

[1465] Specific operation: The server uses a generative AI model (e.g., GPT-3.5) to summarize the text data and create minutes and summaries.

[1466] Step 7:

[1467] The server uses an emotion recognition model based on the text data to analyze the emotions of the conference participants.

[1468] Input: Converted text and audio data

[1469] Output: Emotional state of meeting participants

[1470] What it does: The server uses an emotion recognition model (e.g., Emotion Recognition API) to analyze the emotional state from the text and audio data and adds that information to the minutes and summary.

[1471] Step 8:

[1472] The user inputs the travel schedule and accommodation requirements into the terminal and transmits them to the server.

[1473] Input: Travel schedule and accommodation conditions

[1474] Output: Travel and accommodation information sent to the server

[1475] Specific operation: The user uses the terminal to enter the business trip dates and accommodation requirements, and sends them to the server via a special form.

[1476] Step 9:

[1477] The server receives the travel schedule and accommodation information, and searches for and suggests suitable transportation and accommodation.

[1478] Input: Travel schedule and accommodation information

[1479] Output: A list of transportation and accommodation suggestions

[1480] Specific operation: The server uses online search engines and APIs to list and suggest optimal transportation and accommodation options.

[1481] Step 10:

[1482] The server analyzes the user's emotional state and adjusts the suggestions accordingly.

[1483] Input: User travel schedule and accommodation information, user emotion data

[1484] Output: A list of travel and accommodation plan suggestions based on emotions

[1485] Specific operation: The server uses an emotion recognition model to analyze the user's emotional state and generate optimal suggestions based on that state.

[1486] Step 11:

[1487] The user inputs sales data and cost data into the terminal and uploads it to the server.

[1488] Input: Sales and cost data files

[1489] Output: Sales and cost data sent to the server

[1490] Specific operation: The user uploads sales and cost data in file format from the terminal to the server.

[1491] Step 12:

[1492] The server analyzes the received sales and cost data and automatically generates strategic improvement proposals using a generative AI model.

[1493] Input: Sales data and cost data

[1494] Output: Report of strategic improvement recommendations

[1495] How it works: The server uses data analysis tools (e.g., Pandas and NumPy) to analyze trends in sales and cost data and uses generative AI models to create improvement suggestions.

[1496] Step 13:

[1497] The server uses an emotion recognition model to analyze the user's emotional state and adjust the suggestions.

[1498] Input: User sales data, cost data, and emotion data

[1499] Output: A report of improvement suggestions based on sentiment

[1500] Specific operation: The server analyzes the user's emotional state and suggests solutions that require less effort if the user is feeling stressed.

[1501] Step 14:

[1502] The server analyzes the worker's emotional state in real time and generates optimal work instructions.

[1503] Input: Facial image and voice data of the worker

[1504] Output: Real-time work instructions

[1505] Specific operation: The server analyzes the worker's facial image captured by the smart glasses' camera using an emotion recognition model (e.g., TensorFlow), and then uses a generative AI model to generate work instructions based on the worker's emotional state, which are then displayed on the smart glasses' display.

[1506] As a specific example of operation, if it is analyzed that a worker is feeling stressed, the instruction to "take a short break" will be displayed on the smart glasses' display.

[1507] Example prompt for the generative AI model: "Generate a short message to guide workers who show signs of stress or fatigue on appropriate actions."

[1508] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1509] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1510] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1511] [Third embodiment]

[1512] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1513] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1514] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1515] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1516] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1517] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1518] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1519] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1520] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1521] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1522] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1523] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1524] The system of the present invention provides a means for freelancers to efficiently handle the many miscellaneous tasks they must perform on a daily basis, thereby helping them to focus on their core business. Below, we will explain each function of the system of the present invention and its specific embodiment.

[1525] Support for automatic creation of tax return documents

[1526] 1. Upload your invoice:

[1527] The user prepares a scanner or PDF file of the invoice on the terminal.

[1528] The user uploads the invoice file from the terminal to the server.

[1529] 2. Invoice analysis:

[1530] The server receives the uploaded file and uses optical character recognition (OCR) technology to scan the invoice and extract the text information.

[1531] The server automatically determines necessary data such as transaction date, amount, and trading partner from the extracted information.

[1532] 3. Data transcription and tax return generation:

[1533] The server stores the identified data in an internal Excel template.

[1534] The server arranges the data according to the tax return format and automatically fills in the necessary sections.

[1535] The server generates the completed Excel file and tax return, which the user can download from their device or the server can email to the user.

[1536] Examples:

[1537] When a user uploads a PDF of an invoice to their device, the server uses OCR technology to extract the necessary information from the PDF (e.g., amount 100,000 yen, transaction date October 3, 2023, business partner X), automatically reflects this information in an Excel spreadsheet and tax return, and provides it to the user.

[1538] Creating meeting minutes and summaries

[1539] 1. Uploading audio data:

[1540] A user saves an audio file of an online conference on a terminal and uploads the audio file from the terminal to a server.

[1541] 2. Audio data conversion and transcription:

[1542] The server converts the received audio file into text using a speech recognition engine.

[1543] Based on the converted text, minutes and summaries are automatically generated using a generative artificial intelligence model.

[1544] 3. Providing minutes and summaries:

[1545] The server stores the completed minutes and summary as files, which the user can download from their terminal or send to them by email.

[1546] Examples:

[1547] When a user uploads an audio recording file of an online meeting to their terminal, the server converts the audio into text, generates minutes and a summary summarizing the contents of the meeting, and provides them to the user.

[1548] Transportation and hotel arrangements

[1549] 1. Enter your travel dates and desired conditions:

[1550] The user inputs the business trip schedule, transportation means, and accommodation requirements into the terminal, and transmits the information from the terminal to the server.

[1551] 2. Travel and accommodation search and suggestions:

[1552] Based on the received information, the server performs an online search, lists the best transportation options and accommodations, and makes suggestions to the user.

[1553] 3. Proposal and Arrangement:

[1554] The server makes reservations for transportation and accommodation based on the options selected by the user on the terminal.

[1555] The server will send a confirmation email to the user confirming the reservation.

[1556] Examples:

[1557] When a user inputs the dates of a business trip and specifies the desired means of transportation and accommodation, the server proposes the optimal flight and hotel plan, makes a reservation for the plan selected by the user, and sends a confirmation email.

[1558] Sales and cost information management and strategy proposals

[1559] 1. Upload sales and cost data:

[1560] The user prepares data on sales and costs on the terminal and uploads it to the server from the terminal.

[1561] 2. Data analysis and strategy proposal:

[1562] The server analyzes the received data and extracts sales and cost trends.

[1563] The server uses a generative artificial intelligence model to automatically generate data-based strategic proposals and improvement measures.

[1564] 3. Proposal report provided:

[1565] The server compiles the generated proposals in the form of a report, which the user can download from their terminal or the server can send to the user by email.

[1566] Examples:

[1567] When a user uploads an Excel sheet of recent sales and costs to their device, the server analyzes the data and generates a proposal for the user, such as "reducing advertising costs by 20% will increase profits by 15%."

[1568] As described above, the system of the present invention efficiently handles and supports a wide range of miscellaneous tasks, allowing freelancers to focus their time and efforts on their core business.

[1569] The processing flow will be explained below.

[1570] Support for automatic creation of tax return documents

[1571] Step 1:

[1572] The user prepares a PDF file of the invoice on the terminal.

[1573] Step 2:

[1574] The user uploads the invoice file from the terminal to the server.

[1575] Step 3:

[1576] The server receives the uploaded invoice file.

[1577] Step 4:

[1578] The server uses optical character recognition (OCR) technology to extract text information from the invoice.

[1579] Step 5:

[1580] The server automatically determines necessary data such as transaction date, amount, and trading partner from the extracted information.

[1581] Step 6:

[1582] The server stores the identified data in an internal Excel template.

[1583] Step 7:

[1584] The server arranges the data according to the tax return format and automatically fills in the necessary sections.

[1585] Step 8:

[1586] The server generates the completed Excel file and tax return.

[1587] Step 9:

[1588] The user downloads the generated file from the device, or the server sends it to the user by email.

[1589] Creating meeting minutes and summaries

[1590] Step 1:

[1591] The user saves the audio file of the online conference on the device.

[1592] Step 2:

[1593] The user uploads an audio file from the device to the server.

[1594] Step 3:

[1595] The server converts the received audio file into text using a speech recognition engine.

[1596] Step 4:

[1597] The server inputs the converted text into a generative AI model.

[1598] Step 5:

[1599] The server uses generative AI models to create minutes and summaries.

[1600] Step 6:

[1601] The server saves the created minutes and summaries as files.

[1602] Step 7:

[1603] The user downloads the minutes and summary from their terminal, or the server sends them to the user by email.

[1604] Transportation and hotel arrangements

[1605] Step 1:

[1606] The user inputs the business trip schedule, transportation method, and accommodation requirements into the terminal.

[1607] Step 2:

[1608] The user sends input information from the terminal to the server.

[1609] Step 3:

[1610] The server performs an online search based on the received information.

[1611] Step 4:

[1612] The server will list the best transportation and accommodation options.

[1613] Step 5:

[1614] The server presents the listed candidates to the user.

[1615] Step 6:

[1616] The user selects from the candidates via the terminal and sends it to the server.

[1617] Step 7:

[1618] The server makes transportation and accommodation reservations based on the user's selections.

[1619] Step 8:

[1620] The server will send a confirmation email to the user confirming the reservation.

[1621] Sales and cost information management and strategy proposals

[1622] Step 1:

[1623] The user provides data on sales and costs on the terminal.

[1624] Step 2:

[1625] The user uploads data from the device to the server.

[1626] Step 3:

[1627] The server receives the uploaded data.

[1628] Step 4:

[1629] The server analyzes sales and cost data to extract trends and patterns.

[1630] Step 5:

[1631] The server uses a generative AI model based on the extracted data to automatically generate strategic proposals and improvement measures.

[1632] Step 6:

[1633] The server compiles the generated proposals in the form of a report.

[1634] Step 7:

[1635] The user downloads the proposal report from the terminal, or the server sends it to the user by email.

[1636] Example 1

[1637] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1638] Freelancers and sole proprietors are often overwhelmed with miscellaneous tasks, making it difficult to devote sufficient time to their core work. In particular, they face challenges in efficiently preparing tax return documents, taking meeting minutes, arranging business trips, and managing sales and cost data. There is a need for an environment that efficiently handles these miscellaneous tasks and allows freelancers to concentrate on their core work.

[1639] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1640] In this invention, the server includes means for analyzing invoices using an AI model and automatically transcribing transaction data extracted from the invoices into tax return documents, means for converting voice data into text and generating meeting minutes and summaries, means for inputting travel itineraries and accommodation requirements and proposing and arranging optimal transportation and accommodations, means for analyzing sales data and cost data and making strategic improvement proposals, means for providing an interface for users to upload files from their terminals, storage means for temporarily storing and analyzing files received by the server, and means for automatically generating proposal reports using a generative AI model. This allows freelancers to efficiently handle necessary miscellaneous tasks while focusing on their core business.

[1641] An "artificial intelligence model" is a collection of machine learning algorithms used to analyze data and perform a specific task.

[1642] An "invoice" is a document issued to bill for goods or services.

[1643] "Transaction data" refers to data that includes information related to a transaction, such as the transaction date, amount, and counterparty.

[1644] "Tax return documents" are documents to be submitted to tax authorities and contain information such as income and expenses.

[1645] "Audio data" means a recording of audio or an audio signal stored in digital form.

[1646] A "minutes" is a document that records the contents of a meeting, including the participants, what was said, and the main points of the meeting.

[1647] A "summary" is a summary that briefly summarizes the detailed content.

[1648] A "travel itinerary" is a schedule for business trips or travel.

[1649] "Accommodation conditions" refers to desired conditions and requirements regarding accommodation facilities.

[1650] "Transportation" refers to the means used for transportation, including cars, trains, airplanes, etc.

[1651] "Accommodation facilities" are facilities for staying overnight, including hotels, inns, and guesthouses.

[1652] "Sales data" refers to data that records revenues earned through the purchase and sale of goods and the provision of services.

[1653] "Cost data" is data that records the costs involved in manufacturing a product or providing a service.

[1654] "Improvement proposals" are specific suggestions or advice to solve current problems.

[1655] "File upload" refers to a user sending a file from their own terminal to a server.

[1656] "Storage" refers to a storage device or mechanism for saving data.

[1657] An "interface" refers to the means or screen that a user uses to operate a system.

[1658] A "prompt" refers to an instruction or question that is input into a generative AI model.

[1659] The system of the present invention provides a means for freelancers to efficiently handle the many miscellaneous tasks they must perform on a daily basis, thereby helping them to focus on their core business. Below, we will explain each function of the system of the present invention and its specific embodiment.

[1660] Support for automatic creation of tax return documents

[1661] The user prepares a scanner or PDF file of the invoice on their device and uploads the invoice file from their device to the server. The server receives the uploaded file and scans it using optical character recognition (OCR) technology such as Google Cloud Vision API to extract text information. The server determines necessary data from the extracted text information, such as the transaction date, amount, and client, and saves it in an internal Excel template using a library such as openpyxl. The server then arranges the data according to the tax return format and automatically fills it in. The completed tax return is provided via an HTTP response so that the user can download it from their device, or it is sent by email.

[1662] Examples:

[1663] When a user uploads a PDF of an invoice to their device, the server uses OCR technology to extract the necessary information from the PDF (e.g., 100,000 yen, October 3, 2023, business partner X), automatically reflects this information in an Excel spreadsheet and tax return, and provides it to the user.

[1664] Example prompt sentence:

[1665] "How can I extract transaction dates, amounts, and business partners from uploaded PDF invoices and reflect them in Excel templates and tax returns?"

[1666] Creating meeting minutes and summaries

[1667] Users save audio files of online meetings on their devices and then upload them from their devices to the server. The server converts the received audio files into text using the Amazon Transcribe API. Based on the converted text data, minutes and summaries are automatically generated using OpenAI's generative AI model. The completed minutes and summaries are saved in file format for users to download, provided as an HTTP response, or sent via email.

[1668] Examples:

[1669] When a user uploads an audio file of an online conference to their terminal, the server converts the audio into text, generates minutes and a summary summarizing the contents of the conference, and provides them to the user.

[1670] Example prompt sentence:

[1671] "How can I automatically generate minutes and summaries from audio files of online meetings?"

[1672] Transportation and hotel arrangements

[1673] The user enters their business trip dates, transportation method, and accommodation requirements into their device and sends them from the device to the server. Based on the received information, the server uses the Google Maps API or Expedia API to search for the optimal transportation method and accommodation. The server then lists the optimal plans and presents them to the user. The user selects the desired plan from their device and completes the reservation procedure. The server then reserves transportation and accommodation based on the selected plan and sends a confirmation email to the user.

[1674] Examples:

[1675] When a user inputs the dates of a business trip and specifies the desired means of transportation and accommodation conditions, the server proposes the optimal flight and hotel plan, makes a reservation for the plan selected by the user, and sends a confirmation email.

[1676] Example prompt sentence:

[1677] "How can I suggest and book the best travel and accommodation options based on the travel dates and mode of travel entered by the user?"

[1678] Sales and cost information management and strategy proposals

[1679] The user prepares sales and cost data as an Excel file and uploads it from their device to the server. The server reads the received Excel file using the Pandas library and analyzes the data. The server uses Matplotlib and Seaborn to visualize sales and cost trends, and uses a generative AI model (e.g., OpenAI's GPT-4) to generate strategic proposals and improvement measures based on the data. The proposal report is saved in PDF format and made available for the user to download from their device or sent via email.

[1680] Examples:

[1681] When a user uploads an Excel sheet containing recent sales and cost data to their device, the server analyzes the data, creates graphs that visualize sales and cost trends, and generates strategic suggestions based on the data (e.g., "reducing advertising costs by 20% will increase profits by 15%), which are then provided to the user.

[1682] Example prompt sentence:

[1683] "How can I analyze sales and cost data and automatically generate strategic proposals?"

[1684] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1685] Support for automatic creation of tax return documents

[1686] Step 1:

[1687] The user prepares a scanner or PDF file of the invoice on the device.

[1688] Specifically, users scan a paper invoice or save an existing digital invoice to their PC or smartphone.

[1689] Step 2:

[1690] The user uploads the invoice file from the terminal to the server.

[1691] Input: Invoice PDF or scanned file

[1692] Specifically, the user opens a file selection dialog of the browser, selects a file, and clicks the upload button.

[1693] Output: The invoice file is sent to the server.

[1694] Step 3:

[1695] The server receives the uploaded file.

[1696] Input: Uploaded invoice file

[1697] Specifically, the server receives the HTTP POST request and saves the file in temporary storage.

[1698] Output: Invoice file saved in storage

[1699] Step 4:

[1700] The server calls the Google Cloud Vision API or similar, scans the file using OCR technology, and extracts text information.

[1701] Input: Saved invoice file

[1702] Specifically, the server uses an OCR engine to extract text and obtain character information in JSON format.

[1703] Output: Extracted text information

[1704] Step 5:

[1705] The server automatically determines necessary data such as transaction date, amount, and trading partner from the extracted text information.

[1706] Input: Extracted text information

[1707] Specifically, the server uses a natural language processing library (e.g., spaCy) to analyze the required data and extract specific fields.

[1708] Output: Transaction date, amount, counterparty, etc.

[1709] Step 6:

[1710] The server saves the identified data in an internal Excel template.

[1711] Input: Transaction date, amount, counterparty, etc.

[1712] Specifically, the server uses a library such as openpyxl to embed data into the relevant cells of the Excel template.

[1713] Output: Excel file with completed tax return

[1714] Step 7:

[1715] The server provides the completed Excel file for the user to download from their device or sends it by email.

[1716] Input: Completed Excel file

[1717] Specifically, the server returns the file as an HTTP response, or sends the file to the user using an email sending API.

[1718] Output: Tax return provided to user

[1719] Creating meeting minutes and summaries

[1720] Step 1:

[1721] The user saves the audio file of the online conference on the device.

[1722] Specifically, the user uses the recording function of the conference app to save the audio file.

[1723] Step 2:

[1724] The user uploads an audio file from the device to the server.

[1725] Input: Online meeting audio file

[1726] Specifically, the user selects a file from a dedicated form and clicks the upload button.

[1727] Output: The audio file is sent to the server.

[1728] Step 3:

[1729] The server saves the received audio file.

[1730] Input: Uploaded audio file

[1731] Specifically, the server receives the HTTP POST request and temporarily stores it in storage.

[1732] Output: Audio file saved in storage

[1733] Step 4:

[1734] The server converts the speech to text using the Amazon Transcribe API.

[1735] Input: Saved audio file

[1736] Specifically, the server sends the audio file to the API and receives the converted text.

[1737] Output: Text data

[1738] Step 5:

[1739] The server uses OpenAI's generative AI model to analyze the text data and automatically generate minutes and summaries.

[1740] Input: Text data

[1741] Specifically, the server calls the generative AI model and generates minutes and summaries based on the text data.

[1742] Output: Generated minutes and summary

[1743] Step 6:

[1744] The server saves the generated minutes and summaries in a file format and provides them for users to download or sends them by email.

[1745] Input: Generated minutes and summaries

[1746] Specifically, the server returns the file as an HTTP response or sends it using an email sending API.

[1747] Output: Minutes and summary provided to user

[1748] Transportation and hotel arrangements

[1749] Step 1:

[1750] The user inputs the business trip schedule, transportation method, and accommodation conditions into the terminal and transmits them to the server.

[1751] Input: business trip schedule, transportation, accommodation conditions

[1752] Specifically, the user enters the required information into the input form and clicks the submit button.

[1753] Output: Information sent to the server

[1754] Step 2:

[1755] Based on the information received by the server, the server uses the Google Maps API and Expedia API to search for the best transportation and accommodation options.

[1756] Input: business trip schedule, transportation, accommodation conditions

[1757] Specifically, the server calls the API and retrieves the plan that meets the conditions.

[1758] Output: A list of the best transportation and accommodation options

[1759] Step 3:

[1760] The server organizes the search results and generates an HTML page to present to the user.

[1761] Input: A list of the best transportation and accommodation options

[1762] Specifically, the server embeds the data in an HTML template and provides it to the user as a response.

[1763] Output: The list of plans presented to the user

[1764] Step 4:

[1765] The user selects the desired plan and completes the reservation procedure.

[1766] Input: Select your desired plan

[1767] As a specific operation, the user selects a plan from the list and clicks the reservation button.

[1768] Output: Selected plan information

[1769] Step 5:

[1770] The server makes reservations for transportation and accommodation based on the selected plan.

[1771] Input: Selected plan information

[1772] Specifically, the server uses an API to send a request to the transportation or hotel reservation system and complete the reservation.

[1773] Output: Reservation confirmation information

[1774] Step 6:

[1775] The server will send a confirmation email to the user confirming the reservation.

[1776] Input: Reservation confirmation information

[1777] Specifically, the server sends the confirmation information to the email sending API and sends an email to the user.

[1778] Output: Confirmation email sent to the user

[1779] Sales and cost information management and strategy proposals

[1780] Step 1:

[1781] The user prepares sales and cost data as an Excel file and uploads it to the server from the terminal.

[1782] Input: Excel file containing sales data and cost data

[1783] Specifically, the user selects an Excel file from the file selection form and clicks the upload button.

[1784] Output: Excel file sent to the server

[1785] Step 2:

[1786] Read the Excel file received by the server.

[1787] Input: Uploaded Excel file

[1788] Specifically, the server uses the Pandas library to read the Excel file and import it as a data frame.

[1789] Output: Sales and cost data in data frame format

[1790] Step 3:

[1791] The server analyzes the data and visualizes sales and cost trends.

[1792] Input: Sales and cost data in data frame format

[1793] Specifically, the server generates graphs using Matplotlib and Seaborn.

[1794] Output: Visualized sales and cost data (graphs)

[1795] Step 4:

[1796] The server uses the generative AI model to generate data-based strategy proposals and improvement measures.

[1797] Input: Visualized sales and cost data

[1798] Specifically, the server inputs data into the generative AI model and receives automatically generated suggestions.

[1799] Output: Report with strategic proposals and improvement measures

[1800] Step 5:

[1801] The server saves the generated proposal report in PDF format and provides it for the user to download or sends it by email.

[1802] Input: Report with strategic proposals and improvement measures

[1803] Specifically, the server converts the report into PDF format and returns the file as an HTTP response or sends it using an API for sending emails.

[1804] Output: Proposal report provided to the user

[1805] (Application example 1)

[1806] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1807] Freelancers must spend a large amount of time on miscellaneous tasks in the course of their work. This includes a wide range of tasks, including processing transaction data such as invoices and receipts, tedious administrative tasks such as filing tax returns, creating meeting minutes, and even arranging transportation and accommodation. Not only do these tasks significantly reduce the efficiency of their work, but if not managed properly, they also increase the risk of financial loss and wasted time. To solve this problem, a system that automates and efficiently processes these miscellaneous tasks is needed.

[1808] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1809] In this invention, the server includes means for analyzing invoices using an artificial intelligence model and automatically transcribing transaction data extracted from the invoices into tax return documents, means for converting voice data into text and generating meeting minutes and summaries, means for inputting travel itineraries and accommodation requirements and proposing and arranging optimal transportation and accommodations, means for analyzing sales and cost data and making strategic improvement proposals, means for inputting and automatically analyzing payment information, extracting necessary transaction information using optical character recognition technology, organizing the data, and generating tax return data, and means for tracking expenses and sales and displaying real-time financial status on a dashboard. This allows freelancers to significantly reduce the time they spend on miscellaneous tasks and focus on their core business.

[1810] An "artificial intelligence model" is a system that uses machine learning algorithms to analyze data, recognize patterns, and make predictions and classifications.

[1811] An "invoice" is a document that lists the details and amount of a transaction and requests payment from the other party.

[1812] "Transaction data" is a collection of information about economic activities such as sales and expenditures.

[1813] "Tax return documents" are documents submitted by individuals and corporations to the tax office for tax returns.

[1814] "Audio data" refers to data in which audio is recorded in digital format.

[1815] "Minutes" are documents that record the contents of a meeting and the decisions made.

[1816] A "summary" is a document that briefly summarizes the main points of a meeting or other event.

[1817] A "travel itinerary" is a specific schedule for a business trip or trip.

[1818] "Accommodation conditions" refer to the conditions and requirements required of accommodation facilities.

[1819] "Transportation" refers to the means or methods of transportation used for travel.

[1820] "Accommodation" refers to a place where you stay, such as a hotel or lodging, during a trip or business trip.

[1821] "Sales data" is information relating to income obtained through a transaction.

[1822] "Cost data" refers to information relating to the costs incurred in carrying out a transaction or business.

[1823] "Strategic improvement proposals" are specific proposals for improving efficiency and profits based on the results obtained from data analysis.

[1824] "Payment information" refers to information regarding the payment method and amount used in a transaction.

[1825] "Optical character recognition technology" is a technology for extracting character information from images.

[1826] "Organizing data" refers to classifying information and organizing it systematically.

[1827] A "dashboard" is a screen or tool for visually displaying data and information.

[1828] This invention is a system that supports freelancers in efficiently handling daily chores, allowing them to concentrate on their primary work. The system of the present invention is composed of multiple means with various functions.

[1829] First, the server uses an artificial intelligence model to analyze the invoice uploaded by the user and extract transaction data from the invoice. Optical character recognition (OCR) technology is used to extract text information from PDF files or scanned images, automatically determining the necessary transaction information (e.g., amount, transaction date, and customer). Software used includes pytesseract (OCR) and pdf2image (PDF to image generation).

[1830] For example, when a user uploads a PDF of an invoice to their device, it is sent from the device to the server, which then uses OCR technology to scan the invoice and extract the necessary information, which is then stored in a database on the server and automatically transcribed into the format for tax return documents.

[1831] Next, it includes a function to generate meeting minutes and summaries using audio data. After the user saves the audio file of an online meeting on their device, they upload it from their device to the server. The server uses a speech recognition engine to convert the audio data into text, and then uses a generative AI model to automatically generate minutes and summaries. During this process, the Google Speech-to-Text API and generative AI models are used to create highly accurate minutes.

[1832] On the other hand, the server inputs the user's travel schedule and accommodation requirements, and suggests the optimal means of transportation and accommodation, and also provides the function of making arrangements if necessary. When the user inputs the business trip schedule and desired means of transportation and accommodation requirements on the terminal, the server performs an online search and lists the optimal plans. The reservation procedure is then carried out based on the options selected by the user.

[1833] It also includes a means for management, analysis, and improvement proposals for sales and cost data. When users upload sales and cost data from their devices to the server, the server analyzes the data and extracts sales and cost trends. It uses a generative artificial intelligence model to automatically generate strategic improvement proposals based on the data and creates reports to provide to users.

[1834] It also uses payment information input and automatic analysis functions, extracts necessary transaction information using optical character recognition technology, and automatically organizes the data, automatically generating data for tax returns. It also provides a dashboard that tracks expenses and sales in real time and displays financial status.

[1835] For example, when a user makes a payment through a terminal, the transaction information is automatically entered into the server, and optical character recognition technology is used to extract the necessary data from receipts and invoices, automatically organizing the data and enabling the generation of tax return data and real-time tracking of expenses and sales.

[1836] Example prompt sentence:

[1837] "Parse the uploaded invoice PDF file and extract the invoice date, amount, and account."

[1838] "Convert audio data from online meetings to text and generate minutes and summaries."

[1839] "Search and suggest the best transportation and accommodation options based on the travel dates and transportation preferences entered by the user."

[1840] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1841] Step 1:

[1842] The user prepares a scanned invoice or a PDF file on the terminal and uploads the invoice file from the terminal to the server. The input is the scanned invoice or PDF file, and the output is sending the file to the server. Specifically, the user takes a photo of the invoice using a smartphone or PC, or selects the scanned PDF file and uploads it to the server.

[1843] Step 2:

[1844] The server receives the uploaded file and uses optical character recognition (OCR) technology to scan the file and extract text information. The input is the uploaded invoice file, and the output is the extracted text information. Specifically, the server uses pytesseract to extract text data from PDFs and images, and extracts necessary data such as transaction date, amount, and client.

[1845] Step 3:

[1846] The server stores the extracted data in an internal database and automatically transcribes it into the format of tax return documents. The input is the extracted transaction data, and the output is the generation of tax return documents. Specifically, the server organizes the data using pandas, transcribes it into an Excel template, and generates the final tax return documents.

[1847] Step 4:

[1848] A user saves an audio file of an online conference on their device and uploads it to a server from their device. The input is the audio file of the online conference, and the output is the transfer of the audio file to the server. Specifically, the user uses the conference recording function to obtain the audio data and uploads it to the server.

[1849] Step 5:

[1850] The server converts the received audio file into text using a speech recognition engine, and automatically generates minutes and summaries using a generative AI model. The input is an audio file, and the output is textual minutes and summaries. Specifically, the server converts audio into text using the Google Speech-to-Text API, and generates minutes and summaries using a generative AI model.

[1851] Step 6:

[1852] The user inputs the dates of a business trip, specifies the desired means of transportation and accommodation conditions, and sends the data to the server. The input is the business trip dates, means of transportation, and accommodation conditions, and the output is sending the information to the server. Specifically, the user uses a terminal to enter the necessary information into an input form and sends it to the server.

[1853] Step 7:

[1854] Based on the information received, the server performs an online search, lists the optimal means of transportation and accommodation, and proposes it to the user. The input is the user's travel schedule and accommodation requirements, and the output is a list of proposals. Specifically, the server uses APIs and web scraping to collect information on means of transportation and accommodation, and then lists the optimal plans.

[1855] Step 8:

[1856] The server makes reservations for transportation and accommodation based on the options selected by the user. The input is the plan selected by the user, and the output is a confirmed reservation. Specifically, the server uses the reservation site or API to process the transportation and accommodation reservations.

[1857] Step 9:

[1858] The user uploads data related to sales and costs from the terminal to the server. The input is sales data and cost data, and the output is sending the data to the server. Specifically, the user prepares the data in a format such as an Excel spreadsheet and uploads it from the terminal to the server.

[1859] Step 10:

[1860] The server analyzes the data and automatically generates strategic improvement proposals using a generative AI model. The input is sales data and cost data, and the output is strategic improvement proposals. Specifically, the server analyzes the data using data analysis tools and generative AI models, and generates improvement proposals.

[1861] Step 11:

[1862] When a user makes a payment, they enter their payment information through their device, and the server receives and analyzes the payment data. The input is payment information, and the output is analyzed payment data. Specifically, when a user makes a purchase or uses a service, they enter their payment information into the app, and the server receives and analyzes that information.

[1863] Step 12:

[1864] The server uses optical character recognition technology to extract the necessary transaction information and automatically organize the data. The input is payment information or scanned receipts, and the output is organized transaction information. Specifically, the server uses pytesseract to extract the necessary transaction information from receipts or invoices and saves it in an internal database.

[1865] Step 13:

[1866] The server generates tax return data based on the organized transaction information and provides a dashboard for tracking expenses and sales in real time. The input is transaction information, and the output is tax return data and a dashboard. Specifically, the server uses pandas to organize the data and perform tracking and display.

[1867] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1868] The system of the present invention provides a means for freelancers to efficiently handle the many miscellaneous tasks they must perform on a daily basis, helping them to focus on their core business. In particular, by combining it with an emotion engine that recognizes the user's emotions, a function is added to provide services that are more suited to the user. Each function of the system of the present invention and its specific embodiments are described below.

[1869] Support for automatic creation of tax return documents

[1870] 1. Upload your invoice:

[1871] The user prepares a scanner or PDF file of the invoice on the terminal.

[1872] The user uploads the invoice file from the terminal to the server.

[1873] 2. Invoice analysis:

[1874] The server receives the uploaded invoice file and uses optical character recognition (OCR) technology to scan the invoice and extract text information.

[1875] The server automatically determines necessary data such as transaction date, amount, and trading partner from the extracted information.

[1876] 3. Data transcription and tax return generation:

[1877] The server stores the identified data in an internal Excel template.

[1878] The server arranges the data according to the tax return format and automatically fills in the necessary sections.

[1879] The server generates the completed Excel file and tax return, which the user can download from their device or the server can email to the user.

[1880] Examples:

[1881] When a user uploads a PDF of an invoice to their device, the server uses OCR technology to extract the necessary information from the PDF (e.g., amount 100,000 yen, transaction date October 3, 2023, business partner X), automatically reflects this information in an Excel spreadsheet and tax return, and provides it to the user.

[1882] Creating meeting minutes and summaries

[1883] 1. Uploading audio data:

[1884] A user saves an audio file of an online conference on a terminal and uploads the audio file from the terminal to a server.

[1885] 2. Audio data conversion and transcription:

[1886] The server converts the received audio file into text using a speech recognition engine.

[1887] Based on the converted text, minutes and summaries are automatically generated using a generative artificial intelligence model.

[1888] 3. Add sentiment analysis:

[1889] The server uses an emotion engine based on the voice data and text data to analyze the emotions of the conference participants.

[1890] Based on the analyzed emotional data, the content of the minutes and summaries is adjusted to generate documents that incorporate emotional nuances.

[1891] 4. Provision of minutes and summaries:

[1892] The server stores the completed minutes and summary as files, which the user can download from their terminal or send to them by email.

[1893] Examples:

[1894] When a user uploads an audio recording of an online meeting to their device, the server converts the audio into text and uses an emotion engine to analyze the emotions of the meeting participants based on the text and audio data. The minutes and summaries generated reflect the emotional nuances of the participants and are provided to the user.

[1895] Transportation and hotel arrangements

[1896] 1. Enter your travel dates and desired conditions:

[1897] The user inputs the business trip schedule, transportation means, and accommodation requirements into the terminal, and transmits the information from the terminal to the server.

[1898] 2. Travel and accommodation search and suggestions:

[1899] The server performs an online search based on the received information and lists the best transportation and accommodation options.

[1900] It uses an emotion engine to recognize the user's emotional state and generate optimal suggestions based on that state.

[1901] 3. Proposal and Arrangement:

[1902] The server makes reservations for transportation and accommodation based on the options selected by the user on the terminal.

[1903] The server will send a confirmation email to the user confirming the reservation.

[1904] Examples:

[1905] When a user inputs the dates of their business trip and specifies their preferred means of transportation and accommodation, the server uses an emotion engine to understand the user's emotional state. For example, if the user is feeling stressed, the server will suggest comfortable means of transportation and accommodations that will allow them to relax. The server then makes a reservation for the plan selected by the user and sends a confirmation email.

[1906] Sales and cost information management and strategy proposals

[1907] 1. Upload sales and cost data:

[1908] The user prepares data on sales and costs on the terminal and uploads it to the server from the terminal.

[1909] 2. Data analysis and strategy proposal:

[1910] The server analyzes the received data and extracts sales and cost trends.

[1911] The server uses a generative artificial intelligence model to automatically generate data-based strategic proposals and improvement measures.

[1912] 3. Adding emotional support:

[1913] The server uses an emotion engine to analyze the user's emotional state and adjust the content of the strategy proposal.

[1914] For example, if a user is feeling stressed, we suggest solutions that require less effort.

[1915] 4. Proposal report provided:

[1916] The server compiles the generated proposals in the form of a report, which the user can download from their terminal or the server can send to the user by email.

[1917] Examples:

[1918] When a user uploads sales and cost data to their device, the server analyzes it and uses a generative AI model to generate a suggestion such as "reducing advertising costs by 20% will increase profits by 15%." At the same time, an emotion engine analyzes the user's emotional state and suggests solutions that require less effort to implement if the user is feeling stressed. The generated suggestion report is then provided to the user.

[1919] As described above, the system of the present invention efficiently handles a wide range of miscellaneous tasks, allowing freelancers to focus their time and efforts on their core business. The addition of an emotion engine provides services that are more suited to users, improving their work efficiency and satisfaction.

[1920] The processing flow will be explained below.

[1921] Support for automatic creation of tax return documents

[1922] Step 1:

[1923] The user prepares a PDF file of the invoice on the terminal.

[1924] Step 2:

[1925] The user uploads the invoice file from the terminal to the server.

[1926] Step 3:

[1927] The server receives the uploaded invoice file.

[1928] Step 4:

[1929] The server uses optical character recognition (OCR) technology to extract text information from the invoice.

[1930] Step 5:

[1931] The server automatically determines necessary data such as transaction date, amount, and trading partner from the extracted information.

[1932] Step 6:

[1933] The server stores the identified data in an internal Excel template.

[1934] Step 7:

[1935] The server arranges the data according to the tax return format and automatically fills in the necessary sections.

[1936] Step 8:

[1937] The server generates the completed Excel file and tax return.

[1938] Step 9:

[1939] The server analyzes the user's emotions using an emotion engine and adds a message to the proposal that gives the user a sense of security.

[1940] Step 10:

[1941] The user downloads the generated file from the device, or the server sends it to the user by email.

[1942] Creating meeting minutes and summaries

[1943] Step 1:

[1944] The user saves the audio file of the online conference on the device.

[1945] Step 2:

[1946] The user uploads an audio file from the device to the server.

[1947] Step 3:

[1948] The server converts the received audio file into text using a speech recognition engine.

[1949] Step 4:

[1950] The server inputs the converted text into a generative AI model.

[1951] Step 5:

[1952] The server uses generative AI models to create minutes and summaries.

[1953] Step 6:

[1954] The server uses an emotion engine based on the voice data and text data to analyze the emotions of the conference participants.

[1955] Step 7:

[1956] Based on the results of the sentiment analysis, the server adjusts the content of the minutes and summary to generate documents that reflect emotional nuances.

[1957] Step 8:

[1958] The server saves the created minutes and summaries as files.

[1959] Step 9:

[1960] The user downloads the minutes and summary from their terminal, or the server sends them to the user by email.

[1961] Transportation and hotel arrangements

[1962] Step 1:

[1963] The user inputs the business trip schedule, transportation method, and accommodation requirements into the terminal.

[1964] Step 2:

[1965] The user sends input information from the terminal to the server.

[1966] Step 3:

[1967] The server performs an online search based on the received information.

[1968] Step 4:

[1969] The server will list the best transportation and accommodation options.

[1970] Step 5:

[1971] The server analyzes the listed candidates using an emotion engine to highlight the best candidates based on the user's emotional state.

[1972] Step 6:

[1973] The server presents the list to the user with comments that match the emotions.

[1974] Step 7:

[1975] The user selects from the candidates via the terminal and contacts the server.

[1976] Step 8:

[1977] The server makes transportation and accommodation reservations based on the user's selections.

[1978] Step 9:

[1979] The server will send a confirmation email to the user confirming the reservation.

[1980] Sales and cost information management and strategy proposals

[1981] Step 1:

[1982] The user provides data on sales and costs on the terminal.

[1983] Step 2:

[1984] The user uploads data from the device to the server.

[1985] Step 3:

[1986] The server receives the uploaded data.

[1987] Step 4:

[1988] The server analyzes sales and cost data to extract trends and patterns.

[1989] Step 5:

[1990] The server uses a generative AI model based on the extracted data to automatically generate strategic proposals and improvement measures.

[1991] Step 6:

[1992] The server uses an emotion engine to analyze the user's emotional state and adjust the content of the strategy proposal.

[1993] Step 7:

[1994] The server compiles the suggestions, including sentiment-enabled comments, in the form of a report.

[1995] Step 8:

[1996] The user downloads the proposal report from the terminal, or the server sends it to the user by email.

[1997] Example 2

[1998] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1999] Freelancers have to handle a wide variety of chores on a daily basis, which can prevent them from concentrating on their primary work. This often leads to emotional stress, which can reduce work efficiency and satisfaction. There is a need for a system that can solve the above problems, improve the work efficiency of freelancers, and provide appropriate support based on their emotions.

[2000] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for analyzing invoices using an artificial intelligence model and automatically transcribing transaction data extracted from the invoices into tax return documents; means for scanning invoices using optical character recognition technology and extracting text information; means for converting voice data to text and generating meeting minutes and summaries; means for analyzing the emotions of meeting participants based on the voice data and text data using an emotion engine and adjusting the content of the minutes and summaries; means for inputting travel itineraries and accommodation requirements and proposing and arranging optimal transportation and accommodations; means for recognizing the user's emotional state using the emotion engine and generating optimal proposals based on that state; means for analyzing sales data and cost data and making strategic improvement proposals; and means for analyzing the user's emotional state using the emotion engine and adjusting the content of strategic proposals. This frees freelancers from mundane tasks, allowing them to focus on their core business and receive optimal support tailored to their emotions.

[2001] An "artificial intelligence model" is an algorithm that uses machine learning and deep learning to analyze data and make predictions and classifications.

[2002] "Optical character recognition technology" is a technology that recognizes character information contained in images and PDFs and extracts it as text data.

[2003] A "voice recognition engine" is a technology that analyzes voice data and converts it into text data.

[2004] An "emotion engine" is a technology that analyzes emotions based on voice and text data and uses the results to provide appropriate feedback and adjustments.

[2005] "Tax return documents" are income and expenditure reports to be submitted to tax authorities, and are documents into which transaction data is transcribed.

[2006] "Minutes" are a written record of the contents of a meeting.

[2007] A "summary" is an overview that summarizes the main points of detailed data or documents.

[2008] "Transportation" refers to the vehicles and methods of travel used by people to get around.

[2009] "Accommodation" refers to a place where travelers and business people stay, and includes hotels, inns, etc.

[2010] "Sales data" is a record of income earned through business activities.

[2011] "Cost data" is a record of the costs incurred in business activities.

[2012] "Strategic improvement proposals" involve proposing specific measures to improve business efficiency and profitability based on data analysis.

[2013] The system of the present invention provides a means for freelancers to efficiently handle the many miscellaneous tasks they must perform on a daily basis, helping them to focus on their core business. In particular, by combining it with an emotion engine that recognizes the user's emotions, a function is added to provide services that are more suited to the user. Each function of the system of the present invention and its specific embodiments are described below.

[2014] Support for automatic creation of tax return documents

[2015] 1. The user prepares a scanned image or PDF file of the invoice on the terminal and uploads it to the server through the system's web interface.

[2016] 2. The server receives the uploaded invoice file and uses an OCR engine (e.g., Tesseract or Google Cloud Vision API) to scan the image of the file and extract text information.

[2017] 3. The server analyzes the extracted text information, automatically identifies important data such as transaction date, amount, and trading partner, and saves it in an internal Excel template.

[2018] 4. The server places the data in the appropriate location according to the tax return format and automatically fills in the necessary sections.

[2019] 5. The server generates the completed Excel file and tax return, which the user can download from their device or the server will email to the user.

[2020] As a specific example, when a user uploads a PDF of an invoice to a terminal, the server uses OCR technology to extract information such as "amount 100,000 yen," "transaction date October 3, 2023," and "client XX company" from the PDF, and automatically reflects this information in an Excel spreadsheet and tax return.

[2021] Creating meeting minutes and summaries

[2022] 1. The user saves the audio file of the online meeting on their device and uploads it to the server through the system's web interface.

[2023] 2. The server converts the received audio file into text using a speech recognition engine (e.g., Google Speech-to-Text API or IBM Watson).

[2024] 3. The server automatically generates minutes and summaries based on the converted text using a generative AI model (e.g., OpenAI's GPT or BERT).

[2025] 4. The server uses an emotion engine (e.g., Affectiva or IBM Tone Analyzer) to analyze the emotions of the meeting participants based on the voice and text data, and adjusts the content of the minutes and summary based on the analyzed emotion data.

[2026] 5. The server stores the completed minutes and summary, and the user can download them from their device, or the server will send them to the user by email.

[2027] For example, when a user uploads an audio recording of an online meeting, the server converts the audio into text and analyzes the emotions using an emotion engine, so the generated minutes and summaries reflect the emotional nuances of the participants.

[2028] Transportation and hotel arrangements

[2029] 1. The user enters the business trip schedule, transportation method, and accommodation requirements on the terminal and sends the information to the server through the system's web interface.

[2030] 2. The server performs an online search based on the received information and lists the best transportation and accommodation options. It uses an emotion engine to recognize the user's emotional state and generate optimal suggestions based on that state.

[2031] 3. Based on the user's selection, the server will process the reservation for transportation and accommodation, and automatically send the user a confirmation email.

[2032] As a concrete example, when a user inputs the dates of a business trip, the server uses an emotion engine to understand the user's emotional state, and if the user is feeling stressed, it will suggest and make reservations for comfortable transportation and accommodations where they can relax.

[2033] Sales and cost information management and strategy proposals

[2034] 1. The user prepares sales and cost data as a spreadsheet or CSV file and uploads it to the server through the system's web interface.

[2035] 2. The server analyzes the received data, extracts sales and cost trends, and automatically generates strategic improvement proposals based on the data using a generative AI model.

[2036] 3. The server uses an emotion engine to analyze the user's emotional state and adjust the proposed strategy accordingly. If the user is feeling stressed, it will suggest a solution that requires less effort.

[2037] 4. The server saves the completed proposal report and the user can download it from their device, or the server will send it to the user by email.

[2038] As a concrete example, when a user uploads sales and cost data, the server generates a specific suggestion such as "reducing advertising costs by 20% will increase profits by 15%," and provides a low-stress solution based on the results of analysis by the emotion engine.

[2039] As described above, the system of the present invention efficiently handles a wide range of miscellaneous tasks, allowing freelancers to focus their time and efforts on their core work, and by adding an emotion engine, it provides a service that is more suited to users, improving work efficiency and satisfaction.

[2040] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2041] Support for automatic creation of tax return documents

[2042] Processing Steps

[2043] Step 1: Prepare and upload your invoice

[2044] Users scan invoices and save them as image or PDF files on their devices.

[2045] The user opens a browser on their device, accesses the system's web interface, and logs in.

[2046] The user clicks the "Upload Invoice" button, selects the invoice file from the file selection dialog, and presses the "Upload" button.

[2047] Input: Scanned invoice image / PDF file

[2048] Output: Invoice file is sent to the server

[2049] Step 2: Parse the invoice

[2050] The server receives the uploaded invoice file.

[2051] The server calls an OCR engine (e.g., Tesseract or Google Cloud Vision API) to extract text data from the invoice image / PDF.

[2052] Input: Uploaded invoice file

[2053] Output: Extracted text data

[2054] Step 3: Automatically transcribe data

[2055] The server analyzes the extracted text data and automatically determines important information such as the transaction date, amount, and trading partner.

[2056] The server stores the determined data in an internal Excel template, for example, entering the transaction date in cell A1 and the amount in cell B1.

[2057] Input: Extracted text data

[2058] Output: Transcribed Excel data

[2059] Step 4: Generate your tax return documents

[2060] The server places the data in the appropriate location according to the tax return format and automatically fills in the necessary sections.

[2061] The server generates the completed Excel file and tax return.

[2062] Input: Transcribed Excel data

[2063] Output: Generated tax return documents

[2064] Step 5: Submit your tax return documents

[2065] The server temporarily stores the generated tax return documents and provides a link for the user to download or sends them by email.

[2066] Users can download it from their device or open the email attachment.

[2067] Input: Generated tax return documents

[2068] Output: User retrieves tax return documents

[2069] ---

[2070] Creating meeting minutes and summaries

[2071] Processing Steps

[2072] Step 1: Save and upload your audio data

[2073] Users save the audio files of online meetings to their devices.

[2074] The user accesses the system's web interface, logs in, clicks the "Upload Audio Data" button, selects an audio file, and presses the "Upload" button.

[2075] Input: Meeting audio file

[2076] Output: The audio file is uploaded to the server.

[2077] Step 2: Convert the audio data

[2078] The server converts the received audio file into text using a speech recognition engine (e.g., Google Speech-to-Text API or IBM Watson).

[2079] Input: Audio file

[2080] Output: Converted text data

[2081] Step 3: Generate minutes and summaries

[2082] The server automatically generates minutes and summaries based on the converted text using a generative AI model (e.g., OpenAI's GPT or BERT).

[2083] Input: Converted text data

[2084] Output: Generated minutes and summary

[2085] Step 4: Add sentiment analysis

[2086] The server uses an emotion engine (e.g., Affectiva or IBM Tone Analyzer) based on voice and text data to analyze the emotions of conference participants.

[2087] Based on the analyzed emotional data, the server adjusts the content of the minutes and summary, creating documents that incorporate emotion.

[2088] Input: Audio data, text data

[2089] Output: Minutes and summaries reflecting sentiment analysis data

[2090] Step 5: Provide minutes and summaries

[2091] The server temporarily stores the generated minutes and summaries and provides a link for users to download them or sends them by email.

[2092] Users can download it from their device or open the email attachment.

[2093] Input: Generated minutes and summary

[2094] Output: User gets minutes and summary

[2095] ---

[2096] Transportation and hotel arrangements

[2097] Processing Steps

[2098] Step 1: Enter travel dates and accommodation requirements

[2099] The user inputs the business trip schedule, transportation method, and accommodation requirements into the terminal.

[2100] The user logs into the system's web interface, enters information in the "Travel and Accommodation Input" form, and presses the "Submit" button.

[2101] Input: business trip schedule, transportation, accommodation conditions

[2102] Output: The entered business trip information is sent to the server.

[2103] Step 2: Find transportation and accommodation

[2104] Based on the information received, the server uses online search engines (e.g. Google Flights or Booking.com API) to list the best transportation and accommodation options.

[2105] The server uses an emotion engine to recognize the user's emotional state and generate optimal suggestions according to that state.

[2106] Input: Business trip information, emotional state

[2107] Output: A list of the best transportation options and accommodations

[2108] Step 3: View and select suggestions

[2109] The user can check the suggested options through the device's browser and select the desired transportation and accommodation.

[2110] Input: Suggested candidates

[2111] Output: User selected candidate

[2112] Step 4: Arrange your booking

[2113] The server processes the reservations for the selected transportation and accommodation.

[2114] The server will automatically send a confirmation email to the user confirming the reservation.

[2115] Input: User selected candidate

[2116] Output: Reservation confirmation email

[2117] ---

[2118] Sales and cost information management and strategy proposals

[2119] Processing Steps

[2120] Step 1: Upload your sales and cost data

[2121] Users prepare sales and cost data on their devices as spreadsheets or CSV files.

[2122] The user accesses the system's web interface, clicks the "Upload Cost of Sales Data" button, selects the file, and presses the "Upload" button.

[2123] Input: Sales and cost data spreadsheet / CSV file

[2124] Output: Sales and cost data is uploaded to the server

[2125] Step 2: Analyze the data

[2126] The server analyzes the received data and extracts sales and cost trends.

[2127] Input: Uploaded sales and cost data

[2128] Output: Analyzed sales and cost trends

[2129] Step 3: Generate strategic proposals

[2130] The server uses generative AI models to automatically generate data-based strategic proposals and improvement measures.

[2131] Input: Analyzed sales and cost trends

[2132] Output: Generated strategy proposals

[2133] Step 4: Adding emotional support

[2134] The server uses an emotion engine to analyze the user's emotional state and adjust the content of the strategy proposal.

[2135] For example, if a user is feeling stressed, we suggest solutions that require less effort.

[2136] Input: Strategy proposal, user's emotional state

[2137] Output: Strategy suggestions tailored to emotional state

[2138] Step 5: Providing a proposal report

[2139] The server will temporarily store the generated proposal report and provide a link for the user to download or send it via email.

[2140] Users can download it from their device or open the email attachment.

[2141] Input: Generated proposal report

[2142] Output: User gets proposal report

[2143] (Application example 2)

[2144] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2145] The purpose of this invention is to solve the inefficiencies in the various tasks and work processes that freelancers and factory workers face on a daily basis. In particular, in addition to processing invoices, taking meeting minutes, arranging travel and accommodation, and managing sales and costs, the invention aims to improve work efficiency and reduce worker stress by analyzing the emotional state of workers in real time and providing appropriate work instructions.

[2146] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2147] In this invention, the server includes means for analyzing invoices using an artificial intelligence model and automatically transcribing transaction data extracted from the invoices into tax return documents, means for converting voice data into text and generating meeting minutes and summaries, means for inputting travel itineraries and accommodation requirements and proposing and arranging optimal transportation and accommodations, and means for analyzing the emotional state of workers using an emotion recognition model and generating optimal work instructions. This makes it possible to centrally streamline multiple tasks and further optimize the work environment through emotion analysis.

[2148] An "artificial intelligence model" is a computer program or algorithm that automatically learns from data to perform specific tasks or analyses.

[2149] An "invoice" is a document that indicates payment obligations for a transaction and includes details such as the transaction amount and date.

[2150] "Transaction data" refers to information related to a transaction, including, for example, the transaction date, amount, and counterparty.

[2151] "Tax return documents" refers to a set of documents regarding income and taxable items that an individual or corporation submits to the tax office.

[2152] "Audio data" means data in digital or analog form that is a recording of the human voice.

[2153] "Meeting minutes" are documents that record the contents and decisions of a meeting.

[2154] A "summary" is a short document that summarizes detailed information.

[2155] A "travel itinerary" is a schedule that describes how to travel within a specific period of time.

[2156] "Accommodation conditions" is information indicating the conditions and wishes required when using accommodation facilities such as hotels.

[2157] "Transportation" is the means used to travel from one place to another, examples being trains, buses, and airplanes.

[2158] "Accommodation" refers to a facility that provides a place for people to stay temporarily, and examples include hotels and guesthouses.

[2159] "Sales data" refers to data that represents information about the revenue earned by a company or individual over a certain period of time.

[2160] "Cost data" refers to data that represents information about expenses and costs incurred within a particular period of time.

[2161] An "emotion recognition model" is an algorithm or program for determining a person's emotional state based on audio or image data.

[2162] "Work instructions" are specific instructions or commands given to perform a specific task.

[2163] A "server" is a computer system that provides data and services over a network.

[2164] The system of the present invention is designed to improve the work and work efficiency of freelancers and factory workers. The system includes the following means:

[2165] 1. Automated invoice processing methods:

[2166] The server receives invoice files uploaded by users via their terminals. It uses optical character recognition (OCR) technology to extract transaction data from the invoices and automatically transcribes it into tax return documents. This reduces the burden of invoice processing for users. The software used includes OpenCV and Tesseract for OCR technology.

[2167] 2. Meeting minutes and summary generators:

[2168] The server converts user-provided voice data into text using a speech recognition engine. It then uses a generative artificial intelligence model to automatically generate minutes and summaries from the text. It also uses an emotion engine to analyze the emotions of meeting participants and reflect them in the minutes and summaries. Software used includes TensorFlow and Transformers.

[2169] 3. Travel and accommodation arrangements:

[2170] When a user inputs their travel itinerary and accommodation requirements, the server searches for, proposes, and arranges suitable transportation and accommodation options. Based on emotion analysis, the server generates optimal proposals tailored to the user's emotional state. The technologies used include emotion recognition models and a database search engine.

[2171] 4. Revenue and cost control measures:

[2172] The server analyzes sales and cost data uploaded by users from their devices and uses a generative AI model to make strategic improvement proposals. It also generates proposals that take into account the user's emotional state based on emotion recognition. The software used includes data analysis tools and generative AI models.

[2173] 5. Emotion recognition and work instruction generation method:

[2174] The server analyzes the worker's emotional state in real time and generates optimal work instructions. To do this, it monitors the worker's face using smart glasses and a camera and determines their emotions using an emotion recognition model. Based on the results of the determination, it uses a generative AI model to provide optimal instructions. For example, if a worker is feeling stressed, the instruction to "take a short break" is automatically generated and displayed on the smart glasses' display.

[2175] Example prompt for a generative AI model:

[2176] "Generate short messages to workers who show signs of stress or fatigue, directing them to appropriate actions."

[2177] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2178] Step 1:

[2179] The user prepares the bill file on the terminal and uploads it to the server.

[2180] Input: PDF or image file of invoice

[2181] Output: Invoice file sent to the server

[2182] Specific operation: The user selects an invoice file from the terminal and uploads it to the server via a web form or a dedicated application.

[2183] Step 2:

[2184] The server receives the invoice file and scans it using optical character recognition (OCR) technology to extract text information.

[2185] Input: Submitted invoice file

[2186] Output: Extracted text information (transaction date, amount, trading partner, etc.)

[2187] What it does: The server uses OCR software (e.g., Tesseract) to extract text from the image, then filters out the necessary data (transaction date, amount, counterparty).

[2188] Step 3:

[2189] The server automatically transcribes the extracted transaction data into tax return documents.

[2190] Input: Extracted text information

[2191] Output: Excel or PDF file of tax return documents

[2192] Specific operation: The server embeds the extracted data into a template in the database and formats it as tax return documents.

[2193] Step 4:

[2194] The user saves the audio data on the device and uploads it to the server.

[2195] Input: Audio file

[2196] Output: Audio file sent to the server

[2197] Specific operation: After the meeting ends, the user uploads the recorded audio file to the server via a web form or a dedicated application.

[2198] Step 5:

[2199] The server converts the received audio file into text using a speech recognition engine.

[2200] Input: Submitted audio file

[2201] Output: Text data

[2202] Specific operation: The server uses a speech recognition engine (e.g., Google Speech-to-Text) to convert the voice data into text.

[2203] Step 6:

[2204] The server uses the converted text data to automatically generate minutes and summaries using a generative AI model.

[2205] Input: Text data

[2206] Output: Text data of minutes and summary

[2207] Specific operation: The server uses a generative AI model (e.g., GPT-3.5) to summarize the text data and create minutes and summaries.

[2208] Step 7:

[2209] The server uses an emotion recognition model based on the text data to analyze the emotions of the conference participants.

[2210] Input: Converted text and audio data

[2211] Output: Emotional state of meeting participants

[2212] What it does: The server uses an emotion recognition model (e.g., Emotion Recognition API) to analyze the emotional state from the text and audio data and adds that information to the minutes and summary.

[2213] Step 8:

[2214] The user inputs the travel schedule and accommodation requirements into the terminal and transmits them to the server.

[2215] Input: Travel schedule and accommodation conditions

[2216] Output: Travel and accommodation information sent to the server

[2217] Specific operation: The user uses the terminal to enter the business trip dates and accommodation requirements, and sends them to the server via a special form.

[2218] Step 9:

[2219] The server receives the travel schedule and accommodation information, and searches for and suggests suitable transportation and accommodation.

[2220] Input: Travel schedule and accommodation information

[2221] Output: A list of transportation and accommodation suggestions

[2222] Specific operation: The server uses online search engines and APIs to list and suggest optimal transportation and accommodation options.

[2223] Step 10:

[2224] The server analyzes the user's emotional state and adjusts the suggestions accordingly.

[2225] Input: User travel schedule and accommodation information, user emotion data

[2226] Output: A list of travel and accommodation plan suggestions based on emotions

[2227] Specific operation: The server uses an emotion recognition model to analyze the user's emotional state and generate optimal suggestions based on that state.

[2228] Step 11:

[2229] The user inputs sales data and cost data into the terminal and uploads it to the server.

[2230] Input: Sales and cost data files

[2231] Output: Sales and cost data sent to the server

[2232] Specific operation: The user uploads sales and cost data in file format from the terminal to the server.

[2233] Step 12:

[2234] The server analyzes the received sales and cost data and automatically generates strategic improvement proposals using a generative AI model.

[2235] Input: Sales data and cost data

[2236] Output: Report of strategic improvement recommendations

[2237] How it works: The server uses data analysis tools (e.g., Pandas and NumPy) to analyze trends in sales and cost data and uses generative AI models to create improvement suggestions.

[2238] Step 13:

[2239] The server uses an emotion recognition model to analyze the user's emotional state and adjust the suggestions.

[2240] Input: User sales data, cost data, and emotion data

[2241] Output: A report of improvement suggestions based on sentiment

[2242] Specific operation: The server analyzes the user's emotional state and suggests solutions that require less effort if the user is feeling stressed.

[2243] Step 14:

[2244] The server analyzes the worker's emotional state in real time and generates optimal work instructions.

[2245] Input: Facial image and voice data of the worker

[2246] Output: Real-time work instructions

[2247] Specific operation: The server analyzes the worker's facial image captured by the smart glasses' camera using an emotion recognition model (e.g., TensorFlow), and then uses a generative AI model to generate work instructions based on the worker's emotional state, which are then displayed on the smart glasses' display.

[2248] As a specific example of operation, if it is analyzed that a worker is feeling stressed, the instruction to "take a short break" will be displayed on the smart glasses' display.

[2249] Example prompt for the generative AI model: "Generate a short message to guide workers who show signs of stress or fatigue on appropriate actions."

[2250] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[2251] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2252] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[2253] [Fourth embodiment]

[2254] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[2255] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[2256] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[2257] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[2258] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[2259] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[2260] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[2261] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[2262] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[2263] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[2264] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[2265] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[2266] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2267] The system of the present invention provides a means for freelancers to efficiently handle the many miscellaneous tasks they must perform on a daily basis, thereby helping them to focus on their core business. Below, we will explain each function of the system of the present invention and its specific embodiment.

[2268] Support for automatic creation of tax return documents

[2269] 1. Upload your invoice:

[2270] The user prepares a scanner or PDF file of the invoice on the terminal.

[2271] The user uploads the invoice file from the terminal to the server.

[2272] 2. Invoice analysis:

[2273] The server receives the uploaded file and uses optical character recognition (OCR) technology to scan the invoice and extract the text information.

[2274] The server automatically determines necessary data such as transaction date, amount, and trading partner from the extracted information.

[2275] 3. Data transcription and tax return generation:

[2276] The server stores the identified data in an internal Excel template.

[2277] The server arranges the data according to the tax return format and automatically fills in the necessary sections.

[2278] The server generates the completed Excel file and tax return, which the user can download from their device or the server can email to the user.

[2279] Examples:

[2280] When a user uploads a PDF of an invoice to their device, the server uses OCR technology to extract the necessary information from the PDF (e.g., amount 100,000 yen, transaction date October 3, 2023, business partner X), automatically reflects this information in an Excel spreadsheet and tax return, and provides it to the user.

[2281] Creating meeting minutes and summaries

[2282] 1. Uploading audio data:

[2283] A user saves an audio file of an online conference on a terminal and uploads the audio file from the terminal to a server.

[2284] 2. Audio data conversion and transcription:

[2285] The server converts the received audio file into text using a speech recognition engine.

[2286] Based on the converted text, minutes and summaries are automatically generated using a generative artificial intelligence model.

[2287] 3. Providing minutes and summaries:

[2288] The server stores the completed minutes and summary as files, which the user can download from their terminal or send to them by email.

[2289] Examples:

[2290] When a user uploads an audio recording file of an online meeting to their terminal, the server converts the audio into text, generates minutes and a summary summarizing the contents of the meeting, and provides them to the user.

[2291] Transportation and hotel arrangements

[2292] 1. Enter your travel dates and desired conditions:

[2293] The user inputs the business trip schedule, transportation means, and accommodation requirements into the terminal, and transmits the information from the terminal to the server.

[2294] 2. Travel and accommodation search and suggestions:

[2295] Based on the received information, the server performs an online search, lists the best transportation options and accommodations, and makes suggestions to the user.

[2296] 3. Proposal and Arrangement:

[2297] The server makes reservations for transportation and accommodation based on the options selected by the user on the terminal.

[2298] The server will send a confirmation email to the user confirming the reservation.

[2299] Examples:

[2300] When a user inputs the dates of a business trip and specifies the desired means of transportation and accommodation, the server proposes the optimal flight and hotel plan, makes a reservation for the plan selected by the user, and sends a confirmation email.

[2301] Sales and cost information management and strategy proposals

[2302] 1. Upload sales and cost data:

[2303] The user prepares data on sales and costs on the terminal and uploads it to the server from the terminal.

[2304] 2. Data analysis and strategy proposal:

[2305] The server analyzes the received data and extracts sales and cost trends.

[2306] The server uses a generative artificial intelligence model to automatically generate data-based strategic proposals and improvement measures.

[2307] 3. Proposal report provided:

[2308] The server compiles the generated proposals in the form of a report, which the user can download from their terminal or the server can send to the user by email.

[2309] Examples:

[2310] When a user uploads an Excel sheet of recent sales and costs to their device, the server analyzes the data and generates a proposal for the user, such as "reducing advertising costs by 20% will increase profits by 15%."

[2311] As described above, the system of the present invention efficiently handles and supports a wide range of miscellaneous tasks, allowing freelancers to focus their time and efforts on their core business.

[2312] The processing flow will be explained below.

[2313] Support for automatic creation of tax return documents

[2314] Step 1:

[2315] The user prepares a PDF file of the invoice on the terminal.

[2316] Step 2:

[2317] The user uploads the invoice file from the terminal to the server.

[2318] Step 3:

[2319] The server receives the uploaded invoice file.

[2320] Step 4:

[2321] The server uses optical character recognition (OCR) technology to extract text information from the invoice.

[2322] Step 5:

[2323] The server automatically determines necessary data such as transaction date, amount, and trading partner from the extracted information.

[2324] Step 6:

[2325] The server stores the identified data in an internal Excel template.

[2326] Step 7:

[2327] The server arranges the data according to the tax return format and automatically fills in the necessary sections.

[2328] Step 8:

[2329] The server generates the completed Excel file and tax return.

[2330] Step 9:

[2331] The user downloads the generated file from the device, or the server sends it to the user by email.

[2332] Creating meeting minutes and summaries

[2333] Step 1:

[2334] The user saves the audio file of the online conference on the device.

[2335] Step 2:

[2336] The user uploads an audio file from the device to the server.

[2337] Step 3:

[2338] The server converts the received audio file into text using a speech recognition engine.

[2339] Step 4:

[2340] The server inputs the converted text into a generative AI model.

[2341] Step 5:

[2342] The server uses generative AI models to create minutes and summaries.

[2343] Step 6:

[2344] The server saves the created minutes and summaries as files.

[2345] Step 7:

[2346] The user downloads the minutes and summary from their terminal, or the server sends them to the user by email.

[2347] Transportation and hotel arrangements

[2348] Step 1:

[2349] The user inputs the business trip schedule, transportation method, and accommodation requirements into the terminal.

[2350] Step 2:

[2351] The user sends input information from the terminal to the server.

[2352] Step 3:

[2353] The server performs an online search based on the received information.

[2354] Step 4:

[2355] The server will list the best transportation and accommodation options.

[2356] Step 5:

[2357] The server presents the listed candidates to the user.

[2358] Step 6:

[2359] The user selects from the candidates via the terminal and sends it to the server.

[2360] Step 7:

[2361] The server makes transportation and accommodation reservations based on the user's selections.

[2362] Step 8:

[2363] The server will send a confirmation email to the user confirming the reservation.

[2364] Sales and cost information management and strategy proposals

[2365] Step 1:

[2366] The user provides data on sales and costs on the terminal.

[2367] Step 2:

[2368] The user uploads data from the device to the server.

[2369] Step 3:

[2370] The server receives the uploaded data.

[2371] Step 4:

[2372] The server analyzes sales and cost data to extract trends and patterns.

[2373] Step 5:

[2374] The server uses a generative AI model based on the extracted data to automatically generate strategic proposals and improvement measures.

[2375] Step 6:

[2376] The server compiles the generated proposals in the form of a report.

[2377] Step 7:

[2378] The user downloads the proposal report from the terminal, or the server sends it to the user by email.

[2379] Example 1

[2380] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2381] Freelancers and sole proprietors are often overwhelmed with miscellaneous tasks, making it difficult to devote sufficient time to their core work. In particular, they face challenges in efficiently preparing tax return documents, taking meeting minutes, arranging business trips, and managing sales and cost data. There is a need for an environment that efficiently handles these miscellaneous tasks and allows freelancers to concentrate on their core work.

[2382] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[2383] In this invention, the server includes means for analyzing invoices using an AI model and automatically transcribing transaction data extracted from the invoices into tax return documents, means for converting voice data into text and generating meeting minutes and summaries, means for inputting travel itineraries and accommodation requirements and proposing and arranging optimal transportation and accommodations, means for analyzing sales data and cost data and making strategic improvement proposals, means for providing an interface for users to upload files from their terminals, storage means for temporarily storing and analyzing files received by the server, and means for automatically generating proposal reports using a generative AI model. This allows freelancers to efficiently handle necessary miscellaneous tasks while focusing on their core business.

[2384] An "artificial intelligence model" is a collection of machine learning algorithms used to analyze data and perform a specific task.

[2385] An "invoice" is a document issued to bill for goods or services.

[2386] "Transaction data" refers to data that includes information related to a transaction, such as the transaction date, amount, and counterparty.

[2387] "Tax return documents" are documents to be submitted to tax authorities and contain information such as income and expenses.

[2388] "Audio data" means a recording of audio or an audio signal stored in digital form.

[2389] A "minutes" is a document that records the contents of a meeting, including the participants, what was said, and the main points of the meeting.

[2390] A "summary" is a summary that briefly summarizes the detailed content.

[2391] A "travel itinerary" is a schedule for business trips or travel.

[2392] "Accommodation conditions" refers to desired conditions and requirements regarding accommodation facilities.

[2393] "Transportation" refers to the means used for transportation, including cars, trains, airplanes, etc.

[2394] "Accommodation facilities" are facilities for staying overnight, including hotels, inns, and guesthouses.

[2395] "Sales data" refers to data that records revenues earned through the purchase and sale of goods and the provision of services.

[2396] "Cost data" is data that records the costs involved in manufacturing a product or providing a service.

[2397] "Improvement proposals" are specific suggestions or advice to solve current problems.

[2398] "File upload" refers to a user sending a file from their own terminal to a server.

[2399] "Storage" refers to a storage device or mechanism for saving data.

[2400] An "interface" refers to the means or screen that a user uses to operate a system.

[2401] A "prompt" refers to an instruction or question that is input into a generative AI model.

[2402] The system of the present invention provides a means for freelancers to efficiently handle the many miscellaneous tasks they must perform on a daily basis, thereby helping them to focus on their core business. Below, we will explain each function of the system of the present invention and its specific embodiment.

[2403] Support for automatic creation of tax return documents

[2404] The user prepares a scanner or PDF file of the invoice on their device and uploads the invoice file from their device to the server. The server receives the uploaded file and scans it using optical character recognition (OCR) technology such as Google Cloud Vision API to extract text information. The server determines necessary data from the extracted text information, such as the transaction date, amount, and client, and saves it in an internal Excel template using a library such as openpyxl. The server then arranges the data according to the tax return format and automatically fills it in. The completed tax return is provided via an HTTP response so that the user can download it from their device, or it is sent by email.

[2405] Examples:

[2406] When a user uploads a PDF of an invoice to their device, the server uses OCR technology to extract the necessary information from the PDF (e.g., 100,000 yen, October 3, 2023, business partner X), automatically reflects this information in an Excel spreadsheet and tax return, and provides it to the user.

[2407] Example prompt sentence:

[2408] "How can I extract transaction dates, amounts, and business partners from uploaded PDF invoices and reflect them in Excel templates and tax returns?"

[2409] Creating meeting minutes and summaries

[2410] Users save audio files of online meetings on their devices and then upload them from their devices to the server. The server converts the received audio files into text using the Amazon Transcribe API. Based on the converted text data, minutes and summaries are automatically generated using OpenAI's generative AI model. The completed minutes and summaries are saved in file format for users to download, provided as an HTTP response, or sent via email.

[2411] Examples:

[2412] When a user uploads an audio file of an online conference to their terminal, the server converts the audio into text, generates minutes and a summary summarizing the contents of the conference, and provides them to the user.

[2413] Example prompt sentence:

[2414] "How can I automatically generate minutes and summaries from audio files of online meetings?"

[2415] Transportation and hotel arrangements

[2416] The user enters their business trip dates, transportation method, and accommodation requirements into their device and sends them from the device to the server. Based on the received information, the server uses the Google Maps API or Expedia API to search for the optimal transportation method and accommodation. The server then lists the optimal plans and presents them to the user. The user selects the desired plan from their device and completes the reservation procedure. The server then reserves transportation and accommodation based on the selected plan and sends a confirmation email to the user.

[2417] Examples:

[2418] When a user inputs the dates of a business trip and specifies the desired means of transportation and accommodation conditions, the server proposes the optimal flight and hotel plan, makes a reservation for the plan selected by the user, and sends a confirmation email.

[2419] Example prompt sentence:

[2420] "How can I suggest and book the best travel and accommodation options based on the travel dates and mode of travel entered by the user?"

[2421] Sales and cost information management and strategy proposals

[2422] The user prepares sales and cost data as an Excel file and uploads it from their device to the server. The server reads the received Excel file using the Pandas library and analyzes the data. The server uses Matplotlib and Seaborn to visualize sales and cost trends, and uses a generative AI model (e.g., OpenAI's GPT-4) to generate strategic proposals and improvement measures based on the data. The proposal report is saved in PDF format and made available for the user to download from their device or sent via email.

[2423] Examples:

[2424] When a user uploads an Excel sheet containing recent sales and cost data to their device, the server analyzes the data, creates graphs that visualize sales and cost trends, and generates strategic suggestions based on the data (e.g., "reducing advertising costs by 20% will increase profits by 15%), which are then provided to the user.

[2425] Example prompt sentence:

[2426] "How can I analyze sales and cost data and automatically generate strategic proposals?"

[2427] The flow of the identification process in the first embodiment will be described with reference to FIG.

[2428] Support for automatic creation of tax return documents

[2429] Step 1:

[2430] The user prepares a scanner or PDF file of the invoice on the device.

[2431] Specifically, users scan a paper invoice or save an existing digital invoice to their PC or smartphone.

[2432] Step 2:

[2433] The user uploads the invoice file from the terminal to the server.

[2434] Input: Invoice PDF or scanned file

[2435] Specifically, the user opens a file selection dialog of the browser, selects a file, and clicks the upload button.

[2436] Output: The invoice file is sent to the server.

[2437] Step 3:

[2438] The server receives the uploaded file.

[2439] Input: Uploaded invoice file

[2440] Specifically, the server receives the HTTP POST request and saves the file in temporary storage.

[2441] Output: Invoice file saved in storage

[2442] Step 4:

[2443] The server calls the Google Cloud Vision API or similar, scans the file using OCR technology, and extracts text information.

[2444] Input: Saved invoice file

[2445] Specifically, the server uses an OCR engine to extract text and obtain character information in JSON format.

[2446] Output: Extracted text information

[2447] Step 5:

[2448] The server automatically determines necessary data such as transaction date, amount, and trading partner from the extracted text information.

[2449] Input: Extracted text information

[2450] Specifically, the server uses a natural language processing library (e.g., spaCy) to analyze the required data and extract specific fields.

[2451] Output: Transaction date, amount, counterparty, etc.

[2452] Step 6:

[2453] The server saves the identified data in an internal Excel template.

[2454] Input: Transaction date, amount, counterparty, etc.

[2455] Specifically, the server uses a library such as openpyxl to embed data into the relevant cells of the Excel template.

[2456] Output: Excel file with completed tax return

[2457] Step 7:

[2458] The server provides the completed Excel file for the user to download from their device or sends it by email.

[2459] Input: Completed Excel file

[2460] Specifically, the server returns the file as an HTTP response, or sends the file to the user using an email sending API.

[2461] Output: Tax return provided to user

[2462] Creating meeting minutes and summaries

[2463] Step 1:

[2464] The user saves the audio file of the online conference on the device.

[2465] Specifically, the user uses the recording function of the conference app to save the audio file.

[2466] Step 2:

[2467] The user uploads an audio file from the device to the server.

[2468] Input: Online meeting audio file

[2469] Specifically, the user selects a file from a dedicated form and clicks the upload button.

[2470] Output: The audio file is sent to the server.

[2471] Step 3:

[2472] The server saves the received audio file.

[2473] Input: Uploaded audio file

[2474] Specifically, the server receives the HTTP POST request and temporarily stores it in storage.

[2475] Output: Audio file saved in storage

[2476] Step 4:

[2477] The server converts the speech to text using the Amazon Transcribe API.

[2478] Input: Saved audio file

[2479] Specifically, the server sends the audio file to the API and receives the converted text.

[2480] Output: Text data

[2481] Step 5:

[2482] The server uses OpenAI's generative AI model to analyze the text data and automatically generate minutes and summaries.

[2483] Input: Text data

[2484] Specifically, the server calls the generative AI model and generates minutes and summaries based on the text data.

[2485] Output: Generated minutes and summary

[2486] Step 6:

[2487] The server saves the generated minutes and summaries in a file format and provides them for users to download or sends them by email.

[2488] Input: Generated minutes and summaries

[2489] Specifically, the server returns the file as an HTTP response or sends it using an email sending API.

[2490] Output: Minutes and summary provided to user

[2491] Transportation and hotel arrangements

[2492] Step 1:

[2493] The user inputs the business trip schedule, transportation method, and accommodation conditions into the terminal and transmits them to the server.

[2494] Input: business trip schedule, transportation, accommodation conditions

[2495] Specifically, the user enters the required information into the input form and clicks the submit button.

[2496] Output: Information sent to the server

[2497] Step 2:

[2498] Based on the information received by the server, the server uses the Google Maps API and Expedia API to search for the best transportation and accommodation options.

[2499] Input: business trip schedule, transportation, accommodation conditions

[2500] Specifically, the server calls the API and retrieves the plan that meets the conditions.

[2501] Output: A list of the best transportation and accommodation options

[2502] Step 3:

[2503] The server organizes the search results and generates an HTML page to present to the user.

[2504] Input: A list of the best trans...

Claims

1. A means for analyzing invoices using an artificial intelligence model and automatically transcribing transaction data extracted from the invoices into tax return documents; means for converting the audio data into text and generating minutes and summaries of the meeting; A means for inputting travel dates and accommodation requirements to suggest and arrange optimal transportation and accommodation; A means of analyzing sales and cost data and making strategic improvement proposals; A system including:

2. 10. The system of claim 1, wherein the system analyzes the bill using optical character recognition technology.

3. 10. The system of claim 1, wherein the system generates meeting minutes and summaries using a generative artificial intelligence model.

4. 10. The system of claim 1, wherein the system uses an online search function to suggest transportation and accommodations.

Images