system

Abstract

We provide the system. [Solution] Means for receiving documents, A database means for storing received documents, A natural language processing method for analyzing saved documents and extracting important information, A generative artificial intelligence means for generating a summary using extracted information, A document generation method for automatically generating quotations, delivery notes, invoices, approval documents, and seal application forms based on summarized information, An accounting integration method for generating journal entry information from the generated documents and reflecting it in the accounting system, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot performed by at least one processor, the method including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance as a response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Efficiently managing various documents such as proposal materials, contracts, and emails in business activities and quickly and accurately generating necessary documents is an important issue. However, current systems lack functions for integrated management, analysis, summarization, and automatic generation of these documents, so a lot of manual work is required and errors are likely to occur. In addition, cooperation with the accounting department becomes complicated, resulting in a decrease in the efficiency of the entire business process. It is required to solve such problems and improve the efficiency and accuracy of business activities.

Means for Solving the Problems

[0005] The present invention solves the above problems with a system that includes means for receiving documents, database means for storing received documents, natural language processing means for analyzing stored documents and extracting important information, generative artificial intelligence means for generating summaries using the extracted information, document generation means for automatically generating quotations, delivery notes, invoices, approval documents, and seal application forms based on the summarized information, and accounting linkage means for generating journal entry information from the generated documents and reflecting it in an accounting system.

[0006] Specifically, users upload documents such as proposals, contracts, and emails to the sales system. The server automatically aggregates these documents into a database, analyzes them using natural language processing, and summarizes the necessary information using generative artificial intelligence. Based on this summarized information, quotations, delivery notes, invoices, approval forms, and stamp application forms are automatically generated by embedding information into templates. Subsequently, journal entry information is automatically generated from the generated documents and reflected in the accounting system, thereby reducing manual work, preventing errors, and improving the efficiency and accuracy of operations.

[0007] "Documents" refers to all documents related to sales activities, such as proposals, contracts, and emails.

[0008] A "database" is a storage device used to store and manage received documents.

[0009] "Natural language processing" is a technology that uses computers to analyze human language and extract important information.

[0010] "Generative artificial intelligence" refers to artificial intelligence used to generate summaries and other texts based on extracted information.

[0011] A "quotation" is a document presented before a transaction, which lists the price of goods or the cost of services.

[0012] A "delivery note" is a document that records the contents, quantity, price, and other details of goods delivered to the customer.

[0013] An "invoice" is a document sent to request payment after a transaction has been completed.

[0014] A "proposal document" is an internal document that contains the information necessary to obtain approval within an organization.

[0015] A "seal application form" is an application form used to obtain approval for affixing a seal to official documents within an organization.

[0016] An "accounting system" is a software or hardware system used to manage a company's financial data and process journal entries.

[0017] "Journal entry information" refers to accounting data used to record transactions in accounting operations. [Brief explanation of the drawing]

[0018] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8]It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

Embodiments for Carrying Out the Invention

[0019] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0020] First, the terms used in the following description will be explained.

[0021] In the following embodiments, a labeled processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0022] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0023] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0024] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0026] [First Embodiment]

[0027] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0028] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0031] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0034] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0038] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0039] The system for implementing this invention has the functionality to centrally manage various documents used in sales activities and to automatically perform analysis, summarization, document generation, and accounting processing. The specific program processing and its operation are described below in natural language.

[0040] System Configuration and Basic Operation

[0041] Information aggregation

[0042] As a sales representative, the user creates documents such as proposals, contracts, and emails on their device and uploads them to the sales system.

[0043] The server receives uploaded documents in real time and saves all documents to the database. During saving, each document is assigned a unique ID, and its metadata (creation date, customer name, etc.) is recorded.

[0044] Document reading and analysis

[0045] The server sequentially reads documents from the database and performs analysis using natural language processing capabilities. During the analysis process, optical character recognition (OCR) technology is used to convert PDF and image-based documents into text.

[0046] The server performs analysis on the text data to extract important information such as proper nouns, numerical data, and dates. This analysis uses predefined templates and machine learning models.

[0047] Summary generation

[0048] The server inputs key information extracted through natural language processing into a generative artificial intelligence (AI) to generate a summary. This AI then creates a short, concise summary based on the extracted information.

[0049] The server stores the summary in a database so that it can be used in subsequent processing.

[0050] Automatic document generation

[0051] The server automatically generates quotations, delivery notes, invoices, approval forms, and stamping requests by embedding them into templates based on the summary text and extracted information.

[0052] The server saves each generated document in PDF format and automatically notifies the relevant parties.

[0053] Information sharing with accounting

[0054] The server automatically generates the necessary journal entries from the information in the generated quotes and invoices. Specifically, it creates journal entries for accounts receivable and sales based on information such as customer name, transaction date, and amount.

[0055] The server automatically reflects the generated journal entries in the accounting system. This reduces manual data entry by accounting staff and prevents errors.

[0056] Specific example

[0057] Example 1: Proposal for a new project

[0058] Users create proposal documents for new projects on their devices and upload them to the system.

[0059] The server receives the proposal document and saves it to the database. Simultaneously with saving, it analyzes the content and extracts information such as the project name, client name, proposal details, and budget.

[0060] The server passes this information to a generative AI to create a summary. The summary includes the project's key points and important information.

[0061] Example 2: Automatic generation of quotations

[0062] The user enters the price information agreed upon with the customer into the system.

[0063] The server uses that information to embed a quotation into a template and automatically generates it. The quotation will include the customer name, project name, price, delivery date, etc.

[0064] The server saves the generated quotation as a PDF and automatically sends it to the customer.

[0065] Example 3: Automation of accounting processes

[0066] The server automatically generates journal entries for sales and accounts receivable based on the generated invoices. These generated journal entries are then reflected in the accounting system based on the transaction date, amount, and other factors.

[0067] The user, acting as an accounting professional, reviews the journal entries automatically generated within the accounting system and makes corrections or approvals as needed.

[0068] The system implemented in this invention streamlines the entire process from sales to accounting, significantly reducing errors caused by manual work. As a result, it is expected that overall business productivity will improve and the workflow will run more smoothly.

[0069] The following describes the processing flow.

[0070] Step 1:

[0071] Users create documents such as proposals, contracts, and emails on their devices and upload them to the sales system.

[0072] Step 2:

[0073] The server receives uploaded documents in real time and saves them to the database. A unique ID is assigned to each document upon saving.

[0074] Step 3:

[0075] The server sequentially reads documents from the database and uses optical character recognition (OCR) to convert PDF and image-based documents into text.

[0076] Step 4:

[0077] The server performs natural language processing on the converted text data to extract important information such as customer name, project name, contract amount, and delivery date.

[0078] Step 5:

[0079] The server inputs the extracted information into a generative artificial intelligence (AI) to generate a summary. This summary includes the project's key points and important matters.

[0080] Step 6:

[0081] The server saves the generated summary text to a database, making it available for use in the document generation process.

[0082] Step 7:

[0083] The server automatically generates templates for quotations, delivery notes, invoices, approval documents, and stamping requests, by embedding the information based on the summary text and extracted data.

[0084] Step 8:

[0085] The server saves each generated document in PDF format and automatically notifies the relevant parties. For example, a quotation would be sent to the customer, and a proposal would be sent to the internal approver.

[0086] Step 9:

[0087] The server automatically generates accounting journal entries from the information in the generated quotations and invoices. Specifically, it creates journal entries for accounts receivable and sales.

[0088] Step 10:

[0089] The server automatically reflects the generated journal entries into the accounting system. This eliminates the need for manual data entry for accounting staff.

[0090] Step 11:

[0091] The user, acting as an accounting professional, reviews the automatically generated journal entries in the accounting system and makes corrections or approvals as needed.

[0092] In this way, the entire system is integrated, streamlining the entire process from sales to accounting.

[0093] (Example 1)

[0094] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0095] Traditional document management and processing systems for sales activities require the use of multiple independent tools, making it difficult to efficiently manage a series of business processes. Furthermore, manual data entry and document generation are prone to human error, resulting in inefficiencies and inaccuracies. To address these issues, there is a need for a system that centrally manages the entire sales process, automatically handling analysis, summarization, document generation, and accounting.

[0096] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0097] In this invention, the server includes means for receiving various documents related to sales activities in order to aggregate information, database means for storing the received documents, means for extracting important information from the stored documents using natural language processing means, means for generating summaries using generative artificial intelligence means based on the extracted information, document generation means for automatically generating quotations, delivery notes, invoices, approval documents, and stamp application forms based on the summarized information, and accounting linkage means for generating journal entry information based on the generated quotations and invoices and reflecting it in the accounting system. This makes it possible to centrally manage and automate a series of business processes from sales activities to accounting processing.

[0098] "A means of receiving various documents related to sales activities in order to consolidate information" refers to a system that allows users to upload documents such as proposals, contracts, and emails related to sales activities from their terminals to the server.

[0099] A "database system for storing received documents" refers to a system that receives uploaded documents in real time and stores them in a database. The system includes a function to assign a unique ID and metadata to each document during storage.

[0100] "Natural language processing means" refers to technologies that have the function of analyzing text information from stored documents and extracting important information (proper nouns, numerical information, dates, etc.). Typically, this includes the process of converting PDF or image-format documents into text using OCR technology.

[0101] "Generative artificial intelligence methods" refer to artificial intelligence technologies used to generate summary texts based on extracted information. Generally, natural language generation models are used for summarization.

[0102] The "document generation method" is a system that automatically generates various documents such as quotations, delivery notes, invoices, approval forms, and seal application forms by embedding them into templates based on the generated summary text and extracted information.

[0103] The "accounting integration method" is a function that generates necessary journal entries based on the information in the generated quotations and invoices, and automatically reflects this information in the accounting system. Specifically, it generates journal entries for accounts receivable and sales.

[0104] "Optical character recognition means" refers to a technology used to extract text information from PDF and image-format documents, and is known as OCR (Optical Character Recognition).

[0105] "Information embedding based on templates" is an approach that automatically inserts the necessary information and generates a document according to a predefined format.

[0106] The system for implementing this invention centrally manages various documents used in sales activities and automates document analysis, summary generation, document creation, and accounting processing. The specific program processing and operation are described below.

[0107] System-wide configuration

[0108] This system consists of a server, user terminals (PCs, tablets, smartphones), and various software. Specific software used includes database management systems (e.g., MySQL®), OCR software (e.g., Tesseract), natural language processing libraries (e.g., SpaCy, NLTK), and machine learning models (e.g., TENSORFLOW®, PyTorch).

[0109] Information aggregation

[0110] Users create documents such as proposals, contracts, and emails related to sales activities on their terminals and upload them to the sales system. The server receives the uploaded documents in real time and saves all documents to the database. When saving, each document is assigned a unique ID and metadata (creation date, customer name, etc.).

[0111] Document reading and analysis

[0112] The server sequentially reads documents from the database and uses OCR software (e.g., Tesseract) to convert PDF and image documents into text. Next, the server uses a natural language processing library (e.g., SpaCy, NLTK) to extract important information such as proper nouns, numerical data, and dates from the text data.

[0113] Summary generation

[0114] The server inputs the extracted key information into a generative AI model (e.g., GPT-3®) to generate a summary. The generated summary is then stored in the database for use in subsequent processing.

[0115] Automatic document generation

[0116] The server automatically generates quotations, delivery notes, invoices, approval forms, and stamping requests by embedding them into templates based on the summary text and extracted information. Each generated document is saved in PDF format and automatically notified to the relevant parties.

[0117] Information sharing with accounting

[0118] The server automatically generates the necessary journal entries from the information in the generated quotes and invoices, and reflects them in the accounting system (e.g., Oracle Financials). This reduces manual data entry by accounting staff and prevents errors.

[0119] Specific example

[0120] Example 1: Proposal for a new project

[0121] 1. The user creates a proposal document for a new project on their device and uploads it to the system.

[0122] 2. The server receives the proposal documents and saves them to the database.

[0123] 3. The server performs OCR processing on the proposal documents and converts them into text data.

[0124] 4. The server extracts information such as "customer name" and "proposal details" from the text data.

[0125] 5. The server passes the extracted information to the generative AI model and generates a summary.

[0126] 6. The server saves the generated summary text.

[0127] Example 2: Automatic generation of quotations

[0128] 1. The user enters the price information agreed upon with the customer into the system.

[0129] 2. The server automatically generates a quotation based on that information.

[0130] 3. The server saves the generated quotation in PDF format and automatically sends it to the customer.

[0131] Example 3: Automation of accounting processes

[0132] 1. The server automatically generates sales entries and accounts receivable journal entries based on the generated invoices.

[0133] 2. The server updates the accounting system with the journal entries it has generated.

[0134] 3. The user, acting as an accounting staff member, reviews the journal entries automatically generated within the accounting system and makes corrections or approvals as necessary.

[0135] Example of a prompt

[0136] "Please upload your proposal for a new project. The system will automatically generate a summary."

[0137] "Please enter the pricing information agreed upon with the customer. A quote will be automatically generated and sent to the customer."

[0138] "The invoice has been generated. Please check the journal entry information that will be automatically reflected in the accounting system."

[0139] In this way, it becomes possible to implement a system in which a series of business processes are executed efficiently and automatically.

[0140] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0141] Step 1:

[0142] Users create documents such as proposals, contracts, and emails related to sales activities on their terminals and upload them to the sales system. These input documents are received by the system in real time. Users also select files from the system's upload screen and click the "Upload" button.

[0143] Step 2:

[0144] The server receives uploaded documents in real time. The received documents are saved to a database (e.g., MySQL). During saving, each document is assigned a unique ID and metadata (creation date, customer name, etc.). The input to this process is the uploaded documents, and the output is the documents stored in the database along with their metadata.

[0145] Step 3:

[0146] The server sequentially reads documents stored in the database. These read documents are converted into text using OCR software (e.g., Tesseract). The converted text data becomes the input for the next process, and the PDF or image-formatted documents are output as text data.

[0147] Step 4:

[0148] The server analyzes text data using a natural language processing library (e.g., SpaCy, NLTK) to extract important information such as proper nouns, numerical data, and dates. The input to this process is text data, and the output is the extracted important information. Specifically, it extracts information such as "customer name," "contract amount," and "due date" from documents.

[0149] Step 5:

[0150] The server inputs the extracted key information into a generative AI model (e.g., GPT-3) to generate a summary. The generated summary is then stored in the database and used in the next process. The input is the key information, and the output is the generated summary.

[0151] Step 6:

[0152] The server automatically generates quotations, delivery notes, invoices, approval forms, and stamping requests by embedding them into templates based on the summary text and extracted information. The input for this process is the summary text and extracted information, and the output is the generated PDF documents. The documents are automatically notified to the relevant parties.

[0153] Step 7:

[0154] The server automatically generates the necessary journal entries based on the information from the generated quotes and invoices, and reflects them in the accounting system (e.g., Oracle Financials). This operation generates journal entries for accounts receivable and sales based on customer name, transaction date, amount, etc. The input is the information from the quotes and invoices, and the output is the generated journal entries.

[0155] Step 8:

[0156] The user, acting as an accounting professional, reviews the automatically generated journal entries within the accounting system and makes corrections or approvals as needed. The input is the generated journal entry information, and the output is the final approved journal entry information. This operation is expected to ensure the smooth progress of the overall business process.

[0157] (Application Example 1)

[0158] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0159] In modern manufacturing, a large volume of documents are handled on the production line, and managing and processing them requires considerable effort. Furthermore, manual document management is prone to human error and delays in verification, significantly reducing operational efficiency. In particular, critical documents such as manufacturing instructions, inspection records, and delivery slips require real-time verification and management, but this is not adequately achieved currently. To solve this problem and improve operational efficiency within the factory, a new automated document management system is needed.

[0160] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0161] In this invention, the server includes means for receiving documents, database means for storing the received documents, and natural language processing means for analyzing the stored documents and extracting important information. This makes it possible to efficiently manage stored documents and automatically extract and process the necessary information.

[0162] "Means for receiving documents" refers to a function that allows users to upload documents such as proposals, contracts, and emails to the server via the internet.

[0163] A "database system" is a system for organizing and storing received documents and managing them in a format that can be used for searching and analysis.

[0164] "Natural language processing techniques" are technologies used to analyze the content of stored documents and extract important information such as proper nouns, numerical data, and dates.

[0165] "Generative artificial intelligence methods" refer to technologies that utilize generative AI models to create short, concise summaries based on information extracted through natural language processing.

[0166] A "document generation method" is a system that automatically creates documents such as quotations, delivery notes, invoices, approval documents, and stamp application forms based on summarized and extracted information.

[0167] "Accounting integration means" refers to technology that generates necessary journal entry information based on automatically generated quotation and invoice information, and reflects that information in the accounting system.

[0168] "Optical character recognition means" refers to a technology for analyzing PDF and image-format documents and extracting text information.

[0169] The "wearable device integration method" is a system for displaying extracted and summarized information in real time on wearable devices such as smart glasses.

[0170] The system for implementing this invention is designed to improve document management and operational efficiency within a factory. Specifically, it utilizes technologies such as natural language processing, generative artificial intelligence, optical character recognition, and wearable device integration. The detailed configuration and operation of this system are described below.

[0171] System Configuration and Basic Operation

[0172] Information aggregation

[0173] Users create documents such as manufacturing instructions, inspection records, and delivery slips on their terminals and upload them to the system.

[0174] The server receives uploaded documents and saves all of them to the database. During saving, each document is assigned a unique ID, and its metadata (creation date, product name, etc.) is recorded.

[0175] Document reading and analysis

[0176] The server sequentially reads documents from the database and uses optical character recognition (OCR) technology to convert PDF and image-based documents into text. Specifically, Tesseract OCR is used as the OCR technology.

[0177] The server performs natural language processing on the text data to extract important information such as proper nouns, numerical data, and dates. This analysis uses predefined templates and machine learning models such as spaCy and BERT.

[0178] Summary generation

[0179] The server inputs key information extracted through natural language processing into a generative artificial intelligence (AI) to generate a summary. GPT-3 is used as the generative AI model.

[0180] The server stores the generated summary in a database so that it can be used in subsequent processing.

[0181] Automated document generation and accounting integration

[0182] The server automatically generates quotations, delivery notes, invoices, etc., by embedding them into templates based on the summary text and extracted information.

[0183] The server saves each generated document in PDF format and automatically notifies the relevant parties.

[0184] The server automatically generates the necessary journal entries from the information in the generated invoices and reflects them in the accounting system.

[0185] Wearable device integration

[0186] Users wear smart glasses to view information from manufacturing instructions and inspection records in real time.

[0187] The server displays extracted and summarized information on smart glasses in real time. This allows users to improve their work efficiency in the field.

[0188] Specific example

[0189] Example 1: Processing instructions for a new production line

[0190] The user creates instructions for a new production line on a terminal and uploads them to the system.

[0191] The server receives the instruction sheet and saves it to the database. Simultaneously with saving, it analyzes the contents and extracts information such as the product name, product code, and work procedure.

[0192] The server passes this information to a generative AI, which then creates a summary. The summary includes the key points of the instructions.

[0193] Example 2: Automatic generation of quotations

[0194] The user enters the price information agreed upon with the customer into the system.

[0195] The server uses that information to embed a quotation into a template and automatically generates it. The quotation will include the customer name, product name, price, delivery date, etc.

[0196] The server saves the generated quotation as a PDF and automatically sends it to the customer.

[0197] Example of a prompt

[0198] Examples of prompt statements to input into a generative AI model are as follows:

[0199] The project name is "Introduction of a New Manufacturing Line," the client is "Company A," the budget is "¥5,000,000," and the deadline is "December 31, 2023." Summarize the key points in three short sentences.

[0200] Using this prompt, GPT-3 generates a summary like this:

[0201] 1. A project to introduce a new manufacturing line.

[0202] 2. The customer is a general company A.

[0203] 3. The budget is ¥5,000,000, and the deadline is December 31, 2023.

[0204] In this way, the present invention can significantly improve document management and operational efficiency within a factory.

[0205] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0206] Step 1:

[0207] Upload document

[0208] Users create documents such as manufacturing instructions, inspection records, and delivery slips on their terminals and upload them to the system. At this time, the document files are read as input.

[0209] The uploaded document is sent to the server.

[0210] Step 2:

[0211] Save document

[0212] The server saves received documents to a database. During saving, a unique ID is assigned to each document, and its metadata (creation date, product name, etc.) is recorded. Input is the uploaded document file, and output is the document stored in the database.

[0213] Specifically, files are stored on a file server, and their metadata is registered in a relational database (such as MySQL).

[0214] Step 3:

[0215] Text conversion using Optical Character Recognition (OCR)

[0216] The server converts stored documents into text using OCR technology (Tesseract OCR). The input is a stored document file, and the output is text data.

[0217] In this step, text information is extracted from images and PDFs and taken out as structured text data.

[0218] Step 4:

[0219] Extracting important information using natural language processing

[0220] The server analyzes the text data obtained by OCR using natural language processing (using models such as spaCy or BERT) and extracts important information such as proper nouns, numerical data, and dates. The input is the text data obtained by OCR, and the output is the extracted important information.

[0221] Specifically, it analyzes text data to perform tasks such as entity identification, relationship extraction, and topic classification.

[0222] Step 5:

[0223] Summary generation

[0224] The server inputs the extracted key information into a generative AI (GPT-3) to generate a summary. An example of a prompt is: "The project name is 'Introduction of a new manufacturing line,' the customer name is 'General Company A,' the budget is '¥5,000,000,' and the deadline is 'December 31, 2023.' Please summarize the key points in three short sentences." The input is the extracted key information, and the output is a summary.

[0225] Send prompts to the generating AI to obtain an appropriate summary.

[0226] Step 6:

[0227] Automatic document generation

[0228] The server automatically generates quotations, delivery notes, invoices, etc., by embedding them into templates based on the generated summary text and extracted information. The input is the summary text and extracted information, and the output is the generated documents.

[0229] By embedding information into a template file, a formally correct document is generated.

[0230] Step 7:

[0231] Integration with accounting systems

[0232] The server automatically generates the necessary journal entries from the information in the generated invoices and reflects them in the accounting system. The input is the generated invoice, and the output is the journal entries reflected in the accounting system.

[0233] Accounts receivable and sales data are automatically registered in the accounting system via the accounting API.

[0234] Step 8:

[0235] Use of wearable devices

[0236] The user wears smart glasses to view information from manufacturing instructions and inspection records in real time. Input is the generated summary and extracted information, and output is the information displayed on the smart glasses.

[0237] Data is sent to smart glasses (e.g., Google Glass®) to display highly visible information.

[0238] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0239] The system for implementing this invention combines functions for centrally managing various documents in sales activities, automatically performing analysis, summarization, document generation, and accounting processing, with an emotion engine that recognizes user emotions. The specific program processing and operation are described below in natural language.

[0240] System Configuration and Basic Operation

[0241] Information aggregation and emotion recognition

[0242] As a sales representative, the user creates documents such as proposals, contracts, and emails on their device and uploads them to the sales system.

[0243] The server receives uploaded documents in real time and saves them to the database. Upon saving, each document is assigned a unique ID, and its metadata (creation date, customer name, etc.) is recorded.

[0244] The server activates an emotion engine to recognize the user's emotions and acquire real-time emotion data from the user. For example, it acquires emotion data by integrating factors such as the speed of keyboard input when the user is writing, the tone of voice when using voice input, and facial expression analysis using a camera.

[0245] Document reading and analysis

[0246] The server sequentially reads documents from the database and uses optical character recognition (OCR) to convert PDF and image-based documents into text.

[0247] The server performs natural language processing on the converted text data to extract important information such as customer name, project name, contract amount, and delivery date. Here, sentiment information from the sentiment engine is used to prioritize extracting information that the user wants to emphasize or considers particularly important.

[0248] Summary generation

[0249] The server inputs the extracted key information into a generative artificial intelligence (AI) to generate a summary. This summary includes the project's key points and important matters. Furthermore, based on data from the emotion engine, it creates a summary that reflects the emotions the user wants to emphasize.

[0250] The server saves the generated summary text to a database, making it available for use in the document generation process.

[0251] Automatic document generation

[0252] The server automatically generates templates for quotations, delivery notes, invoices, approval forms, and stamping requests, embedding the information based on the summary and extracted data. Sentimental information is also used to adjust the tone and content of the documents to match the user's emotions.

[0253] The server saves each generated document in PDF format and automatically notifies the relevant parties. For example, a quotation would be sent to the customer, and a proposal would be sent to the internal approver.

[0254] Information sharing with accounting

[0255] The server automatically generates accounting journal entries from the information in the generated quotations and invoices. Specifically, it creates journal entries for accounts receivable and sales.

[0256] The server automatically reflects the generated journal entries into the accounting system. This eliminates the need for manual data entry for accounting staff.

[0257] Specific example

[0258] Example 1: Proposal for a new project

[0259] The user creates a proposal document for a new project on their device and uploads it to the system. The emotion engine analyzes the user's typing speed, facial expressions, and voice to determine how passionate the user is about the project.

[0260] The server receives the proposal documents and saves them to the database, also recording sentiment data. The server then analyzes the content and extracts information such as the project name, client name, proposal details, and budget.

[0261] The server then passes this information and sentiment data to a generative AI to create a summary. The summary includes the core points of the project and information that the user particularly wants to emphasize.

[0262] Example 2: Automatic generation of quotations

[0263] The user enters the price information agreed upon with the customer into the system. Here again, the emotion engine analyzes the user's emotions at the time of input.

[0264] The server uses that information to embed a quotation into a template and automatically generates it. The quotation includes the customer name, project name, price, delivery date, etc., and the content is adjusted to a tone that matches the user's sentiment.

[0265] The server saves the generated quotation as a PDF and automatically sends it to the customer.

[0266] Example 3: Automation of accounting processes

[0267] The server automatically generates sales entries and accounts receivable journal entries based on the generated invoices. During this process, sentiment information is used to prioritize accounting processes and display important notes.

[0268] The server automatically reflects the generated journal entries in the accounting system. Accounting staff are freed from manual entry and can review the entries with reference to notes based on sentiment data from the system.

[0269] The system implementing this invention streamlines the entire process from sales to accounting, and enables the generation of more personalized documents that reflect the user's emotions. As a result, overall business productivity improves, and more effective customer service is achieved.

[0270] The following describes the processing flow.

[0271] Step 1:

[0272] Users, acting as sales representatives, create documents such as proposals, contracts, and emails on their devices and upload them to the sales system.

[0273] Step 2:

[0274] The server receives uploaded documents in real time and saves them to the database. During saving, each document is assigned a unique ID, and metadata (creation date, customer name, etc.) is also recorded.

[0275] Step 3:

[0276] The server activates an emotion engine to acquire emotional data from the user's input speed, clicks, facial expressions, and voice input. This allows for the collection of real-time emotional data from the user.

[0277] Step 4:

[0278] The server sequentially reads out the documents in the database and uses optical character recognition (OCR) to convert documents in PDF or image format into text.

[0279] Step 5:

[0280] The server performs natural language processing on the converted text data and extracts important information such as customer name, project name, contract amount, delivery date, etc. Here, the sentiment data from the sentiment engine is used to preferentially extract the information that the user particularly values.

[0281] Step 6:

[0282] The server inputs the extracted important information into generative artificial intelligence (AI) to generate a summary. The summary includes the key points and important matters of the project, and is further adjusted to reflect the sentiment that the user wants to emphasize based on the sentiment data.

[0283] Step 7:

[0284] The server saves the generated summary text in the database so that it can be used in the document generation process.

[0285] Step 8:

[0286] The server automatically generates by embedding information into each template of quotation, delivery note, invoice, memo, and seal application based on the summary text and the extracted information. Here, the sentiment data is also used to adjust the tone and content of the document to match the user's sentiment.

[0287] Step 9:

[0288] The server saves each generated document in PDF format and automatically notifies the relevant parties. For example, in the case of a quotation, it is sent to the customer, and in the case of a memo, it is sent to the internal approver.

[0289] Step 10:

[0290] The server automatically generates accounting journal entries from the information in the generated quotations and invoices. Specifically, it creates journal entries for accounts receivable and sales.

[0291] Step 11:

[0292] The server automatically reflects the generated journal entries into the accounting system. This eliminates the need for manual data entry for accounting staff.

[0293] Step 12:

[0294] The user, acting as an accountant, reviews automatically generated journal entries in the accounting system and makes corrections or approvals as needed. Sentimental data is also used here to highlight points that require particular attention.

[0295] In this way, user sentiment data is reflected throughout the entire process, from sales activities to accounting, enabling more effective and personalized responses. This improves operational efficiency and accuracy, and increases overall business productivity.

[0296] (Example 2)

[0297] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0298] Traditional systems struggled to efficiently manage sales activity documents, extract key information, and handle accounting processes consistently through automatically generated documents. Furthermore, generating documents that reflected user sentiment was difficult, resulting in insufficient individual support. This led to cumbersome sales and accounting processes and decreased productivity.

[0299] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0300] In this invention, the server includes means for receiving documents, database means for storing received documents, natural language processing means for analyzing stored documents and extracting important information, generative artificial intelligence means for generating summaries using the extracted information, emotion recognition means for acquiring and analyzing emotion data, means for adjusting the priority of information based on the acquired emotion data, document generation means for automatically generating quotations, delivery notes, invoices, approval documents, and stamp application forms based on the summarized information, accounting linkage means for generating journal entry information from the generated documents and reflecting it in the accounting system, and means for saving the generated various documents in PDF format and automatically notifying relevant parties. This streamlines the entire process from sales to accounting and enables the generation of personalized documents that reflect the user's emotions.

[0301] "Means for receiving documents" refers to the means by which the server receives documents uploaded by the user from their device.

[0302] "Database means for saving received documents" refers to database technology for recording and storing received documents along with a unique ID and metadata.

[0303] "Natural language processing means for analyzing saved documents and extracting important information" refers to natural language processing technology that analyzes text data within saved documents and extracts important information such as customer names, project names, contract amounts, and delivery dates.

[0304] "Generative artificial intelligence means for generating summaries using extracted information" refers to generative artificial intelligence technology for generating summary texts based on extracted important information.

[0305] The "emotion recognition means for acquiring and analyzing emotion data" is a technology for acquiring and analyzing the user's real-time emotion data (such as the speed of keyboard input, voice tone, expression, etc.).

[0306] The "means for adjusting the priority of information based on the acquired emotion data" is a technology for adjusting the importance and priority of information based on the user's emotion data.

[0307] The "document generation means for automatically generating quotations, delivery notes, invoices, reports, and seal application forms based on the summarized information" is a technology for automatically generating various documents (quotations, delivery notes, invoices, reports, seal application forms) based on the summarized information.

[0308] The "accounting linking means for generating accounting transaction information from the generated documents and reflecting it in the accounting system" is a linking technology for automatically generating accounting transaction information from the generated documents and reflecting it in the accounting system.

[0309] The "means for saving the generated various documents in PDF format and automatically notifying the related parties" is a means for saving the automatically generated various documents in PDF format and automatically notifying the related parties (such as customers or internal approvers) related to each document.

[0310] The system for implementing this invention has the function of centrally managing documents in business activities and automatically performing analysis, summarization, document generation, and accounting processing. By incorporating an emotion engine that recognizes the user's emotions, this system realizes document generation that can be more individually tailored. The following describes the specific embodiments of this system.

[0311] System Configuration and Basic Operations

[0312] Aggregation of Information and Emotion Recognition

[0313] As a sales representative, the user creates documents such as proposals, contracts, and emails on their device and uploads them to the sales system. The devices used are PCs and mobile devices.

[0314] The server receives uploaded documents in real time and saves them to the database. Upon saving, each document is assigned a unique ID, and metadata (creation date, customer name, etc.) is recorded.

[0315] The server runs an emotion engine to acquire real-time emotion data from the user. Specifically, it acquires emotion data by integrating the user's keyboard input speed, tone of voice in the case of voice input, and facial expression analysis using a camera. Software used for emotion analysis includes TensorFlow.

[0316] Document reading and analysis

[0317] The server sequentially reads documents from the database and uses optical character recognition (OCR) to convert PDF and image-based documents into text. Specifically, Tesseract OCR is used.

[0318] The server performs natural language processing (NLP) on the converted text data to extract important information such as customer name, project name, contract amount, and delivery date. Sentimental information from an emotion engine is used to prioritize extracting information that the user wants to emphasize or considers particularly important. Examples of NLP engines used include spaCy and NLTK.

[0319] Summary generation

[0320] The server inputs the extracted key information into a generative artificial intelligence model to generate a summary. OpenAI's GPT-3 is used as the generative AI model. This summary includes project points and important matters. Furthermore, based on sentiment engine data, it creates a summary that reflects the emotions the user wants to emphasize.

[0321] Example prompt: "Generate a summary based on the following information: Customer name is ABC Corporation, contract amount is 1 million yen, proposal details are xxxx"

[0322] The server saves the generated summary text to a database so that it can be used in subsequent document generation processes.

[0323] Automatic document generation

[0324] The server automatically generates templates for quotations, delivery notes, invoices, approval forms, and stamping requests, embedding the information based on the summary text and extracted data. Sentimental information is also used to adjust the tone and content of the documents to match the user's emotions.

[0325] The server saves each generated document in PDF format and automatically notifies the relevant parties. For example, a quotation would be sent to the customer, and a proposal would be sent to the internal approver.

[0326] Information sharing with accounting

[0327] The server automatically generates accounting journal entries from the information in the generated quotations and invoices. Specifically, it creates journal entries for accounts receivable and sales.

[0328] The server automatically reflects the generated journal entries into the accounting system. This eliminates the need for manual data entry for accounting staff. Specific accounting systems used include SAP and QuickBooks.

[0329] Specific example

[0330] Example 1: Proposal for a new project

[0331] Users create proposal documents for new projects on their devices and upload them to the system. The emotion engine analyzes the user's typing speed, facial expressions, and voice to determine how passionate the user is about the project.

[0332] The server receives the proposal documents and saves them to the database, also recording sentiment data. The server then analyzes the content and extracts information such as the project name, client name, proposal details, and budget.

[0333] The server then passes this information and sentiment data to a generative AI to create a summary. The summary includes the core points of the project and information that the user particularly wants to emphasize.

[0334] Example prompt: "Generate a summary based on the following information: Customer name is X Corporation, contract amount is 5 million yen, proposed content is the introduction of a new project."

[0335] Example 2: Automatic generation of quotations

[0336] The user inputs the price information agreed upon with the customer into the system. Here too, the emotion engine analyzes the user's emotions at the time of input.

[0337] The server uses that information to embed a quotation into a template and automatically generates it. The quotation includes the customer name, project name, price, delivery date, etc., and the content is adjusted to a tone that matches the user's mood.

[0338] The server saves the generated quotation as a PDF and automatically sends it to the customer.

[0339] Example 3: Automation of accounting processes

[0340] The server automatically generates sales entries and accounts receivable journal entries based on the generated invoices. During this process, sentiment information is used to prioritize accounting processes and display important notes.

[0341] The server automatically reflects the generated journal entries into the accounting system. Accounting staff are freed from manual entry and can review the entries while referring to notes based on sentiment data from the system.

[0342] The system implemented in this way streamlines the entire process from sales activities to accounting and enables the generation of personalized documents that reflect the user's emotions. As a result, overall business productivity improves, and the quality of customer service also improves.

[0343] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0344] Step 1:

[0345] The user uploads a document.

[0346] Input: Document files created by the user, such as proposals, contracts, and emails.

[0347] Operation: The user selects a document using the sales system's dedicated upload function and clicks the "Upload" button.

[0348] Output: The document file is sent to the server.

[0349] Step 2:

[0350] The server receives and saves the document.

[0351] Input: Uploaded document file.

[0352] Operation: The server receives uploaded documents in real time and saves them to the database. When saving, a unique ID is assigned to each document, and metadata (creation date, customer name, etc.) is recorded.

[0353] Output: Document files and associated metadata stored in the database.

[0354] Step 3:

[0355] The server retrieves user sentiment data.

[0356] Input: User input speed, voice tone, and facial expression data.

[0357] Operation: The emotion engine is activated, and emotion data is acquired by integrating keyboard input speed, voice input tone, and facial expression analysis from the camera.

[0358] Output: Real-time sentiment data.

[0359] Step 4:

[0360] The server converts the document to text.

[0361] Input: A saved document file.

[0362] Operation: Converts PDF and image-format documents into text data using Optical Character Recognition (OCR, e.g., Tesseract OCR).

[0363] Output: Text data of the document.

[0364] Step 5:

[0365] The server extracts important information.

[0366] Input: Text data and sentiment data.

[0367] Operation: Uses natural language processing (NLP, e.g., spaCy, NLTK) to extract important information such as customer name, project name, contract amount, and delivery date. Based on sentiment information, it prioritizes extracting information that the user wants to emphasize or considers particularly important.

[0368] Output: Extracted key information.

[0369] Step 6:

[0370] The server generates a summary.

[0371] Input: Extracted important information.

[0372] Operation: Sends a prompt to a generative artificial intelligence model (e.g., OpenAI GPT-3) to generate a summary. Example prompt: "Generate a summary based on the following information: Customer name is X Corporation, contract amount is 5 million yen, proposed content is the introduction of a new project."

[0373] Output: The generated summary.

[0374] Step 7:

[0375] The server saves the summary.

[0376] Input: Generated summary text.

[0377] Operation: The generated summary is saved to a database and made available for use in subsequent document generation processes.

[0378] Output: Summary text stored in the database.

[0379] Step 8:

[0380] The server automatically generates various documents.

[0381] Input: Summary text and extracted key information.

[0382] Operation: Based on the summary text and extracted information, it automatically generates templates for documents such as quotations, delivery notes, invoices, approval forms, and stamp application forms by embedding the information. It uses sentiment information to adjust the tone and content of the documents to match the user's emotions.

[0383] Output: Various generated documents.

[0384] Step 9:

[0385] The server sends the document.

[0386] Input: Various generated documents.

[0387] Operation: Each generated document is saved in PDF format, and notifications are automatically sent to the relevant parties. For example, a quotation is sent to the customer, and a proposal is sent to the internal approver.

[0388] Output: Sent PDF document and transmission log.

[0389] Step 10:

[0390] The server automatically generates accounting journal entries.

[0391] Input: Generated quotes and invoices.

[0392] Function: Automatically generates accounting journal entries (accounts receivable and sales) from quotations and invoices. Displays accounting processing priority and notes based on sentiment information.

[0393] Output: Generated journal entry information.

[0394] Step 11:

[0395] The server reflects the journal entry information in the accounting system.

[0396] Input: Generated journal entry information.

[0397] Operation: Uses the API of accounting systems (e.g., SAP, QuickBooks) to automatically register and update journal entry information.

[0398] Output: Journal entry information reflected in the accounting system.

[0399] Through the processing steps described above, this system streamlines the entire process from sales to accounting and enables the generation of personalized documents that reflect the user's emotions.

[0400] (Application Example 2)

[0401] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0402] In factory production management, the emotional state of workers often affects production efficiency and work quality, but conventional systems struggle to generate documents and process accounting that reflect emotional information. Furthermore, there is a lack of mechanisms to highlight highly important information in template-based document generation. A new system is needed to solve these problems and improve work efficiency and quality.

[0403] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for receiving documents, database means for storing the received documents, natural language processing means for analyzing the stored documents and extracting important information, generative artificial intelligence means for generating summaries using the extracted information, document generation means for automatically generating quotations, delivery notes, invoices, approval documents, and stamp application forms based on the summarized information, accounting linkage means for generating journal entry information from the generated documents and reflecting it in the accounting system, emotion recognition means for recognizing the emotional state of workers, means for adjusting the tone of documents using generative artificial intelligence based on emotion data, and means for highlighting particularly important information based on the emotional data of workers. This makes it possible to generate production management documents that reflect the emotional state of workers and to efficiently create documents by highlighting particularly important information.

[0404] "Means for receiving documents" refers to methods for electronically uploading various factory work data and reports to a server.

[0405] A "database system" is a system configured to securely store received documents and allow for quick access as needed.

[0406] "Natural language processing means" refers to technologies for analyzing text data from stored documents and extracting important information.

[0407] "Generative artificial intelligence means" refers to artificial intelligence technology for automatically generating summaries based on extracted information.

[0408] A "document generation method" is a method for automatically creating quotations, delivery notes, invoices, approval documents, and seal application forms based on summarized information.

[0409] "Accounting integration means" refers to technology that generates journal entry information from generated documents and automatically reflects it in the accounting system.

[0410] "Emotion recognition means" refers to methods for recognizing the emotional state of workers, and includes technologies such as facial expression analysis and voice analysis.

[0411] "Means for adjusting the tone of a document using generative artificial intelligence based on emotional data" refers to a technology for adjusting the expression and tone of a generated document based on emotional data acquired by an emotion recognition means.

[0412] "Methods for highlighting particularly important information based on worker sentiment data" refers to methods for taking sentiment data into consideration and making particularly important information stand out within the generated document.

[0413] The system for implementing this invention combines functions for centrally managing documents in factory production management, automatically analyzing, summarizing, generating documents, and performing accounting processing, with an emotion engine that recognizes the emotions of workers. The program of this system will now be described.

[0414] System Configuration and Basic Operation

[0415] Information aggregation and emotion recognition

[0416] The server receives documents from terminals used by workers. Workers upload documents such as work orders, reports, and inspection results to the server via smart glasses. The server stores the uploaded documents in a database and assigns a unique ID to each document. When saving, the document's metadata (creation date, project name, etc.) is also recorded.

[0417] The system uses a camera and microphone built into smart glasses as a means of emotion recognition. The server uses OpenCV and TensorFlow to analyze the worker's facial expressions and voice in real time, acquiring emotion data. This emotion data is recorded as metadata when the document is saved.

[0418] Document reading and analysis

[0419] The server sequentially reads documents from the database and converts PDF and image documents into text using an optical character recognition (OCR) engine. For example, Tesseract is used as the OCR engine. The converted text data is then analyzed using a natural language processing (NLP) library (e.g., spaCy, NLTK). This analysis extracts important information such as project name, materials, work procedures, and expected completion date.

[0420] Here, emotion data from emotion recognition tools is used. The server refers to the emotion data and prioritizes extracting information that the worker appears to have paid particular attention to.

[0421] Summary generation

[0422] The server inputs the extracted key information into a generative artificial intelligence (generative AI model) to generate a summary. Examples of such generative AI models include GPT-3 and BERT. The generated summary includes the core points and important details of the work. It also creates a summary text with a tone that reflects sentiment data.

[0423] Automatic document generation

[0424] The server automatically generates templates for work reports, inspection reports, material requirements, etc., by embedding the information based on the generated summary and extracted data. Sentimental information is also used to adjust the tone and content of the documents to match the workers' emotions. Each generated document is saved in PDF format and automatically notified to relevant stakeholders as needed.

[0425] Information sharing with accounting

[0426] The server automatically generates accounting entries from the information in the generated reports and material requirements documents. Specifically, it creates journal entries for material costs and labor costs. These journal entries are automatically reflected in the accounting system, saving accounting staff the trouble of manual data entry.

[0427] Specific example

[0428] Example 1: Starting a new project

[0429] Workers create instructions for new projects using smart glasses and upload them to the system. An emotion engine analyzes the worker's motivation based on their facial expressions and voice. A server receives the instructions and saves them to a database. The server analyzes the content and extracts information such as the project name, work procedures, and materials used.

[0430] Example 2: Automatic generation of work reports

[0431] Workers enter reports into the system upon completion of their tasks. The emotion engine analyzes the workers' emotions. The server automatically generates reports by embedding the report content into a template. The reports include the project name, work details, and time taken. The server saves the generated reports as PDFs and automatically sends them to the project manager.

[0432] Examples of prompts for generative AI models

[0433] "Obtain emotional data from the workers' facial expressions, extract key information for production planning, and generate a report that reflects those emotions."

[0434] This system enables efficient progress management and reporting of factory work, and allows for the generation of documents that reflect the emotional state of workers. This, in turn, increases worker motivation and improves overall work efficiency and quality.

[0435] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0436] Step 1: Receiving the document

[0437] Users upload documents such as work orders and reports to the system using smart glasses or a device. The uploaded documents are sent to the server. Inputs include the electronic file of the document, the user's ID, and the current project. The server receives these documents in real time and stores them in a database. An output notification is generated when the document has been successfully saved.

[0438] Step 2: Acquiring emotional data

[0439] The server uses the smart glasses' camera and microphone to acquire data in real time to recognize the user's emotional state. Input includes video and audio data. This data is analyzed using OpenCV and TensorFlow to recognize emotional states from facial expressions. Emotional data is generated as output and stored in a database as document metadata.

[0440] Step 3: Reading the document

[0441] The server sequentially reads the stored documents from the database. The input includes the electronic files and metadata of the stored documents. The server uses an OCR engine (e.g., Tesseract) to convert PDF and image-based documents into text. The output is the converted text data.

[0442] Step 4: Extracting important information

[0443] The server analyzes the converted text data using a natural language processing (NLP) library (e.g., spaCy, NLTK). Input includes text data and sentiment data. Based on the analysis results, the server extracts important information such as project name, materials, work procedures, and expected completion date. The extracted important information is then generated as output.

[0444] Step 5: Generating a summary

[0445] The server inputs the extracted key information into a generative artificial intelligence (generative AI model) to generate a summary. The input includes the extracted key information and sentiment data. The generative AI model (e.g., GPT-3, BERT) generates a summary based on this information. The output is the summary.

[0446] Step 6: Automatic document generation

[0447] The server embeds various reports into templates based on the generated summary and extracted information. Inputs include the summary, extracted information, and templates. The server utilizes sentiment data to adjust the document's tone and content to match the worker's emotions. The output is the completed report (in PDF format).

[0448] Step 7: Notify relevant parties

[0449] The server automatically sends the completed report to the project manager and other stakeholders. Input includes the completed report as a PDF file and the contact information of the stakeholders. The server sends the report via email or a notification system. Output is a notification confirming successful transmission.

[0450] Step 8: Generating and reflecting accounting information

[0451] The server automatically generates accounting journal entries based on the information in the generated report. Inputs include report data and accounting templates. The server generates journal entries for sales and material costs and automatically reflects them in the accounting system. The generated journal entries are output.

[0452] This series of processes enables the creation of documents that reflect the emotional state of workers in factory production management, and also streamlines accounting processes.

[0453] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0454] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0455] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0456] [Second Embodiment]

[0457] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0458] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0459] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0460] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0461] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0462] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0463] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0464] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0465] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0466] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0467] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0468] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0469] The system for implementing this invention has the functionality to centrally manage various documents used in sales activities and to automatically perform analysis, summarization, document generation, and accounting processing. The specific program processing and its operation are described below in natural language.

[0470] System Configuration and Basic Operation

[0471] Information aggregation

[0472] As a sales representative, the user creates documents such as proposals, contracts, and emails on their device and uploads them to the sales system.

[0473] The server receives uploaded documents in real time and saves all documents to the database. During saving, each document is assigned a unique ID, and its metadata (creation date, customer name, etc.) is recorded.

[0474] Document reading and analysis

[0475] The server sequentially reads documents from the database and performs analysis using natural language processing capabilities. During the analysis process, optical character recognition (OCR) technology is used to convert PDF and image-based documents into text.

[0476] The server performs analysis on the text data to extract important information such as proper nouns, numerical data, and dates. This analysis uses predefined templates and machine learning models.

[0477] Summary generation

[0478] The server inputs key information extracted through natural language processing into a generative artificial intelligence (AI) to generate a summary. This AI then creates a short, concise summary based on the extracted information.

[0479] The server stores the summary in a database so that it can be used in subsequent processing.

[0480] Automatic document generation

[0481] The server automatically generates quotations, delivery notes, invoices, approval forms, and stamping requests by embedding them into templates based on the summary text and extracted information.

[0482] The server saves each generated document in PDF format and automatically notifies the relevant parties.

[0483] Information sharing with accounting

[0484] The server automatically generates the necessary journal entries from the information in the generated quotes and invoices. Specifically, it creates journal entries for accounts receivable and sales based on information such as customer name, transaction date, and amount.

[0485] The server automatically reflects the generated journal entries in the accounting system. This reduces manual data entry by accounting staff and prevents errors.

[0486] Specific example

[0487] Example 1: Proposal for a new project

[0488] Users create proposal documents for new projects on their devices and upload them to the system.

[0489] The server receives the proposal document and saves it to the database. Simultaneously with saving, it analyzes the content and extracts information such as the project name, client name, proposal details, and budget.

[0490] The server passes this information to a generative AI to create a summary. The summary includes the project's key points and important information.

[0491] Example 2: Automatic generation of quotations

[0492] The user enters the price information agreed upon with the customer into the system.

[0493] The server uses that information to embed a quotation into a template and automatically generates it. The quotation will include the customer name, project name, price, delivery date, etc.

[0494] The server saves the generated quotation as a PDF and automatically sends it to the customer.

[0495] Example 3: Automation of accounting processes

[0496] The server automatically generates journal entries for sales and accounts receivable based on the generated invoices. These generated journal entries are then reflected in the accounting system based on the transaction date, amount, and other factors.

[0497] The user, acting as an accounting professional, reviews the journal entries automatically generated within the accounting system and makes corrections or approvals as needed.

[0498] The system implemented in this invention streamlines the entire process from sales to accounting, significantly reducing errors caused by manual work. As a result, it is expected that overall business productivity will improve and the workflow will run more smoothly.

[0499] The following describes the processing flow.

[0500] Step 1:

[0501] Users create documents such as proposals, contracts, and emails on their devices and upload them to the sales system.

[0502] Step 2:

[0503] The server receives uploaded documents in real time and saves them to the database. A unique ID is assigned to each document upon saving.

[0504] Step 3:

[0505] The server sequentially reads documents from the database and uses optical character recognition (OCR) to convert PDF and image-based documents into text.

[0506] Step 4:

[0507] The server performs natural language processing on the converted text data to extract important information such as customer name, project name, contract amount, and delivery date.

[0508] Step 5:

[0509] The server inputs the extracted information into a generative artificial intelligence (AI) to generate a summary. This summary includes the project's key points and important matters.

[0510] Step 6:

[0511] The server saves the generated summary text to a database, making it available for use in the document generation process.

[0512] Step 7:

[0513] The server automatically generates templates for quotations, delivery notes, invoices, approval documents, and stamping requests, by embedding the information based on the summary text and extracted data.

[0514] Step 8:

[0515] The server saves each generated document in PDF format and automatically notifies the relevant parties. For example, a quotation would be sent to the customer, and a proposal would be sent to the internal approver.

[0516] Step 9:

[0517] The server automatically generates accounting journal entries from the information in the generated quotations and invoices. Specifically, it creates journal entries for accounts receivable and sales.

[0518] Step 10:

[0519] The server automatically reflects the generated journal entries into the accounting system. This eliminates the need for manual data entry for accounting staff.

[0520] Step 11:

[0521] The user, acting as an accounting professional, reviews the automatically generated journal entries in the accounting system and makes corrections or approvals as needed.

[0522] In this way, the entire system is integrated, streamlining the entire process from sales to accounting.

[0523] (Example 1)

[0524] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0525] Traditional document management and processing systems for sales activities require the use of multiple independent tools, making it difficult to efficiently manage a series of business processes. Furthermore, manual data entry and document generation are prone to human error, resulting in inefficiencies and inaccuracies. To address these issues, there is a need for a system that centrally manages the entire sales process, automatically handling analysis, summarization, document generation, and accounting.

[0526] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0527] In this invention, the server includes means for receiving various documents related to sales activities in order to aggregate information, database means for storing the received documents, means for extracting important information from the stored documents using natural language processing means, means for generating summaries using generative artificial intelligence means based on the extracted information, document generation means for automatically generating quotations, delivery notes, invoices, approval documents, and stamp application forms based on the summarized information, and accounting linkage means for generating journal entry information based on the generated quotations and invoices and reflecting it in the accounting system. This makes it possible to centrally manage and automate a series of business processes from sales activities to accounting processing.

[0528] "A means of receiving various documents related to sales activities in order to consolidate information" refers to a system that allows users to upload documents such as proposals, contracts, and emails related to sales activities from their terminals to the server.

[0529] A "database system for storing received documents" refers to a system that receives uploaded documents in real time and stores them in a database. The system includes a function to assign a unique ID and metadata to each document during storage.

[0530] "Natural language processing means" refers to technologies that have the function of analyzing text information from stored documents and extracting important information (proper nouns, numerical information, dates, etc.). Typically, this includes the process of converting PDF or image-format documents into text using OCR technology.

[0531] "Generative artificial intelligence methods" refer to artificial intelligence technologies used to generate summary texts based on extracted information. Generally, natural language generation models are used for summarization.

[0532] The "document generation method" is a system that automatically generates various documents such as quotations, delivery notes, invoices, approval forms, and seal application forms by embedding them into templates based on the generated summary text and extracted information.

[0533] The "accounting integration method" is a function that generates necessary journal entries based on the information in the generated quotations and invoices, and automatically reflects this information in the accounting system. Specifically, it generates journal entries for accounts receivable and sales.

[0534] "Optical character recognition means" refers to a technology used to extract text information from PDF and image-format documents, and is known as OCR (Optical Character Recognition).

[0535] "Information embedding based on templates" is an approach that automatically inserts the necessary information and generates a document according to a predefined format.

[0536] The system for implementing this invention centrally manages various documents used in sales activities and automates document analysis, summary generation, document creation, and accounting processing. The specific program processing and operation are described below.

[0537] System-wide configuration

[0538] This system consists of a server, user terminals (PCs, tablets, smartphones), and various software. Specific software used includes database management systems (e.g., MySQL), OCR software (e.g., Tesseract), natural language processing libraries (e.g., SpaCy, NLTK), and machine learning models (e.g., TensorFlow, PyTorch).

[0539] Information aggregation

[0540] Users create documents such as proposals, contracts, and emails related to sales activities on their terminals and upload them to the sales system. The server receives the uploaded documents in real time and saves all documents to the database. When saving, each document is assigned a unique ID and metadata (creation date, customer name, etc.).

[0541] Document reading and analysis

[0542] The server sequentially reads documents from the database and uses OCR software (e.g., Tesseract) to convert PDF and image documents into text. Next, the server uses a natural language processing library (e.g., SpaCy, NLTK) to extract important information such as proper nouns, numerical data, and dates from the text data.

[0543] Summary generation

[0544] The server inputs the extracted key information into a generative AI model (e.g., GPT-3) to generate a summary. The generated summary is then saved back into the database and used for subsequent processing.

[0545] Automatic document generation

[0546] The server automatically generates quotations, delivery notes, invoices, approval forms, and stamping requests by embedding them into templates based on the summary text and extracted information. Each generated document is saved in PDF format and automatically notified to the relevant parties.

[0547] Information sharing with accounting

[0548] The server automatically generates the necessary journal entries from the information in the generated quotes and invoices, and reflects them in the accounting system (e.g., Oracle Financials). This reduces manual data entry by accounting staff and prevents errors.

[0549] Specific example

[0550] Example 1: Proposal for a new project

[0551] 1. The user creates a proposal document for a new project on their device and uploads it to the system.

[0552] 2. The server receives the proposal documents and saves them to the database.

[0553] 3. The server performs OCR processing on the proposal documents and converts them into text data.

[0554] 4. The server extracts information such as "customer name" and "proposal details" from the text data.

[0555] 5. The server passes the extracted information to the generative AI model and generates a summary.

[0556] 6. The server saves the generated summary text.

[0557] Example 2: Automatic generation of quotations

[0558] 1. The user enters the price information agreed upon with the customer into the system.

[0559] 2. The server automatically generates a quotation based on that information.

[0560] 3. The server saves the generated quotation in PDF format and automatically sends it to the customer.

[0561] Example 3: Automation of accounting processes

[0562] 1. The server automatically generates sales entries and accounts receivable journal entries based on the generated invoices.

[0563] 2. The server updates the accounting system with the journal entries it has generated.

[0564] 3. The user, acting as an accounting staff member, reviews the journal entries automatically generated within the accounting system and makes corrections or approvals as necessary.

[0565] Example of a prompt

[0566] "Please upload your proposal for a new project. The system will automatically generate a summary."

[0567] "Please enter the pricing information agreed upon with the customer. A quote will be automatically generated and sent to the customer."

[0568] "The invoice has been generated. Please check the journal entry information that will be automatically reflected in the accounting system."

[0569] In this way, it becomes possible to implement a system in which a series of business processes are executed efficiently and automatically.

[0570] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0571] Step 1:

[0572] Users create documents such as proposals, contracts, and emails related to sales activities on their terminals and upload them to the sales system. These input documents are received by the system in real time. Users also select files from the system's upload screen and click the "Upload" button.

[0573] Step 2:

[0574] The server receives uploaded documents in real time. The received documents are saved to a database (e.g., MySQL). During saving, each document is assigned a unique ID and metadata (creation date, customer name, etc.). The input to this process is the uploaded documents, and the output is the documents stored in the database along with their metadata.

[0575] Step 3:

[0576] The server sequentially reads documents stored in the database. These read documents are converted into text using OCR software (e.g., Tesseract). The converted text data becomes the input for the next process, and the PDF or image-formatted documents are output as text data.

[0577] Step 4:

[0578] The server analyzes text data using a natural language processing library (e.g., SpaCy, NLTK) to extract important information such as proper nouns, numerical data, and dates. The input to this process is text data, and the output is the extracted important information. Specifically, it extracts information such as "customer name," "contract amount," and "due date" from documents.

[0579] Step 5:

[0580] The server inputs the extracted key information into a generative AI model (e.g., GPT-3) to generate a summary. The generated summary is then stored in the database and used in the next process. The input is the key information, and the output is the generated summary.

[0581] Step 6:

[0582] The server automatically generates quotations, delivery notes, invoices, approval forms, and stamping requests by embedding them into templates based on the summary text and extracted information. The input for this process is the summary text and extracted information, and the output is the generated PDF documents. The documents are automatically notified to the relevant parties.

[0583] Step 7:

[0584] The server automatically generates the necessary journal entries based on the information from the generated quotes and invoices, and reflects them in the accounting system (e.g., Oracle Financials). This operation generates journal entries for accounts receivable and sales based on customer name, transaction date, amount, etc. The input is the information from the quotes and invoices, and the output is the generated journal entries.

[0585] Step 8:

[0586] The user, acting as an accounting professional, reviews the automatically generated journal entries within the accounting system and makes corrections or approvals as needed. The input is the generated journal entry information, and the output is the final approved journal entry information. This operation is expected to ensure the smooth progress of the overall business process.

[0587] (Application Example 1)

[0588] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0589] In modern manufacturing, a large volume of documents are handled on the production line, and managing and processing them requires considerable effort. Furthermore, manual document management is prone to human error and delays in verification, significantly reducing operational efficiency. In particular, critical documents such as manufacturing instructions, inspection records, and delivery slips require real-time verification and management, but this is not adequately achieved currently. To solve this problem and improve operational efficiency within the factory, a new automated document management system is needed.

[0590] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0591] In this invention, the server includes means for receiving documents, database means for storing the received documents, and natural language processing means for analyzing the stored documents and extracting important information. This makes it possible to efficiently manage stored documents and automatically extract and process the necessary information.

[0592] "Means for receiving documents" refers to a function that allows users to upload documents such as proposals, contracts, and emails to the server via the internet.

[0593] A "database system" is a system for organizing and storing received documents and managing them in a format that can be used for searching and analysis.

[0594] "Natural language processing techniques" are technologies used to analyze the content of stored documents and extract important information such as proper nouns, numerical data, and dates.

[0595] "Generative artificial intelligence methods" refer to technologies that utilize generative AI models to create short, concise summaries based on information extracted through natural language processing.

[0596] A "document generation method" is a system that automatically creates documents such as quotations, delivery notes, invoices, approval documents, and stamp application forms based on summarized and extracted information.

[0597] "Accounting integration means" refers to technology that generates necessary journal entry information based on automatically generated quotation and invoice information, and reflects that information in the accounting system.

[0598] "Optical character recognition means" refers to a technology for analyzing PDF and image-format documents and extracting text information.

[0599] The "wearable device integration method" is a system for displaying extracted and summarized information in real time on wearable devices such as smart glasses.

[0600] The system for implementing this invention is designed to improve document management and operational efficiency within a factory. Specifically, it utilizes technologies such as natural language processing, generative artificial intelligence, optical character recognition, and wearable device integration. The detailed configuration and operation of this system are described below.

[0601] System Configuration and Basic Operation

[0602] Information aggregation

[0603] Users create documents such as manufacturing instructions, inspection records, and delivery slips on their terminals and upload them to the system.

[0604] The server receives uploaded documents and saves all of them to the database. During saving, each document is assigned a unique ID, and its metadata (creation date, product name, etc.) is recorded.

[0605] Document reading and analysis

[0606] The server sequentially reads documents from the database and uses optical character recognition (OCR) technology to convert PDF and image-based documents into text. Specifically, Tesseract OCR is used as the OCR technology.

[0607] The server performs natural language processing on the text data to extract important information such as proper nouns, numerical data, and dates. This analysis uses predefined templates and machine learning models such as spaCy and BERT.

[0608] Summary generation

[0609] The server inputs key information extracted through natural language processing into a generative artificial intelligence (AI) to generate a summary. GPT-3 is used as the generative AI model.

[0610] The server stores the generated summary in a database so that it can be used in subsequent processing.

[0611] Automated document generation and accounting integration

[0612] The server automatically generates quotations, delivery notes, invoices, etc., by embedding them into templates based on the summary text and extracted information.

[0613] The server saves each generated document in PDF format and automatically notifies the relevant parties.

[0614] The server automatically generates the necessary journal entries from the information in the generated invoices and reflects them in the accounting system.

[0615] Wearable device integration

[0616] Users wear smart glasses to view information from manufacturing instructions and inspection records in real time.

[0617] The server displays extracted and summarized information on smart glasses in real time. This allows users to improve their work efficiency in the field.

[0618] Specific example

[0619] Example 1: Processing instructions for a new production line

[0620] The user creates instructions for a new production line on a terminal and uploads them to the system.

[0621] The server receives the instruction sheet and saves it to the database. Simultaneously with saving, it analyzes the contents and extracts information such as the product name, product code, and work procedure.

[0622] The server passes this information to a generative AI, which then creates a summary. The summary includes the key points of the instructions.

[0623] Example 2: Automatic generation of quotations

[0624] The user enters the price information agreed upon with the customer into the system.

[0625] The server uses that information to embed a quotation into a template and automatically generates it. The quotation will include the customer name, product name, price, delivery date, etc.

[0626] The server saves the generated quotation as a PDF and automatically sends it to the customer.

[0627] Example of a prompt

[0628] Examples of prompt statements to input into a generative AI model are as follows:

[0629] The project name is "Introduction of a New Manufacturing Line," the client is "Company A," the budget is "¥5,000,000," and the deadline is "December 31, 2023." Summarize the key points in three short sentences.

[0630] Using this prompt, GPT-3 generates a summary like this:

[0631] 1. A project to introduce a new manufacturing line.

[0632] 2. The customer is a general company A.

[0633] 3. The budget is ¥5,000,000, and the deadline is December 31, 2023.

[0634] In this way, the present invention can significantly improve document management and operational efficiency within a factory.

[0635] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0636] Step 1:

[0637] Upload document

[0638] Users create documents such as manufacturing instructions, inspection records, and delivery slips on their terminals and upload them to the system. At this time, the document files are read as input.

[0639] The uploaded document is sent to the server.

[0640] Step 2:

[0641] Save document

[0642] The server saves received documents to a database. During saving, a unique ID is assigned to each document, and its metadata (creation date, product name, etc.) is recorded. Input is the uploaded document file, and output is the document stored in the database.

[0643] Specifically, files are stored on a file server, and their metadata is registered in a relational database (such as MySQL).

[0644] Step 3:

[0645] Text conversion using Optical Character Recognition (OCR)

[0646] The server converts stored documents into text using OCR technology (Tesseract OCR). The input is a stored document file, and the output is text data.

[0647] In this step, text information is extracted from images and PDFs and taken out as structured text data.

[0648] Step 4:

[0649] Extracting important information using natural language processing

[0650] The server analyzes the text data obtained by OCR using natural language processing (using models such as spaCy or BERT) and extracts important information such as proper nouns, numerical data, and dates. The input is the text data obtained by OCR, and the output is the extracted important information.

[0651] Specifically, it analyzes text data to perform tasks such as entity identification, relationship extraction, and topic classification.

[0652] Step 5:

[0653] Summary generation

[0654] The server inputs the extracted key information into a generative AI (GPT-3) to generate a summary. An example of a prompt is: "The project name is 'Introduction of a new manufacturing line,' the customer name is 'General Company A,' the budget is '¥5,000,000,' and the deadline is 'December 31, 2023.' Please summarize the key points in three short sentences." The input is the extracted key information, and the output is a summary.

[0655] Send prompts to the generating AI to obtain an appropriate summary.

[0656] Step 6:

[0657] Automatic document generation

[0658] The server automatically generates quotations, delivery notes, invoices, etc., by embedding them into templates based on the generated summary text and extracted information. The input is the summary text and extracted information, and the output is the generated documents.

[0659] By embedding information into a template file, a formally correct document is generated.

[0660] Step 7:

[0661] Integration with accounting systems

[0662] The server automatically generates the necessary journal entries from the information in the generated invoices and reflects them in the accounting system. The input is the generated invoice, and the output is the journal entries reflected in the accounting system.

[0663] Accounts receivable and sales data are automatically registered in the accounting system via the accounting API.

[0664] Step 8:

[0665] Use of wearable devices

[0666] The user wears smart glasses to view information from manufacturing instructions and inspection records in real time. Input is the generated summary and extracted information, and output is the information displayed on the smart glasses.

[0667] It sends data to smart glasses (e.g., Google Glass) to display information with high visibility.

[0668] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0669] The system for implementing this invention combines functions for centrally managing various documents in sales activities, automatically performing analysis, summarization, document generation, and accounting processing, with an emotion engine that recognizes user emotions. The specific program processing and operation are described below in natural language.

[0670] System Configuration and Basic Operation

[0671] Information aggregation and emotion recognition

[0672] As a sales representative, the user creates documents such as proposals, contracts, and emails on their device and uploads them to the sales system.

[0673] The server receives uploaded documents in real time and saves them to the database. Upon saving, each document is assigned a unique ID, and its metadata (creation date, customer name, etc.) is recorded.

[0674] The server activates an emotion engine to recognize the user's emotions and acquire real-time emotion data from the user. For example, it acquires emotion data by integrating factors such as the speed of keyboard input when the user is writing, the tone of voice when using voice input, and facial expression analysis using a camera.

[0675] Document reading and analysis

[0676] The server sequentially reads documents from the database and uses optical character recognition (OCR) to convert PDF and image-based documents into text.

[0677] The server performs natural language processing on the converted text data to extract important information such as customer name, project name, contract amount, and delivery date. Here, sentiment information from the sentiment engine is used to prioritize extracting information that the user wants to emphasize or considers particularly important.

[0678] Summary generation

[0679] The server inputs the extracted key information into a generative artificial intelligence (AI) to generate a summary. This summary includes the project's key points and important matters. Furthermore, based on data from the emotion engine, it creates a summary that reflects the emotions the user wants to emphasize.

[0680] The server saves the generated summary text to a database, making it available for use in the document generation process.

[0681] Automatic document generation

[0682] The server automatically generates templates for quotations, delivery notes, invoices, approval forms, and stamping requests, embedding the information based on the summary and extracted data. Sentimental information is also used to adjust the tone and content of the documents to match the user's emotions.

[0683] The server saves each generated document in PDF format and automatically notifies the relevant parties. For example, a quotation would be sent to the customer, and a proposal would be sent to the internal approver.

[0684] Information sharing with accounting

[0685] The server automatically generates accounting journal entries from the information in the generated quotations and invoices. Specifically, it creates journal entries for accounts receivable and sales.

[0686] The server automatically reflects the generated journal entries into the accounting system. This eliminates the need for manual data entry for accounting staff.

[0687] Specific example

[0688] Example 1: Proposal for a new project

[0689] The user creates a proposal document for a new project on their device and uploads it to the system. The emotion engine analyzes the user's typing speed, facial expressions, and voice to determine how passionate the user is about the project.

[0690] The server receives the proposal documents and saves them to the database, also recording sentiment data. The server then analyzes the content and extracts information such as the project name, client name, proposal details, and budget.

[0691] The server then passes this information and sentiment data to a generative AI to create a summary. The summary includes the core points of the project and information that the user particularly wants to emphasize.

[0692] Example 2: Automatic generation of quotations

[0693] The user enters the price information agreed upon with the customer into the system. Here again, the emotion engine analyzes the user's emotions at the time of input.

[0694] The server uses that information to embed a quotation into a template and automatically generates it. The quotation includes the customer name, project name, price, delivery date, etc., and the content is adjusted to a tone that matches the user's sentiment.

[0695] The server saves the generated quotation as a PDF and automatically sends it to the customer.

[0696] Example 3: Automation of accounting processes

[0697] The server automatically generates sales entries and accounts receivable journal entries based on the generated invoices. During this process, sentiment information is used to prioritize accounting processes and display important notes.

[0698] The server automatically reflects the generated journal entries in the accounting system. Accounting staff are freed from manual entry and can review the entries with reference to notes based on sentiment data from the system.

[0699] The system implementing this invention streamlines the entire process from sales to accounting, and enables the generation of more personalized documents that reflect the user's emotions. As a result, overall business productivity improves, and more effective customer service is achieved.

[0700] The following describes the processing flow.

[0701] Step 1:

[0702] Users, acting as sales representatives, create documents such as proposals, contracts, and emails on their devices and upload them to the sales system.

[0703] Step 2:

[0704] The server receives uploaded documents in real time and saves them to the database. During saving, each document is assigned a unique ID, and metadata (creation date, customer name, etc.) is also recorded.

[0705] Step 3:

[0706] The server activates an emotion engine to acquire emotional data from the user's input speed, clicks, facial expressions, and voice input. This allows for the collection of real-time emotional data from the user.

[0707] Step 4:

[0708] The server sequentially reads documents from the database and uses optical character recognition (OCR) to convert PDF and image-based documents into text.

[0709] Step 5:

[0710] The server performs natural language processing on the converted text data to extract important information such as customer name, project name, contract amount, and delivery date. Here, sentiment data from the sentiment engine is used to prioritize the extraction of information that the user considers particularly important.

[0711] Step 6:

[0712] The server inputs the extracted key information into a generative artificial intelligence (AI) to generate a summary. The summary includes project points and important matters, and is further adjusted to reflect the emotions the user wants to emphasize based on sentiment data.

[0713] Step 7:

[0714] The server saves the generated summary text to a database, making it available for use in the document generation process.

[0715] Step 8:

[0716] The server automatically generates templates for quotations, delivery notes, invoices, approval forms, and stamping requests, embedding the information based on the summary text and extracted data. Sentiment data is also used here to adjust the tone and content of the documents to match the user's emotions.

[0717] Step 9:

[0718] The server saves each generated document in PDF format and automatically notifies the relevant parties. For example, a quotation is sent to the customer, and a proposal is sent to the internal approver.

[0719] Step 10:

[0720] The server automatically generates accounting journal entries from the information in the generated quotations and invoices. Specifically, it creates journal entries for accounts receivable and sales.

[0721] Step 11:

[0722] The server automatically reflects the generated journal entries into the accounting system. This eliminates the need for manual data entry for accounting staff.

[0723] Step 12:

[0724] The user, acting as an accountant, reviews automatically generated journal entries in the accounting system and makes corrections or approvals as needed. Sentimental data is also used here to highlight points that require particular attention.

[0725] In this way, user sentiment data is reflected throughout the entire process, from sales activities to accounting, enabling more effective and personalized responses. This improves operational efficiency and accuracy, and increases overall business productivity.

[0726] (Example 2)

[0727] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0728] Traditional systems struggled to efficiently manage sales activity documents, extract key information, and handle accounting processes consistently through automatically generated documents. Furthermore, generating documents that reflected user sentiment was difficult, resulting in insufficient individual support. This led to cumbersome sales and accounting processes and decreased productivity.

[0729] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0730] In this invention, the server includes means for receiving documents, database means for storing received documents, natural language processing means for analyzing stored documents and extracting important information, generative artificial intelligence means for generating summaries using the extracted information, emotion recognition means for acquiring and analyzing emotion data, means for adjusting the priority of information based on the acquired emotion data, document generation means for automatically generating quotations, delivery notes, invoices, approval documents, and stamp application forms based on the summarized information, accounting linkage means for generating journal entry information from the generated documents and reflecting it in the accounting system, and means for saving the generated various documents in PDF format and automatically notifying relevant parties. This streamlines the entire process from sales to accounting and enables the generation of personalized documents that reflect the user's emotions.

[0731] "Means for receiving documents" refers to the means by which the server receives documents uploaded by the user from their device.

[0732] "Database means for saving received documents" refers to database technology for recording and storing received documents along with a unique ID and metadata.

[0733] "Natural language processing means for analyzing saved documents and extracting important information" refers to natural language processing technology that analyzes text data within saved documents and extracts important information such as customer names, project names, contract amounts, and delivery dates.

[0734] "Generative artificial intelligence means for generating summaries using extracted information" refers to generative artificial intelligence technology for generating summary texts based on extracted important information.

[0735] "An emotion recognition method for acquiring and analyzing emotion data" refers to a technology for acquiring and analyzing a user's real-time emotion data (for example, keyboard input speed, voice tone, facial expressions, etc.).

[0736] "Means for adjusting information priority based on acquired emotional data" refers to technologies that adjust the importance and priority of information based on users' emotional data.

[0737] "Document generation means for automatically generating quotations, delivery notes, invoices, approval documents, and seal application forms based on summarized information" refers to technology for automatically generating various documents (quotations, delivery notes, invoices, approval documents, and seal application forms) based on summarized information.

[0738] "An accounting integration method for generating journal entries from generated documents and reflecting them in an accounting system" refers to an integration technology that automatically generates accounting journal entries from generated documents and reflects them in an accounting system.

[0739] "A means of saving generated documents in PDF format and automatically notifying relevant parties" refers to a means of saving automatically generated documents in PDF format and automatically notifying relevant parties (for example, customers or internal approvers) related to each document.

[0740] The system for implementing this invention has the functionality to centrally manage documents used in sales activities and to automatically perform analysis, summarization, document generation, and accounting processing. By incorporating an emotion engine that recognizes user emotions, this system enables more personalized document generation. The following describes a specific embodiment of this system.

[0741] System Configuration and Basic Operation

[0742] Information aggregation and emotion recognition

[0743] As a sales representative, the user creates documents such as proposals, contracts, and emails on their device and uploads them to the sales system. The devices used are PCs and mobile devices.

[0744] The server receives uploaded documents in real time and saves them to the database. Upon saving, each document is assigned a unique ID, and metadata (creation date, customer name, etc.) is recorded.

[0745] The server runs an emotion engine to acquire real-time emotion data from the user. Specifically, it acquires emotion data by integrating the user's keyboard input speed, tone of voice in the case of voice input, and facial expression analysis using a camera. Software used for emotion analysis includes TensorFlow.

[0746] Document reading and analysis

[0747] The server sequentially reads documents from the database and uses optical character recognition (OCR) to convert PDF and image-based documents into text. Specifically, Tesseract OCR is used.

[0748] The server performs natural language processing (NLP) on the converted text data to extract important information such as customer name, project name, contract amount, and delivery date. Sentimental information from an emotion engine is used to prioritize extracting information that the user wants to emphasize or considers particularly important. Examples of NLP engines used include spaCy and NLTK.

[0749] Summary generation

[0750] The server inputs the extracted key information into a generative artificial intelligence model to generate a summary. OpenAI's GPT-3 is used as the generative AI model. This summary includes project points and important details. Furthermore, based on sentiment engine data, it creates a summary that reflects the emotions the user wants to emphasize.

[0751] Example prompt: "Generate a summary based on the following information: Customer name is ABC Corporation, contract amount is 1 million yen, proposal details are xxxx"

[0752] The server saves the generated summary text to a database so that it can be used in subsequent document generation processes.

[0753] Automatic document generation

[0754] The server automatically generates templates for quotations, delivery notes, invoices, approval forms, and stamping requests, embedding the information based on the summary text and extracted data. Sentimental information is also used to adjust the tone and content of the documents to match the user's emotions.

[0755] The server saves each generated document in PDF format and automatically notifies the relevant parties. For example, a quotation would be sent to the customer, and a proposal would be sent to the internal approver.

[0756] Information sharing with accounting

[0757] The server automatically generates accounting journal entries from the information in the generated quotations and invoices. Specifically, it creates journal entries for accounts receivable and sales.

[0758] The server automatically reflects the generated journal entries into the accounting system. This eliminates the need for manual data entry for accounting staff. Specific accounting systems used include SAP and QuickBooks.

[0759] Specific example

[0760] Example 1: Proposal for a new project

[0761] Users create proposal documents for new projects on their devices and upload them to the system. The emotion engine analyzes the user's typing speed, facial expressions, and voice to determine how passionate the user is about the project.

[0762] The server receives the proposal documents and saves them to the database, also recording sentiment data. The server then analyzes the content and extracts information such as the project name, client name, proposal details, and budget.

[0763] The server then passes this information and sentiment data to a generative AI to create a summary. The summary includes the core points of the project and information that the user particularly wants to emphasize.

[0764] Example prompt: "Generate a summary based on the following information: Customer name is X Corporation, contract amount is 5 million yen, proposed content is the introduction of a new project."

[0765] Example 2: Automatic generation of quotations

[0766] The user inputs the price information agreed upon with the customer into the system. Here too, the emotion engine analyzes the user's emotions at the time of input.

[0767] The server uses that information to embed a quotation into a template and automatically generates it. The quotation includes the customer name, project name, price, delivery date, etc., and the content is adjusted to a tone that matches the user's mood.

[0768] The server saves the generated quotation as a PDF and automatically sends it to the customer.

[0769] Example 3: Automation of accounting processes

[0770] The server automatically generates sales entries and accounts receivable journal entries based on the generated invoices. During this process, sentiment information is used to prioritize accounting processes and display important notes.

[0771] The server automatically reflects the generated journal entries into the accounting system. Accounting staff are freed from manual entry and can review the entries while referring to notes based on sentiment data from the system.

[0772] The system implemented in this way streamlines the entire process from sales activities to accounting and enables the generation of personalized documents that reflect the user's emotions. As a result, overall business productivity improves, and the quality of customer service also improves.

[0773] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0774] Step 1:

[0775] The user uploads a document.

[0776] Input: Document files created by the user, such as proposals, contracts, and emails.

[0777] Operation: The user selects a document using the sales system's dedicated upload function and clicks the "Upload" button.

[0778] Output: The document file is sent to the server.

[0779] Step 2:

[0780] The server receives and saves the document.

[0781] Input: Uploaded document file.

[0782] Operation: The server receives uploaded documents in real time and saves them to the database. When saving, a unique ID is assigned to each document, and metadata (creation date, customer name, etc.) is recorded.

[0783] Output: Document files and associated metadata stored in the database.

[0784] Step 3:

[0785] The server retrieves user sentiment data.

[0786] Input: User input speed, voice tone, and facial expression data.

[0787] Operation: The emotion engine is activated, and emotion data is acquired by integrating keyboard input speed, voice input tone, and facial expression analysis from the camera.

[0788] Output: Real-time sentiment data.

[0789] Step 4:

[0790] The server converts the document to text.

[0791] Input: A saved document file.

[0792] Operation: Converts PDF and image-format documents into text data using Optical Character Recognition (OCR, e.g., Tesseract OCR).

[0793] Output: Text data of the document.

[0794] Step 5:

[0795] The server extracts important information.

[0796] Input: Text data and sentiment data.

[0797] Operation: Uses natural language processing (NLP, e.g., spaCy, NLTK) to extract important information such as customer name, project name, contract amount, and delivery date. Based on sentiment information, it prioritizes extracting information that the user wants to emphasize or considers particularly important.

[0798] Output: Extracted key information.

[0799] Step 6:

[0800] The server generates a summary.

[0801] Input: Extracted important information.

[0802] Operation: Sends a prompt to a generative artificial intelligence model (e.g., OpenAI GPT-3) to generate a summary. Example prompt: "Generate a summary based on the following information: Customer name is X Corporation, contract amount is 5 million yen, proposed content is the introduction of a new project."

[0803] Output: The generated summary.

[0804] Step 7:

[0805] The server saves the summary.

[0806] Input: Generated summary text.

[0807] Operation: The generated summary is saved to a database and made available for use in subsequent document generation processes.

[0808] Output: Summary text stored in the database.

[0809] Step 8:

[0810] The server automatically generates various documents.

[0811] Input: Summary text and extracted key information.

[0812] Operation: Based on the summary text and extracted information, it automatically generates templates for documents such as quotations, delivery notes, invoices, approval forms, and stamp application forms by embedding the information. It uses sentiment information to adjust the tone and content of the documents to match the user's emotions.

[0813] Output: Various generated documents.

[0814] Step 9:

[0815] The server sends the document.

[0816] Input: Various generated documents.

[0817] Operation: Each generated document is saved in PDF format, and notifications are automatically sent to the relevant parties. For example, a quotation is sent to the customer, and a proposal is sent to the internal approver.

[0818] Output: Sent PDF document and transmission log.

[0819] Step 10:

[0820] The server automatically generates accounting journal entries.

[0821] Input: Generated quotes and invoices.

[0822] Function: Automatically generates accounting journal entries (accounts receivable and sales) from quotations and invoices. Displays accounting processing priority and notes based on sentiment information.

[0823] Output: Generated journal entry information.

[0824] Step 11:

[0825] The server reflects the journal entry information in the accounting system.

[0826] Input: Generated journal entry information.

[0827] Operation: Uses the API of accounting systems (e.g., SAP, QuickBooks) to automatically register and update journal entry information.

[0828] Output: Journal entry information reflected in the accounting system.

[0829] Through the processing steps described above, this system streamlines the entire process from sales to accounting and enables the generation of personalized documents that reflect the user's emotions.

[0830] (Application Example 2)

[0831] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0832] In factory production management, the emotional state of workers often affects production efficiency and work quality, but conventional systems struggle to generate documents and process accounting that reflect emotional information. Furthermore, there is a lack of mechanisms to highlight highly important information in template-based document generation. A new system is needed to solve these problems and improve work efficiency and quality.

[0833] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for receiving documents, database means for storing the received documents, natural language processing means for analyzing the stored documents and extracting important information, generative artificial intelligence means for generating summaries using the extracted information, document generation means for automatically generating quotations, delivery notes, invoices, approval documents, and stamp application forms based on the summarized information, accounting linkage means for generating journal entry information from the generated documents and reflecting it in the accounting system, emotion recognition means for recognizing the emotional state of workers, means for adjusting the tone of documents using generative artificial intelligence based on emotion data, and means for highlighting particularly important information based on the emotional data of workers. This makes it possible to generate production management documents that reflect the emotional state of workers and to efficiently create documents by highlighting particularly important information.

[0834] "Means for receiving documents" refers to methods for electronically uploading various factory work data and reports to a server.

[0835] A "database system" is a system configured to securely store received documents and allow for quick access as needed.

[0836] "Natural language processing means" refers to technologies for analyzing text data from stored documents and extracting important information.

[0837] "Generative artificial intelligence means" refers to artificial intelligence technology for automatically generating summaries based on extracted information.

[0838] A "document generation method" is a method for automatically creating quotations, delivery notes, invoices, approval documents, and seal application forms based on summarized information.

[0839] "Accounting integration means" refers to technology that generates journal entry information from generated documents and automatically reflects it in the accounting system.

[0840] "Emotion recognition means" refers to methods for recognizing the emotional state of workers, and includes technologies such as facial expression analysis and voice analysis.

[0841] "Means for adjusting the tone of a document using generative artificial intelligence based on emotional data" refers to a technology for adjusting the expression and tone of a generated document based on emotional data acquired by an emotion recognition means.

[0842] "Methods for highlighting particularly important information based on worker sentiment data" refers to methods for taking sentiment data into consideration and making particularly important information stand out within the generated document.

[0843] The system for implementing this invention combines functions for centrally managing documents in factory production management, automatically analyzing, summarizing, generating documents, and performing accounting processing, with an emotion engine that recognizes the emotions of workers. The program of this system will now be described.

[0844] System Configuration and Basic Operation

[0845] Information aggregation and emotion recognition

[0846] The server receives documents from terminals used by workers. Workers upload documents such as work orders, reports, and inspection results to the server via smart glasses. The server stores the uploaded documents in a database and assigns a unique ID to each document. When saving, the document's metadata (creation date, project name, etc.) is also recorded.

[0847] The system uses a camera and microphone built into smart glasses as a means of emotion recognition. The server analyzes the worker's facial expressions and voice in real time using OpenCV and TensorFlow, and acquires emotion data. This emotion data is recorded as metadata when the document is saved.

[0848] Document reading and analysis

[0849] The server sequentially reads documents from the database and converts PDF and image documents into text using an optical character recognition (OCR) engine. For example, Tesseract is used as the OCR engine. The converted text data is then analyzed using a natural language processing (NLP) library (e.g., spaCy, NLTK). This analysis extracts important information such as project name, materials, work procedures, and expected completion date.

[0850] Here, emotion data from emotion recognition tools is used. The server refers to the emotion data and prioritizes extracting information that the worker appears to have paid particular attention to.

[0851] Summary generation

[0852] The server inputs the extracted key information into a generative artificial intelligence (generative AI model) to generate a summary. Examples of such generative AI models include GPT-3 and BERT. The generated summary includes the core points and important details of the work. It also creates a summary text with a tone that reflects sentiment data.

[0853] Automatic document generation

[0854] The server automatically generates templates for work reports, inspection reports, material requirements, etc., by embedding the information based on the generated summary and extracted data. Sentimental information is also used to adjust the tone and content of the documents to match the workers' emotions. Each generated document is saved in PDF format and automatically notified to relevant stakeholders as needed.

[0855] Information sharing with accounting

[0856] The server automatically generates accounting entries from the information in the generated reports and material requirements documents. Specifically, it creates journal entries for material costs and labor costs. These journal entries are automatically reflected in the accounting system, saving accounting staff the trouble of manual data entry.

[0857] Specific example

[0858] Example 1: Starting a new project

[0859] Workers create instructions for new projects using smart glasses and upload them to the system. An emotion engine analyzes the worker's motivation based on their facial expressions and voice. A server receives the instructions and saves them to a database. The server analyzes the content and extracts information such as the project name, work procedures, and materials used.

[0860] Example 2: Automatic generation of work reports

[0861] Workers enter reports into the system upon completion of their tasks. The emotion engine analyzes the workers' emotions. The server automatically generates reports by embedding the report content into a template. The reports include the project name, work details, and time taken. The server saves the generated reports as PDFs and automatically sends them to the project manager.

[0862] Examples of prompts for generative AI models

[0863] "Obtain emotional data from the workers' facial expressions, extract key information for production planning, and generate a report that reflects those emotions."

[0864] This system enables efficient progress management and reporting of factory work, and allows for the generation of documents that reflect the emotional state of workers. This, in turn, increases worker motivation and improves overall work efficiency and quality.

[0865] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0866] Step 1: Receiving the document

[0867] Users upload documents such as work orders and reports to the system using smart glasses or a device. The uploaded documents are sent to the server. Inputs include the electronic file of the document, the user's ID, and the current project. The server receives these documents in real time and stores them in a database. An output notification is generated when the document has been successfully saved.

[0868] Step 2: Acquiring emotional data

[0869] The server uses the smart glasses' camera and microphone to acquire data in real time to recognize the user's emotional state. Input includes video and audio data. This data is analyzed using OpenCV and TensorFlow to recognize emotional states from facial expressions. Emotional data is generated as output and stored in a database as document metadata.

[0870] Step 3: Reading the document

[0871] The server sequentially reads the stored documents from the database. The input includes the electronic files and metadata of the stored documents. The server uses an OCR engine (e.g., Tesseract) to convert PDF and image-based documents into text. The output is the converted text data.

[0872] Step 4: Extracting important information

[0873] The server analyzes the converted text data using a natural language processing (NLP) library (e.g., spaCy, NLTK). Input includes text data and sentiment data. Based on the analysis results, the server extracts important information such as project name, materials, work procedures, and expected completion date. The extracted important information is then generated as output.

[0874] Step 5: Generating a summary

[0875] The server inputs the extracted key information into a generative artificial intelligence (generative AI model) to generate a summary. The input includes the extracted key information and sentiment data. The generative AI model (e.g., GPT-3, BERT) generates a summary based on this information. The output is the summary.

[0876] Step 6: Automatic document generation

[0877] The server embeds various reports into templates based on the generated summary and extracted information. Inputs include the summary, extracted information, and templates. The server utilizes sentiment data to adjust the document's tone and content to match the worker's emotions. The output is the completed report (in PDF format).

[0878] Step 7: Notify relevant parties

[0879] The server automatically sends the completed report to the project manager and other stakeholders. Input includes the completed report as a PDF file and the contact information of the stakeholders. The server sends the report via email or a notification system. Output is a notification confirming successful transmission.

[0880] Step 8: Generating and reflecting accounting information

[0881] The server automatically generates accounting journal entries based on the information in the generated report. Inputs include report data and accounting templates. The server generates journal entries for sales and material costs and automatically reflects them in the accounting system. The generated journal entries are output.

[0882] This series of processes enables the creation of documents that reflect the emotional state of workers in factory production management, and also streamlines accounting processes.

[0883] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0884] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0885] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0886] [Third Embodiment]

[0887] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0888] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0889] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0890] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0891] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0892] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0893] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0894] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0895] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0896] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0897] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0898] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0899] The system for implementing this invention has the functionality to centrally manage various documents used in sales activities and to automatically perform analysis, summarization, document generation, and accounting processing. The specific program processing and its operation are described below in natural language.

[0900] System Configuration and Basic Operation

[0901] Information aggregation

[0902] As a sales representative, the user creates documents such as proposals, contracts, and emails on their device and uploads them to the sales system.

[0903] The server receives uploaded documents in real time and saves all documents to the database. During saving, each document is assigned a unique ID, and its metadata (creation date, customer name, etc.) is recorded.

[0904] Document reading and analysis

[0905] The server sequentially reads documents from the database and performs analysis using natural language processing capabilities. During the analysis process, optical character recognition (OCR) technology is used to convert PDF and image-based documents into text.

[0906] The server performs analysis on the text data to extract important information such as proper nouns, numerical data, and dates. This analysis uses predefined templates and machine learning models.

[0907] Summary generation

[0908] The server inputs key information extracted through natural language processing into a generative artificial intelligence (AI) to generate a summary. This AI then creates a short, concise summary based on the extracted information.

[0909] The server stores the summary in a database so that it can be used in subsequent processing.

[0910] Automatic document generation

[0911] The server automatically generates quotations, delivery notes, invoices, approval forms, and stamping requests by embedding them into templates based on the summary text and extracted information.

[0912] The server saves each generated document in PDF format and automatically notifies the relevant parties.

[0913] Information sharing with accounting

[0914] The server automatically generates the necessary journal entries from the information in the generated quotes and invoices. Specifically, it creates journal entries for accounts receivable and sales based on information such as customer name, transaction date, and amount.

[0915] The server automatically reflects the generated journal entries in the accounting system. This reduces manual data entry by accounting staff and prevents errors.

[0916] Specific example

[0917] Example 1: Proposal for a new project

[0918] Users create proposal documents for new projects on their devices and upload them to the system.

[0919] The server receives the proposal document and saves it to the database. Simultaneously with saving, it analyzes the content and extracts information such as the project name, client name, proposal details, and budget.

[0920] The server passes this information to a generative AI to create a summary. The summary includes the project's key points and important information.

[0921] Example 2: Automatic generation of quotations

[0922] The user enters the price information agreed upon with the customer into the system.

[0923] The server uses that information to embed a quotation into a template and automatically generates it. The quotation will include the customer name, project name, price, delivery date, etc.

[0924] The server saves the generated quotation as a PDF and automatically sends it to the customer.

[0925] Example 3: Automation of accounting processes

[0926] The server automatically generates journal entries for sales and accounts receivable based on the generated invoices. These generated journal entries are then reflected in the accounting system based on the transaction date, amount, and other factors.

[0927] The user, acting as an accounting professional, reviews the journal entries automatically generated within the accounting system and makes corrections or approvals as needed.

[0928] The system implemented in this invention streamlines the entire process from sales to accounting, significantly reducing errors caused by manual work. As a result, it is expected that overall business productivity will improve and the workflow will run more smoothly.

[0929] The following describes the processing flow.

[0930] Step 1:

[0931] Users create documents such as proposals, contracts, and emails on their devices and upload them to the sales system.

[0932] Step 2:

[0933] The server receives uploaded documents in real time and saves them to the database. A unique ID is assigned to each document upon saving.

[0934] Step 3:

[0935] The server sequentially reads documents from the database and uses optical character recognition (OCR) to convert PDF and image-based documents into text.

[0936] Step 4:

[0937] The server performs natural language processing on the converted text data to extract important information such as customer name, project name, contract amount, and delivery date.

[0938] Step 5:

[0939] The server inputs the extracted information into a generative artificial intelligence (AI) to generate a summary. This summary includes the project's key points and important matters.

[0940] Step 6:

[0941] The server saves the generated summary text to a database, making it available for use in the document generation process.

[0942] Step 7:

[0943] The server automatically generates templates for quotations, delivery notes, invoices, approval documents, and stamping requests, by embedding the information based on the summary text and extracted data.

[0944] Step 8:

[0945] The server saves each generated document in PDF format and automatically notifies the relevant parties. For example, a quotation would be sent to the customer, and a proposal would be sent to the internal approver.

[0946] Step 9:

[0947] The server automatically generates accounting journal entries from the information in the generated quotations and invoices. Specifically, it creates journal entries for accounts receivable and sales.

[0948] Step 10:

[0949] The server automatically reflects the generated journal entries into the accounting system. This eliminates the need for manual data entry for accounting staff.

[0950] Step 11:

[0951] The user, acting as an accounting professional, reviews the automatically generated journal entries in the accounting system and makes corrections or approvals as needed.

[0952] In this way, the entire system is integrated, streamlining the entire process from sales to accounting.

[0953] (Example 1)

[0954] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0955] Traditional document management and processing systems for sales activities require the use of multiple independent tools, making it difficult to efficiently manage a series of business processes. Furthermore, manual data entry and document generation are prone to human error, resulting in inefficiencies and inaccuracies. To address these issues, there is a need for a system that centrally manages the entire sales process, automatically handling analysis, summarization, document generation, and accounting.

[0956] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0957] In this invention, the server includes means for receiving various documents related to sales activities in order to aggregate information, database means for storing the received documents, means for extracting important information from the stored documents using natural language processing means, means for generating summaries using generative artificial intelligence means based on the extracted information, document generation means for automatically generating quotations, delivery notes, invoices, approval documents, and stamp application forms based on the summarized information, and accounting linkage means for generating journal entry information based on the generated quotations and invoices and reflecting it in the accounting system. This makes it possible to centrally manage and automate a series of business processes from sales activities to accounting processing.

[0958] "A means of receiving various documents related to sales activities in order to consolidate information" refers to a system that allows users to upload documents such as proposals, contracts, and emails related to sales activities from their terminals to the server.

[0959] A "database system for storing received documents" refers to a system that receives uploaded documents in real time and stores them in a database. The system includes a function to assign a unique ID and metadata to each document during storage.

[0960] "Natural language processing means" refers to technologies that have the function of analyzing text information from stored documents and extracting important information (proper nouns, numerical information, dates, etc.). Typically, this includes the process of converting PDF or image-format documents into text using OCR technology.

[0961] "Generative artificial intelligence methods" refer to artificial intelligence technologies used to generate summary texts based on extracted information. Generally, natural language generation models are used for summarization.

[0962] The "document generation method" is a system that automatically generates various documents such as quotations, delivery notes, invoices, approval forms, and seal application forms by embedding them into templates based on the generated summary text and extracted information.

[0963] The "accounting integration method" is a function that generates necessary journal entries based on the information in the generated quotations and invoices, and automatically reflects this information in the accounting system. Specifically, it generates journal entries for accounts receivable and sales.

[0964] "Optical character recognition means" refers to a technology used to extract text information from PDF and image-format documents, and is known as OCR (Optical Character Recognition).

[0965] "Information embedding based on templates" is an approach that automatically inserts the necessary information and generates a document according to a predefined format.

[0966] The system for implementing this invention centrally manages various documents used in sales activities and automates document analysis, summary generation, document creation, and accounting processing. The specific program processing and operation are described below.

[0967] System-wide configuration

[0968] This system consists of a server, user terminals (PCs, tablets, smartphones), and various software. Specific software used includes database management systems (e.g., MySQL), OCR software (e.g., Tesseract), natural language processing libraries (e.g., SpaCy, NLTK), and machine learning models (e.g., TensorFlow, PyTorch).

[0969] Information aggregation

[0970] Users create documents such as proposals, contracts, and emails related to sales activities on their terminals and upload them to the sales system. The server receives the uploaded documents in real time and saves all documents to the database. When saving, each document is assigned a unique ID and metadata (creation date, customer name, etc.).

[0971] Document reading and analysis

[0972] The server sequentially reads documents from the database and uses OCR software (e.g., Tesseract) to convert PDF and image documents into text. Next, the server uses a natural language processing library (e.g., SpaCy, NLTK) to extract important information such as proper nouns, numerical data, and dates from the text data.

[0973] Summary generation

[0974] The server inputs the extracted key information into a generative AI model (e.g., GPT-3) to generate a summary. The generated summary is then saved back into the database and used for subsequent processing.

[0975] Automatic document generation

[0976] The server automatically generates quotations, delivery notes, invoices, approval forms, and stamping requests by embedding them into templates based on the summary text and extracted information. Each generated document is saved in PDF format and automatically notified to the relevant parties.

[0977] Information sharing with accounting

[0978] The server automatically generates the necessary journal entries from the information in the generated quotes and invoices, and reflects them in the accounting system (e.g., Oracle Financials). This reduces manual data entry by accounting staff and prevents errors.

[0979] Specific example

[0980] Example 1: Proposal for a new project

[0981] 1. The user creates a proposal document for a new project on their device and uploads it to the system.

[0982] 2. The server receives the proposal documents and saves them to the database.

[0983] 3. The server performs OCR processing on the proposal documents and converts them into text data.

[0984] 4. The server extracts information such as "customer name" and "proposal details" from the text data.

[0985] 5. The server passes the extracted information to the generative AI model and generates a summary.

[0986] 6. The server saves the generated summary text.

[0987] Example 2: Automatic generation of quotations

[0988] 1. The user enters the price information agreed upon with the customer into the system.

[0989] 2. The server automatically generates a quotation based on that information.

[0990] 3. The server saves the generated quotation in PDF format and automatically sends it to the customer.

[0991] Example 3: Automation of accounting processes

[0992] 1. The server automatically generates sales entries and accounts receivable journal entries based on the generated invoices.

[0993] 2. The server updates the accounting system with the journal entries it has generated.

[0994] 3. The user, acting as an accounting staff member, reviews the journal entries automatically generated within the accounting system and makes corrections or approvals as necessary.

[0995] Example of a prompt

[0996] "Please upload your proposal for a new project. The system will automatically generate a summary."

[0997] "Please enter the pricing information agreed upon with the customer. A quote will be automatically generated and sent to the customer."

[0998] "The invoice has been generated. Please check the journal entry information that will be automatically reflected in the accounting system."

[0999] In this way, it becomes possible to implement a system in which a series of business processes are executed efficiently and automatically.

[1000] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1001] Step 1:

[1002] Users create documents such as proposals, contracts, and emails related to sales activities on their terminals and upload them to the sales system. These input documents are received by the system in real time. Users also select files from the system's upload screen and click the "Upload" button.

[1003] Step 2:

[1004] The server receives uploaded documents in real time. The received documents are saved to a database (e.g., MySQL). During saving, each document is assigned a unique ID and metadata (creation date, customer name, etc.). The input to this process is the uploaded documents, and the output is the documents stored in the database along with their metadata.

[1005] Step 3:

[1006] The server sequentially reads documents stored in the database. These read documents are converted into text using OCR software (e.g., Tesseract). The converted text data becomes the input for the next process, and the PDF or image-formatted documents are output as text data.

[1007] Step 4:

[1008] The server analyzes text data using a natural language processing library (e.g., SpaCy, NLTK) to extract important information such as proper nouns, numerical data, and dates. The input to this process is text data, and the output is the extracted important information. Specifically, it extracts information such as "customer name," "contract amount," and "due date" from documents.

[1009] Step 5:

[1010] The server inputs the extracted key information into a generative AI model (e.g., GPT-3) to generate a summary. The generated summary is then stored in the database and used in the next process. The input is the key information, and the output is the generated summary.

[1011] Step 6:

[1012] The server automatically generates quotations, delivery notes, invoices, approval forms, and stamping requests by embedding them into templates based on the summary text and extracted information. The input for this process is the summary text and extracted information, and the output is the generated PDF documents. The documents are automatically notified to the relevant parties.

[1013] Step 7:

[1014] The server automatically generates the necessary journal entries based on the information from the generated quotes and invoices, and reflects them in the accounting system (e.g., Oracle Financials). This operation generates journal entries for accounts receivable and sales based on customer name, transaction date, amount, etc. The input is the information from the quotes and invoices, and the output is the generated journal entries.

[1015] Step 8:

[1016] The user, acting as an accounting professional, reviews the automatically generated journal entries within the accounting system and makes corrections or approvals as needed. The input is the generated journal entry information, and the output is the final approved journal entry information. This operation is expected to ensure the smooth progress of the overall business process.

[1017] (Application Example 1)

[1018] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1019] In modern manufacturing, a large volume of documents are handled on the production line, and managing and processing them requires considerable effort. Furthermore, manual document management is prone to human error and delays in verification, significantly reducing operational efficiency. In particular, critical documents such as manufacturing instructions, inspection records, and delivery slips require real-time verification and management, but this is not adequately achieved currently. To solve this problem and improve operational efficiency within the factory, a new automated document management system is needed.

[1020] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1021] In this invention, the server includes means for receiving documents, database means for storing the received documents, and natural language processing means for analyzing the stored documents and extracting important information. This makes it possible to efficiently manage stored documents and automatically extract and process the necessary information.

[1022] "Means for receiving documents" refers to a function that allows users to upload documents such as proposals, contracts, and emails to the server via the internet.

[1023] A "database system" is a system for organizing and storing received documents and managing them in a format that can be used for searching and analysis.

[1024] "Natural language processing techniques" are technologies used to analyze the content of stored documents and extract important information such as proper nouns, numerical data, and dates.

[1025] "Generative artificial intelligence methods" refer to technologies that utilize generative AI models to create short, concise summaries based on information extracted through natural language processing.

[1026] A "document generation method" is a system that automatically creates documents such as quotations, delivery notes, invoices, approval documents, and stamp application forms based on summarized and extracted information.

[1027] "Accounting integration means" refers to technology that generates necessary journal entry information based on automatically generated quotation and invoice information, and reflects that information in the accounting system.

[1028] "Optical character recognition means" refers to a technology for analyzing PDF and image-format documents and extracting text information.

[1029] The "wearable device integration method" is a system for displaying extracted and summarized information in real time on wearable devices such as smart glasses.

[1030] The system for implementing this invention is designed to improve document management and operational efficiency within a factory. Specifically, it utilizes technologies such as natural language processing, generative artificial intelligence, optical character recognition, and wearable device integration. The detailed configuration and operation of this system are described below.

[1031] System Configuration and Basic Operation

[1032] Information aggregation

[1033] Users create documents such as manufacturing instructions, inspection records, and delivery slips on their terminals and upload them to the system.

[1034] The server receives uploaded documents and saves all of them to the database. During saving, each document is assigned a unique ID, and its metadata (creation date, product name, etc.) is recorded.

[1035] Document reading and analysis

[1036] The server sequentially reads documents from the database and uses optical character recognition (OCR) technology to convert PDF and image-based documents into text. Specifically, Tesseract OCR is used as the OCR technology.

[1037] The server performs natural language processing on the text data to extract important information such as proper nouns, numerical data, and dates. This analysis uses predefined templates and machine learning models such as spaCy and BERT.

[1038] Summary generation

[1039] The server inputs key information extracted through natural language processing into a generative artificial intelligence (AI) to generate a summary. GPT-3 is used as the generative AI model.

[1040] The server stores the generated summary in a database so that it can be used in subsequent processing.

[1041] Automated document generation and accounting integration

[1042] The server automatically generates quotations, delivery notes, invoices, etc., by embedding them into templates based on the summary text and extracted information.

[1043] The server saves each generated document in PDF format and automatically notifies the relevant parties.

[1044] The server automatically generates the necessary journal entries from the information in the generated invoices and reflects them in the accounting system.

[1045] Wearable device integration

[1046] Users wear smart glasses to view information from manufacturing instructions and inspection records in real time.

[1047] The server displays extracted and summarized information on smart glasses in real time. This allows users to improve their work efficiency in the field.

[1048] Specific example

[1049] Example 1: Processing instructions for a new production line

[1050] The user creates instructions for a new production line on a terminal and uploads them to the system.

[1051] The server receives the instruction sheet and saves it to the database. Simultaneously with saving, it analyzes the contents and extracts information such as the product name, product code, and work procedure.

[1052] The server passes this information to a generative AI, which then creates a summary. The summary includes the key points of the instructions.

[1053] Example 2: Automatic generation of quotations

[1054] The user enters the price information agreed upon with the customer into the system.

[1055] The server uses that information to embed a quotation into a template and automatically generates it. The quotation will include the customer name, product name, price, delivery date, etc.

[1056] The server saves the generated quotation as a PDF and automatically sends it to the customer.

[1057] Example of a prompt

[1058] Examples of prompt statements to input into a generative AI model are as follows:

[1059] The project name is "Introduction of a New Manufacturing Line," the client is "Company A," the budget is "¥5,000,000," and the deadline is "December 31, 2023." Summarize the key points in three short sentences.

[1060] Using this prompt, GPT-3 generates a summary like this:

[1061] 1. A project to introduce a new manufacturing line.

[1062] 2. The customer is a general company A.

[1063] 3. The budget is ¥5,000,000, and the deadline is December 31, 2023.

[1064] In this way, the present invention can significantly improve document management and operational efficiency within a factory.

[1065] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1066] Step 1:

[1067] Upload document

[1068] Users create documents such as manufacturing instructions, inspection records, and delivery slips on their terminals and upload them to the system. At this time, the document files are read as input.

[1069] The uploaded document is sent to the server.

[1070] Step 2:

[1071] Save document

[1072] The server saves received documents to a database. During saving, a unique ID is assigned to each document, and its metadata (creation date, product name, etc.) is recorded. Input is the uploaded document file, and output is the document stored in the database.

[1073] Specifically, files are stored on a file server, and their metadata is registered in a relational database (such as MySQL).

[1074] Step 3:

[1075] Text conversion using Optical Character Recognition (OCR)

[1076] The server converts stored documents into text using OCR technology (Tesseract OCR). The input is a stored document file, and the output is text data.

[1077] In this step, text information is extracted from images and PDFs and taken out as structured text data.

[1078] Step 4:

[1079] Extracting important information using natural language processing

[1080] The server analyzes the text data obtained by OCR using natural language processing (using models such as spaCy or BERT) and extracts important information such as proper nouns, numerical data, and dates. The input is the text data obtained by OCR, and the output is the extracted important information.

[1081] Specifically, it analyzes text data to perform tasks such as entity identification, relationship extraction, and topic classification.

[1082] Step 5:

[1083] Summary generation

[1084] The server inputs the extracted key information into a generative AI (GPT-3) to generate a summary. An example of a prompt is: "The project name is 'Introduction of a new manufacturing line,' the customer name is 'General Company A,' the budget is '¥5,000,000,' and the deadline is 'December 31, 2023.' Please summarize the key points in three short sentences." The input is the extracted key information, and the output is a summary.

[1085] Send prompts to the generating AI to obtain an appropriate summary.

[1086] Step 6:

[1087] Automatic document generation

[1088] The server automatically generates quotations, delivery notes, invoices, etc., by embedding them into templates based on the generated summary text and extracted information. The input is the summary text and extracted information, and the output is the generated documents.

[1089] By embedding information into a template file, a formally correct document is generated.

[1090] Step 7:

[1091] Integration with accounting systems

[1092] The server automatically generates the necessary journal entries from the information in the generated invoices and reflects them in the accounting system. The input is the generated invoice, and the output is the journal entries reflected in the accounting system.

[1093] Accounts receivable and sales data are automatically registered in the accounting system via the accounting API.

[1094] Step 8:

[1095] Use of wearable devices

[1096] The user wears smart glasses to view information from manufacturing instructions and inspection records in real time. Input is the generated summary and extracted information, and output is the information displayed on the smart glasses.

[1097] It sends data to smart glasses (e.g., Google Glass) to display information with high visibility.

[1098] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1099] The system for implementing this invention combines functions for centrally managing various documents in sales activities, automatically performing analysis, summarization, document generation, and accounting processing, with an emotion engine that recognizes user emotions. The specific program processing and operation are described below in natural language.

[1100] System Configuration and Basic Operation

[1101] Information aggregation and emotion recognition

[1102] As a sales representative, the user creates documents such as proposals, contracts, and emails on their device and uploads them to the sales system.

[1103] The server receives uploaded documents in real time and saves them to the database. Upon saving, each document is assigned a unique ID, and its metadata (creation date, customer name, etc.) is recorded.

[1104] The server activates an emotion engine to recognize the user's emotions and acquire real-time emotion data from the user. For example, it acquires emotion data by integrating factors such as the speed of keyboard input when the user is writing, the tone of voice when using voice input, and facial expression analysis using a camera.

[1105] Document reading and analysis

[1106] The server sequentially reads documents from the database and uses optical character recognition (OCR) to convert PDF and image-based documents into text.

[1107] The server performs natural language processing on the converted text data to extract important information such as customer name, project name, contract amount, and delivery date. Here, sentiment information from the sentiment engine is used to prioritize extracting information that the user wants to emphasize or considers particularly important.

[1108] Summary generation

[1109] The server inputs the extracted key information into a generative artificial intelligence (AI) to generate a summary. This summary includes the project's key points and important matters. Furthermore, based on data from the emotion engine, it creates a summary that reflects the emotions the user wants to emphasize.

[1110] The server saves the generated summary text to a database, making it available for use in the document generation process.

[1111] Automatic document generation

[1112] The server automatically generates templates for quotations, delivery notes, invoices, approval forms, and stamping requests, embedding the information based on the summary and extracted data. Sentimental information is also used to adjust the tone and content of the documents to match the user's emotions.

[1113] The server saves each generated document in PDF format and automatically notifies the relevant parties. For example, a quotation would be sent to the customer, and a proposal would be sent to the internal approver.

[1114] Information sharing with accounting

[1115] The server automatically generates accounting journal entries from the information in the generated quotations and invoices. Specifically, it creates journal entries for accounts receivable and sales.

[1116] The server automatically reflects the generated journal entries into the accounting system. This eliminates the need for manual data entry for accounting staff.

[1117] Specific example

[1118] Example 1: Proposal for a new project

[1119] The user creates a proposal document for a new project on their device and uploads it to the system. The emotion engine analyzes the user's typing speed, facial expressions, and voice to determine how passionate the user is about the project.

[1120] The server receives the proposal documents and saves them to the database, also recording sentiment data. The server then analyzes the content and extracts information such as the project name, client name, proposal details, and budget.

[1121] The server then passes this information and sentiment data to a generative AI to create a summary. The summary includes the core points of the project and information that the user particularly wants to emphasize.

[1122] Example 2: Automatic generation of quotations

[1123] The user enters the price information agreed upon with the customer into the system. Here again, the emotion engine analyzes the user's emotions at the time of input.

[1124] The server uses that information to embed a quotation into a template and automatically generates it. The quotation includes the customer name, project name, price, delivery date, etc., and the content is adjusted to a tone that matches the user's sentiment.

[1125] The server saves the generated quotation as a PDF and automatically sends it to the customer.

[1126] Example 3: Automation of accounting processes

[1127] The server automatically generates sales entries and accounts receivable journal entries based on the generated invoices. During this process, sentiment information is used to prioritize accounting processes and display important notes.

[1128] The server automatically reflects the generated journal entries in the accounting system. Accounting staff are freed from manual entry and can review the entries with reference to notes based on sentiment data from the system.

[1129] The system implementing this invention streamlines the entire process from sales to accounting, and enables the generation of more personalized documents that reflect the user's emotions. As a result, overall business productivity improves, and more effective customer service is achieved.

[1130] The following describes the processing flow.

[1131] Step 1:

[1132] Users, acting as sales representatives, create documents such as proposals, contracts, and emails on their devices and upload them to the sales system.

[1133] Step 2:

[1134] The server receives uploaded documents in real time and saves them to the database. During saving, each document is assigned a unique ID, and metadata (creation date, customer name, etc.) is also recorded.

[1135] Step 3:

[1136] The server activates an emotion engine to acquire emotional data from the user's input speed, clicks, facial expressions, and voice input. This allows for the collection of real-time emotional data from the user.

[1137] Step 4:

[1138] The server sequentially reads documents from the database and uses optical character recognition (OCR) to convert PDF and image-based documents into text.

[1139] Step 5:

[1140] The server performs natural language processing on the converted text data to extract important information such as customer name, project name, contract amount, and delivery date. Here, sentiment data from the sentiment engine is used to prioritize the extraction of information that the user considers particularly important.

[1141] Step 6:

[1142] The server inputs the extracted key information into a generative artificial intelligence (AI) to generate a summary. The summary includes project points and important matters, and is further adjusted to reflect the emotions the user wants to emphasize based on sentiment data.

[1143] Step 7:

[1144] The server saves the generated summary text to a database, making it available for use in the document generation process.

[1145] Step 8:

[1146] The server automatically generates templates for quotations, delivery notes, invoices, approval forms, and stamping requests, embedding the information based on the summary text and extracted data. Sentiment data is also used here to adjust the tone and content of the documents to match the user's emotions.

[1147] Step 9:

[1148] The server saves each generated document in PDF format and automatically notifies the relevant parties. For example, a quotation is sent to the customer, and a proposal is sent to the internal approver.

[1149] Step 10:

[1150] The server automatically generates accounting journal entries from the information in the generated quotations and invoices. Specifically, it creates journal entries for accounts receivable and sales.

[1151] Step 11:

[1152] The server automatically reflects the generated journal entries into the accounting system. This eliminates the need for manual data entry for accounting staff.

[1153] Step 12:

[1154] The user, acting as an accountant, reviews automatically generated journal entries in the accounting system and makes corrections or approvals as needed. Sentimental data is also used here to highlight points that require particular attention.

[1155] In this way, user sentiment data is reflected throughout the entire process, from sales activities to accounting, enabling more effective and personalized responses. This improves operational efficiency and accuracy, and increases overall business productivity.

[1156] (Example 2)

[1157] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1158] Traditional systems struggled to efficiently manage sales activity documents, extract key information, and handle accounting processes consistently through automatically generated documents. Furthermore, generating documents that reflected user sentiment was difficult, resulting in insufficient individual support. This led to cumbersome sales and accounting processes and decreased productivity.

[1159] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1160] In this invention, the server includes means for receiving documents, database means for storing received documents, natural language processing means for analyzing stored documents and extracting important information, generative artificial intelligence means for generating summaries using the extracted information, emotion recognition means for acquiring and analyzing emotion data, means for adjusting the priority of information based on the acquired emotion data, document generation means for automatically generating quotations, delivery notes, invoices, approval documents, and stamp application forms based on the summarized information, accounting linkage means for generating journal entry information from the generated documents and reflecting it in the accounting system, and means for saving the generated various documents in PDF format and automatically notifying relevant parties. This streamlines the entire process from sales to accounting and enables the generation of personalized documents that reflect the user's emotions.

[1161] "Means for receiving documents" refers to the means by which the server receives documents uploaded by the user from their device.

[1162] "Database means for saving received documents" refers to database technology for recording and storing received documents along with a unique ID and metadata.

[1163] "Natural language processing means for analyzing saved documents and extracting important information" refers to natural language processing technology that analyzes text data within saved documents and extracts important information such as customer names, project names, contract amounts, and delivery dates.

[1164] "Generative artificial intelligence means for generating summaries using extracted information" refers to generative artificial intelligence technology for generating summary texts based on extracted important information.

[1165] "An emotion recognition method for acquiring and analyzing emotion data" refers to a technology for acquiring and analyzing a user's real-time emotion data (for example, keyboard input speed, voice tone, facial expressions, etc.).

[1166] "Means for adjusting information priority based on acquired sentiment data" refers to technologies that adjust the importance and priority of information based on users' sentiment data.

[1167] "Document generation means for automatically generating quotations, delivery notes, invoices, approval documents, and seal application forms based on summarized information" refers to technology for automatically generating various documents (quotations, delivery notes, invoices, approval documents, and seal application forms) based on summarized information.

[1168] "An accounting integration method for generating journal entries from generated documents and reflecting them in an accounting system" refers to an integration technology that automatically generates accounting journal entries from generated documents and reflects them in an accounting system.

[1169] "A means of saving generated documents in PDF format and automatically notifying relevant parties" refers to a means of saving automatically generated documents in PDF format and automatically notifying relevant parties (for example, customers or internal approvers) related to each document.

[1170] The system for implementing this invention has the functionality to centrally manage documents used in sales activities and to automatically perform analysis, summarization, document generation, and accounting processing. By incorporating an emotion engine that recognizes user emotions, this system enables more personalized document generation. The following describes a specific embodiment of this system.

[1171] System Configuration and Basic Operation

[1172] Information aggregation and emotion recognition

[1173] As a sales representative, the user creates documents such as proposals, contracts, and emails on their device and uploads them to the sales system. The devices used are PCs and mobile devices.

[1174] The server receives uploaded documents in real time and saves them to the database. Upon saving, each document is assigned a unique ID, and metadata (creation date, customer name, etc.) is recorded.

[1175] The server runs an emotion engine to acquire real-time emotion data from the user. Specifically, it acquires emotion data by integrating the user's keyboard input speed, tone of voice in the case of voice input, and facial expression analysis using a camera. Software used for emotion analysis includes TensorFlow.

[1176] Document reading and analysis

[1177] The server sequentially reads documents from the database and uses optical character recognition (OCR) to convert PDF and image-based documents into text. Specifically, Tesseract OCR is used.

[1178] The server performs natural language processing (NLP) on the converted text data to extract important information such as customer name, project name, contract amount, and delivery date. Sentimental information from an emotion engine is used to prioritize extracting information that the user wants to emphasize or considers particularly important. Examples of NLP engines used include spaCy and NLTK.

[1179] Summary generation

[1180] The server inputs the extracted key information into a generative artificial intelligence model to generate a summary. OpenAI's GPT-3 is used as the generative AI model. This summary includes project points and important details. Furthermore, based on sentiment engine data, it creates a summary that reflects the emotions the user wants to emphasize.

[1181] Example prompt: "Generate a summary based on the following information: Customer name is ABC Corporation, contract amount is 1 million yen, proposal details are xxxx"

[1182] The server saves the generated summary text to a database so that it can be used in subsequent document generation processes.

[1183] Automatic document generation

[1184] The server automatically generates templates for quotations, delivery notes, invoices, approval forms, and stamping requests, embedding the information based on the summary text and extracted data. Sentimental information is also used to adjust the tone and content of the documents to match the user's emotions.

[1185] The server saves each generated document in PDF format and automatically notifies the relevant parties. For example, a quotation would be sent to the customer, and a proposal would be sent to the internal approver.

[1186] Information sharing with accounting

[1187] The server automatically generates accounting journal entries from the information in the generated quotations and invoices. Specifically, it creates journal entries for accounts receivable and sales.

[1188] The server automatically reflects the generated journal entries into the accounting system. This eliminates the need for manual data entry for accounting staff. Specific accounting systems used include SAP and QuickBooks.

[1189] Specific example

[1190] Example 1: Proposal for a new project

[1191] Users create proposal documents for new projects on their devices and upload them to the system. The emotion engine analyzes the user's typing speed, facial expressions, and voice to determine how passionate the user is about the project.

[1192] The server receives the proposal documents and saves them to the database, also recording sentiment data. The server then analyzes the content and extracts information such as the project name, client name, proposal details, and budget.

[1193] The server then passes this information and sentiment data to a generative AI to create a summary. The summary includes the core points of the project and information that the user particularly wants to emphasize.

[1194] Example prompt: "Generate a summary based on the following information: Customer name is X Corporation, contract amount is 5 million yen, proposed content is the introduction of a new project."

[1195] Example 2: Automatic generation of quotations

[1196] The user inputs the price information agreed upon with the customer into the system. Here too, the emotion engine analyzes the user's emotions at the time of input.

[1197] The server uses that information to embed a quotation into a template and automatically generates it. The quotation includes the customer name, project name, price, delivery date, etc., and the content is adjusted to a tone that matches the user's mood.

[1198] The server saves the generated quotation as a PDF and automatically sends it to the customer.

[1199] Example 3: Automation of accounting processes

[1200] The server automatically generates sales entries and accounts receivable journal entries based on the generated invoices. During this process, sentiment information is used to prioritize accounting processes and display important notes.

[1201] The server automatically reflects the generated journal entries into the accounting system. Accounting staff are freed from manual entry and can review the entries while referring to notes based on sentiment data from the system.

[1202] The system implemented in this way streamlines the entire process from sales activities to accounting and enables the generation of personalized documents that reflect the user's emotions. As a result, overall business productivity improves, and the quality of customer service also improves.

[1203] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1204] Step 1:

[1205] The user uploads a document.

[1206] Input: Document files created by the user, such as proposals, contracts, and emails.

[1207] Operation: The user selects a document using the sales system's dedicated upload function and clicks the "Upload" button.

[1208] Output: The document file is sent to the server.

[1209] Step 2:

[1210] The server receives and saves the document.

[1211] Input: Uploaded document file.

[1212] Operation: The server receives uploaded documents in real time and saves them to the database. When saving, a unique ID is assigned to each document, and metadata (creation date, customer name, etc.) is recorded.

[1213] Output: Document files and associated metadata stored in the database.

[1214] Step 3:

[1215] The server retrieves user sentiment data.

[1216] Input: User input speed, voice tone, and facial expression data.

[1217] Operation: The emotion engine is activated, and emotion data is acquired by integrating keyboard input speed, voice input tone, and facial expression analysis from the camera.

[1218] Output: Real-time sentiment data.

[1219] Step 4:

[1220] The server converts the document to text.

[1221] Input: A saved document file.

[1222] Operation: Converts PDF and image-format documents into text data using Optical Character Recognition (OCR, e.g., Tesseract OCR).

[1223] Output: Text data of the document.

[1224] Step 5:

[1225] The server extracts important information.

[1226] Input: Text data and sentiment data.

[1227] Operation: Uses natural language processing (NLP, e.g., spaCy, NLTK) to extract important information such as customer name, project name, contract amount, and delivery date. Based on sentiment information, it prioritizes extracting information that the user wants to emphasize or considers particularly important.

[1228] Output: Extracted key information.

[1229] Step 6:

[1230] The server generates a summary.

[1231] Input: Extracted important information.

[1232] Operation: Sends a prompt to a generative artificial intelligence model (e.g., OpenAI GPT-3) to generate a summary. Example prompt: "Generate a summary based on the following information: Customer name is X Corporation, contract amount is 5 million yen, proposed content is the introduction of a new project."

[1233] Output: The generated summary.

[1234] Step 7:

[1235] The server saves the summary.

[1236] Input: Generated summary text.

[1237] Operation: The generated summary is saved to a database and made available for use in subsequent document generation processes.

[1238] Output: Summary text stored in the database.

[1239] Step 8:

[1240] The server automatically generates various documents.

[1241] Input: Summary text and extracted key information.

[1242] Operation: Based on the summary text and extracted information, it automatically generates templates for documents such as quotations, delivery notes, invoices, approval forms, and stamp application forms by embedding the information. It uses sentiment information to adjust the tone and content of the documents to match the user's emotions.

[1243] Output: Various generated documents.

[1244] Step 9:

[1245] The server sends the document.

[1246] Input: Various generated documents.

[1247] Operation: Each generated document is saved in PDF format, and notifications are automatically sent to the relevant parties. For example, a quotation is sent to the customer, and a proposal is sent to the internal approver.

[1248] Output: Sent PDF document and transmission log.

[1249] Step 10:

[1250] The server automatically generates accounting journal entries.

[1251] Input: Generated quotes and invoices.

[1252] Function: Automatically generates accounting journal entries (accounts receivable and sales) from quotations and invoices. Displays accounting processing priority and notes based on sentiment information.

[1253] Output: Generated journal entry information.

[1254] Step 11:

[1255] The server reflects the journal entry information in the accounting system.

[1256] Input: Generated journal entry information.

[1257] Operation: Uses the API of accounting systems (e.g., SAP, QuickBooks) to automatically register and update journal entry information.

[1258] Output: Journal entry information reflected in the accounting system.

[1259] Through the processing steps described above, this system streamlines the entire process from sales to accounting and enables the generation of personalized documents that reflect the user's emotions.

[1260] (Application Example 2)

[1261] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1262] In factory production management, the emotional state of workers often affects production efficiency and work quality, but conventional systems struggle to generate documents and process accounting that reflect emotional information. Furthermore, there is a lack of mechanisms to highlight highly important information in template-based document generation. A new system is needed to solve these problems and improve work efficiency and quality.

[1263] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for receiving documents, database means for storing the received documents, natural language processing means for analyzing the stored documents and extracting important information, generative artificial intelligence means for generating summaries using the extracted information, document generation means for automatically generating quotations, delivery notes, invoices, approval documents, and stamp application forms based on the summarized information, accounting linkage means for generating journal entry information from the generated documents and reflecting it in the accounting system, emotion recognition means for recognizing the emotional state of workers, means for adjusting the tone of documents using generative artificial intelligence based on emotion data, and means for highlighting particularly important information based on the emotional data of workers. This makes it possible to generate production management documents that reflect the emotional state of workers and to efficiently create documents by highlighting particularly important information.

[1264] "Means for receiving documents" refers to methods for electronically uploading various factory work data and reports to a server.

[1265] A "database system" is a system configured to securely store received documents and allow for quick access as needed.

[1266] "Natural language processing means" refers to technologies for analyzing text data from stored documents and extracting important information.

[1267] "Generative artificial intelligence means" refers to artificial intelligence technology for automatically generating summaries based on extracted information.

[1268] A "document generation method" is a method for automatically creating quotations, delivery notes, invoices, approval documents, and seal application forms based on summarized information.

[1269] "Accounting integration means" refers to technology that generates journal entry information from generated documents and automatically reflects it in the accounting system.

[1270] "Emotion recognition means" refers to methods for recognizing the emotional state of workers, and includes technologies such as facial expression analysis and voice analysis.

[1271] "Means for adjusting the tone of a document using generative artificial intelligence based on emotional data" refers to a technology for adjusting the expression and tone of a generated document based on emotional data acquired by an emotion recognition means.

[1272] "Methods for highlighting particularly important information based on worker sentiment data" refers to methods for taking sentiment data into consideration and making particularly important information stand out within the generated document.

[1273] The system for implementing this invention combines functions for centrally managing documents in factory production management, automatically analyzing, summarizing, generating documents, and performing accounting processing, with an emotion engine that recognizes the emotions of workers. The program of this system will now be described.

[1274] System Configuration and Basic Operation

[1275] Information aggregation and emotion recognition

[1276] The server receives documents from terminals used by workers. Workers upload documents such as work orders, reports, and inspection results to the server via smart glasses. The server stores the uploaded documents in a database and assigns a unique ID to each document. When saving, the document's metadata (creation date, project name, etc.) is also recorded.

[1277] The system uses a camera and microphone built into smart glasses as a means of emotion recognition. The server uses OpenCV and TensorFlow to analyze the worker's facial expressions and voice in real time, acquiring emotion data. This emotion data is recorded as metadata when the document is saved.

[1278] Document reading and analysis

[1279] The server sequentially reads documents from the database and converts PDF and image documents into text using an optical character recognition (OCR) engine. For example, Tesseract is used as the OCR engine. The converted text data is then analyzed using a natural language processing (NLP) library (e.g., spaCy, NLTK). This analysis extracts important information such as project name, materials, work procedures, and expected completion date.

[1280] Here, emotion data from emotion recognition tools is used. The server refers to the emotion data and prioritizes extracting information that the worker appears to have paid particular attention to.

[1281] Summary generation

[1282] The server inputs the extracted key information into a generative artificial intelligence (generative AI model) to generate a summary. Examples of such generative AI models include GPT-3 and BERT. The generated summary includes the core points and important details of the work. It also creates a summary text with a tone that reflects sentiment data.

[1283] Automatic document generation

[1284] The server automatically generates templates for work reports, inspection reports, material requirements, etc., by embedding the information based on the generated summary and extracted data. Sentimental information is also used to adjust the tone and content of the documents to match the workers' emotions. Each generated document is saved in PDF format and automatically notified to relevant stakeholders as needed.

[1285] Information sharing with accounting

[1286] The server automatically generates accounting entries from the information in the generated reports and material requirements documents. Specifically, it creates journal entries for material costs and labor costs. These journal entries are automatically reflected in the accounting system, saving accounting staff the trouble of manual data entry.

[1287] Specific example

[1288] Example 1: Starting a new project

[1289] Workers create instructions for new projects using smart glasses and upload them to the system. An emotion engine analyzes the worker's motivation based on their facial expressions and voice. A server receives the instructions and saves them to a database. The server analyzes the content and extracts information such as the project name, work procedures, and materials used.

[1290] Example 2: Automatic generation of work reports

[1291] Workers enter reports into the system upon completion of their tasks. The emotion engine analyzes the workers' emotions. The server automatically generates reports by embedding the report content into a template. The reports include the project name, work details, and time taken. The server saves the generated reports as PDFs and automatically sends them to the project manager.

[1292] Examples of prompts for generative AI models

[1293] "Obtain emotional data from the workers' facial expressions, extract key information for production planning, and generate a report that reflects those emotions."

[1294] This system enables efficient progress management and reporting of factory work, and allows for the generation of documents that reflect the emotional state of workers. This, in turn, increases worker motivation and improves overall work efficiency and quality.

[1295] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1296] Step 1: Receiving the document

[1297] Users upload documents such as work orders and reports to the system using smart glasses or a device. The uploaded documents are sent to the server. Inputs include the electronic file of the document, the user's ID, and the current project. The server receives these documents in real time and stores them in a database. An output notification is generated when the document has been successfully saved.

[1298] Step 2: Acquiring emotional data

[1299] The server uses the smart glasses' camera and microphone to acquire data in real time to recognize the user's emotional state. Input includes video and audio data. This data is analyzed using OpenCV and TensorFlow to recognize emotional states from facial expressions. Emotional data is generated as output and stored in a database as document metadata.

[1300] Step 3: Reading the document

[1301] The server sequentially reads the stored documents from the database. The input includes the electronic files and metadata of the stored documents. The server uses an OCR engine (e.g., Tesseract) to convert PDF and image-based documents into text. The output is the converted text data.

[1302] Step 4: Extracting important information

[1303] The server analyzes the converted text data using a natural language processing (NLP) library (e.g., spaCy, NLTK). Input includes text data and sentiment data. Based on the analysis results, the server extracts important information such as project name, materials, work procedures, and expected completion date. The extracted important information is then generated as output.

[1304] Step 5: Generating a summary

[1305] The server inputs the extracted key information into a generative artificial intelligence (generative AI model) to generate a summary. The input includes the extracted key information and sentiment data. The generative AI model (e.g., GPT-3, BERT) generates a summary based on this information. The output is the summary.

[1306] Step 6: Automatic document generation

[1307] The server embeds various reports into templates based on the generated summary and extracted information. Inputs include the summary, extracted information, and templates. The server utilizes sentiment data to adjust the document's tone and content to match the worker's emotions. The output is the completed report (in PDF format).

[1308] Step 7: Notify relevant parties

[1309] The server automatically sends the completed report to the project manager and other stakeholders. Input includes the completed report as a PDF file and the contact information of the stakeholders. The server sends the report via email or a notification system. Output is a notification confirming successful transmission.

[1310] Step 8: Generating and reflecting accounting information

[1311] The server automatically generates accounting journal entries based on the information in the generated report. Inputs include report data and accounting templates. The server generates journal entries for sales and material costs and automatically reflects them in the accounting system. The generated journal entries are output.

[1312] This series of processes enables the creation of documents that reflect the emotional state of workers in factory production management, and also streamlines accounting processes.

[1313] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1314] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1315] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1316] [Fourth Embodiment]

[1317] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1318] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1319] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1320] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1321] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1322] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1323] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1324] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1325] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1326] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1327] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1328] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1329] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1330] The system for implementing this invention has the functionality to centrally manage various documents used in sales activities and to automatically perform analysis, summarization, document generation, and accounting processing. The specific program processing and its operation are described below in natural language.

[1331] System Configuration and Basic Operation

[1332] Information aggregation

[1333] As a sales representative, the user creates documents such as proposals, contracts, and emails on their device and uploads them to the sales system.

[1334] The server receives uploaded documents in real time and saves all documents to the database. During saving, each document is assigned a unique ID, and its metadata (creation date, customer name, etc.) is recorded.

[1335] Document reading and analysis

[1336] The server sequentially reads documents from the database and performs analysis using natural language processing capabilities. During the analysis process, optical character recognition (OCR) technology is used to convert PDF and image-based documents into text.

[1337] The server performs analysis on the text data to extract important information such as proper nouns, numerical data, and dates. This analysis uses predefined templates and machine learning models.

[1338] Summary generation

[1339] The server inputs key information extracted through natural language processing into a generative artificial intelligence (AI) to generate a summary. This AI then creates a short, concise summary based on the extracted information.

[1340] The server stores the summary in a database so that it can be used in subsequent processing.

[1341] Automatic document generation

[1342] The server automatically generates quotations, delivery notes, invoices, approval forms, and stamping requests by embedding them into templates based on the summary text and extracted information.

[1343] The server saves each generated document in PDF format and automatically notifies the relevant parties.

[1344] Information sharing with accounting

[1345] The server automatically generates the necessary journal entries from the information in the generated quotes and invoices. Specifically, it creates journal entries for accounts receivable and sales based on information such as customer name, transaction date, and amount.

[1346] The server automatically reflects the generated journal entries in the accounting system. This reduces manual data entry by accounting staff and prevents errors.

[1347] Specific example

[1348] Example 1: Proposal for a new project

[1349] Users create proposal documents for new projects on their devices and upload them to the system.

[1350] The server receives the proposal document and saves it to the database. Simultaneously with saving, it analyzes the content and extracts information such as the project name, client name, proposal details, and budget.

[1351] The server passes this information to a generative AI to create a summary. The summary includes the project's key points and important information.

[1352] Example 2: Automatic generation of quotations

[1353] The user enters the price information agreed upon with the customer into the system.

[1354] The server uses that information to embed a quotation into a template and automatically generates it. The quotation will include the customer name, project name, price, delivery date, etc.

[1355] The server saves the generated quotation as a PDF and automatically sends it to the customer.

[1356] Example 3: Automation of accounting processes

[1357] The server automatically generates journal entries for sales and accounts receivable based on the generated invoices. These generated journal entries are then reflected in the accounting system based on the transaction date, amount, and other factors.

[1358] The user, acting as an accounting professional, reviews the journal entries automatically generated within the accounting system and makes corrections or approvals as needed.

[1359] The system implemented in this invention streamlines the entire process from sales to accounting, significantly reducing errors caused by manual work. As a result, it is expected that overall business productivity will improve and the workflow will run more smoothly.

[1360] The following describes the processing flow.

[1361] Step 1:

[1362] Users create documents such as proposals, contracts, and emails on their devices and upload them to the sales system.

[1363] Step 2:

[1364] The server receives uploaded documents in real time and saves them to the database. A unique ID is assigned to each document upon saving.

[1365] Step 3:

[1366] The server sequentially reads documents from the database and uses optical character recognition (OCR) to convert PDF and image-based documents into text.

[1367] Step 4:

[1368] The server performs natural language processing on the converted text data to extract important information such as customer name, project name, contract amount, and delivery date.

[1369] Step 5:

[1370] The server inputs the extracted information into a generative artificial intelligence (AI) to generate a summary. This summary includes the project's key points and important matters.

[1371] Step 6:

[1372] The server saves the generated summary text to a database, making it available for use in the document generation process.

[1373] Step 7:

[1374] The server automatically generates templates for quotations, delivery notes, invoices, approval documents, and stamping requests, by embedding the information based on the summary text and extracted data.

[1375] Step 8:

[1376] The server saves each generated document in PDF format and automatically notifies the relevant parties. For example, a quotation would be sent to the customer, and a proposal would be sent to the internal approver.

[1377] Step 9:

[1378] The server automatically generates accounting journal entries from the information in the generated quotations and invoices. Specifically, it creates journal entries for accounts receivable and sales.

[1379] Step 10:

[1380] The server automatically reflects the generated journal entries into the accounting system. This eliminates the need for manual data entry for accounting staff.

[1381] Step 11:

[1382] The user, acting as an accounting professional, reviews the automatically generated journal entries in the accounting system and makes corrections or approvals as needed.

[1383] In this way, the entire system is integrated, streamlining the entire process from sales to accounting.

[1384] (Example 1)

[1385] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1386] Traditional document management and processing systems for sales activities require the use of multiple independent tools, making it difficult to efficiently manage a series of business processes. Furthermore, manual data entry and document generation are prone to human error, resulting in inefficiencies and inaccuracies. To address these issues, there is a need for a system that centrally manages the entire sales process, automatically handling analysis, summarization, document generation, and accounting.

[1387] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1388] In this invention, the server includes means for receiving various documents related to sales activities in order to aggregate information, database means for storing the received documents, means for extracting important information from the stored documents using natural language processing means, means for generating summaries using generative artificial intelligence means based on the extracted information, document generation means for automatically generating quotations, delivery notes, invoices, approval documents, and stamp application forms based on the summarized information, and accounting linkage means for generating journal entry information based on the generated quotations and invoices and reflecting it in the accounting system. This makes it possible to centrally manage and automate a series of business processes from sales activities to accounting processing.

[1389] "A means of receiving various documents related to sales activities in order to consolidate information" refers to a system that allows users to upload documents such as proposals, contracts, and emails related to sales activities from their terminals to the server.

[1390] A "database system for storing received documents" refers to a system that receives uploaded documents in real time and stores them in a database. The system includes a function to assign a unique ID and metadata to each document during storage.

[1391] "Natural language processing means" refers to technologies that have the function of analyzing text information from stored documents and extracting important information (proper nouns, numerical information, dates, etc.). Typically, this includes the process of converting PDF or image-format documents into text using OCR technology.

[1392] "Generative artificial intelligence methods" refer to artificial intelligence technologies used to generate summary texts based on extracted information. Generally, natural language generation models are used for summarization.

[1393] The "document generation method" is a system that automatically generates various documents such as quotations, delivery notes, invoices, approval forms, and seal application forms by embedding them into templates based on the generated summary text and extracted information.

[1394] The "accounting integration method" is a function that generates necessary journal entries based on the information in the generated quotations and invoices, and automatically reflects this information in the accounting system. Specifically, it generates journal entries for accounts receivable and sales.

[1395] "Optical character recognition means" refers to a technology used to extract text information from PDF and image-format documents, and is known as OCR (Optical Character Recognition).

[1396] "Information embedding based on templates" is an approach that automatically inserts the necessary information and generates a document according to a predefined format.

[1397] The system for implementing this invention centrally manages various documents used in sales activities and automates document analysis, summary generation, document creation, and accounting processing. The specific program processing and operation are described below.

[1398] System-wide configuration

[1399] This system consists of a server, user terminals (PCs, tablets, smartphones), and various software. Specific software used includes database management systems (e.g., MySQL), OCR software (e.g., Tesseract), natural language processing libraries (e.g., SpaCy, NLTK), and machine learning models (e.g., TensorFlow, PyTorch).

[1400] Information aggregation

[1401] Users create documents such as proposals, contracts, and emails related to sales activities on their terminals and upload them to the sales system. The server receives the uploaded documents in real time and saves all documents to the database. When saving, each document is assigned a unique ID and metadata (creation date, customer name, etc.).

[1402] Document reading and analysis

[1403] The server sequentially reads documents from the database and uses OCR software (e.g., Tesseract) to convert PDF and image documents into text. Next, the server uses a natural language processing library (e.g., SpaCy, NLTK) to extract important information such as proper nouns, numerical data, and dates from the text data.

[1404] Summary generation

[1405] The server inputs the extracted key information into a generative AI model (e.g., GPT-3) to generate a summary. The generated summary is then saved back into the database and used for subsequent processing.

[1406] Automatic document generation

[1407] The server automatically generates quotations, delivery notes, invoices, approval forms, and stamping requests by embedding them into templates based on the summary text and extracted information. Each generated document is saved in PDF format and automatically notified to the relevant parties.

[1408] Information sharing with accounting

[1409] The server automatically generates the necessary journal entries from the information in the generated quotes and invoices, and reflects them in the accounting system (e.g., Oracle Financials). This reduces manual data entry by accounting staff and prevents errors.

[1410] Specific example

[1411] Example 1: Proposal for a new project

[1412] 1. The user creates a proposal document for a new project on their device and uploads it to the system.

[1413] 2. The server receives the proposal documents and saves them to the database.

[1414] 3. The server performs OCR processing on the proposal documents and converts them into text data.

[1415] 4. The server extracts information such as "customer name" and "proposal details" from the text data.

[1416] 5. The server passes the extracted information to the generative AI model and generates a summary.

[1417] 6. The server saves the generated summary text.

[1418] Example 2: Automatic generation of quotations

[1419] 1. The user enters the price information agreed upon with the customer into the system.

[1420] 2. The server automatically generates a quotation based on that information.

[1421] 3. The server saves the generated quotation in PDF format and automatically sends it to the customer.

[1422] Example 3: Automation of accounting processes

[1423] 1. The server automatically generates sales entries and accounts receivable journal entries based on the generated invoices.

[1424] 2. The server updates the accounting system with the journal entries it has generated.

[1425] 3. The user, acting as an accounting staff member, reviews the journal entries automatically generated within the accounting system and makes corrections or approvals as necessary.

[1426] Example of a prompt

[1427] "Please upload your proposal for a new project. The system will automatically generate a summary."

[1428] "Please enter the pricing information agreed upon with the customer. A quote will be automatically generated and sent to the customer."

[1429] "The invoice has been generated. Please check the journal entry information that will be automatically reflected in the accounting system."

[1430] In this way, it becomes possible to implement a system in which a series of business processes are executed efficiently and automatically.

[1431] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1432] Step 1:

[1433] Users create documents such as proposals, contracts, and emails related to sales activities on their terminals and upload them to the sales system. These input documents are received by the system in real time. Users also select files from the system's upload screen and click the "Upload" button.

[1434] Step 2:

[1435] The server receives uploaded documents in real time. The received documents are saved to a database (e.g., MySQL). During saving, each document is assigned a unique ID and metadata (creation date, customer name, etc.). The input to this process is the uploaded documents, and the output is the documents stored in the database along with their metadata.

[1436] Step 3:

[1437] The server sequentially reads documents stored in the database. These read documents are converted into text using OCR software (e.g., Tesseract). The converted text data becomes the input for the next process, and the PDF or image-formatted documents are output as text data.

[1438] Step 4:

[1439] The server analyzes text data using a natural language processing library (e.g., SpaCy, NLTK) to extract important information such as proper nouns, numerical data, and dates. The input to this process is text data, and the output is the extracted important information. Specifically, it extracts information such as "customer name," "contract amount," and "due date" from documents.

[1440] Step 5:

[1441] The server inputs the extracted key information into a generative AI model (e.g., GPT-3) to generate a summary. The generated summary is then stored in the database and used in the next process. The input is the key information, and the output is the generated summary.

[1442] Step 6:

[1443] The server automatically generates quotations, delivery notes, invoices, approval forms, and stamping requests by embedding them into templates based on the summary text and extracted information. The input for this process is the summary text and extracted information, and the output is the generated PDF documents. The documents are automatically notified to the relevant parties.

[1444] Step 7:

[1445] The server automatically generates the necessary journal entries based on the information from the generated quotes and invoices, and reflects them in the accounting system (e.g., Oracle Financials). This operation generates journal entries for accounts receivable and sales based on customer name, transaction date, amount, etc. The input is the information from the quotes and invoices, and the output is the generated journal entries.

[1446] Step 8:

[1447] The user, acting as an accounting professional, reviews the automatically generated journal entries within the accounting system and makes corrections or approvals as needed. The input is the generated journal entry information, and the output is the final approved journal entry information. This operation is expected to ensure the smooth progress of the overall business process.

[1448] (Application Example 1)

[1449] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1450] In modern manufacturing, a large volume of documents are handled on the production line, and managing and processing them requires considerable effort. Furthermore, manual document management is prone to human error and delays in verification, significantly reducing operational efficiency. In particular, critical documents such as manufacturing instructions, inspection records, and delivery slips require real-time verification and management, but this is not adequately achieved currently. To solve this problem and improve operational efficiency within the factory, a new automated document management system is needed.

[1451] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1452] In this invention, the server includes means for receiving documents, database means for storing the received documents, and natural language processing means for analyzing the stored documents and extracting important information. This makes it possible to efficiently manage stored documents and automatically extract and process the necessary information.

[1453] "Means for receiving documents" refers to a function that allows users to upload documents such as proposals, contracts, and emails to the server via the internet.

[1454] A "database system" is a system for organizing and storing received documents and managing them in a format that can be used for searching and analysis.

[1455] "Natural language processing techniques" are technologies used to analyze the content of stored documents and extract important information such as proper nouns, numerical data, and dates.

[1456] "Generative artificial intelligence methods" refer to technologies that utilize generative AI models to create short, concise summaries based on information extracted through natural language processing.

[1457] A "document generation method" is a system that automatically creates documents such as quotations, delivery notes, invoices, approval documents, and stamp application forms based on summarized and extracted information.

[1458] "Accounting integration means" refers to technology that generates necessary journal entry information based on automatically generated quotation and invoice information, and reflects that information in the accounting system.

[1459] "Optical character recognition means" refers to a technology for analyzing PDF and image-format documents and extracting text information.

[1460] The "wearable device integration method" is a system for displaying extracted and summarized information in real time on wearable devices such as smart glasses.

[1461] The system for implementing this invention is designed to improve document management and operational efficiency within a factory. Specifically, it utilizes technologies such as natural language processing, generative artificial intelligence, optical character recognition, and wearable device integration. The detailed configuration and operation of this system are described below.

[1462] System Configuration and Basic Operation

[1463] Information aggregation

[1464] Users create documents such as manufacturing instructions, inspection records, and delivery slips on their terminals and upload them to the system.

[1465] The server receives uploaded documents and saves all of them to the database. During saving, each document is assigned a unique ID, and its metadata (creation date, product name, etc.) is recorded.

[1466] Document reading and analysis

[1467] The server sequentially reads documents from the database and uses optical character recognition (OCR) technology to convert PDF and image-based documents into text. Specifically, Tesseract OCR is used as the OCR technology.

[1468] The server performs natural language processing on the text data to extract important information such as proper nouns, numerical data, and dates. This analysis uses predefined templates and machine learning models such as spaCy and BERT.

[1469] Summary generation

[1470] The server inputs key information extracted through natural language processing into a generative artificial intelligence (AI) to generate a summary. GPT-3 is used as the generative AI model.

[1471] The server stores the generated summary in a database so that it can be used in subsequent processing.

[1472] Automated document generation and accounting integration

[1473] The server automatically generates quotations, delivery notes, invoices, etc., by embedding them into templates based on the summary text and extracted information.

[1474] The server saves each generated document in PDF format and automatically notifies the relevant parties.

[1475] The server automatically generates the necessary journal entries from the information in the generated invoices and reflects them in the accounting system.

[1476] Wearable device integration

[1477] Users wear smart glasses to view information from manufacturing instructions and inspection records in real time.

[1478] The server displays extracted and summarized information on smart glasses in real time. This allows users to improve their work efficiency in the field.

[1479] Specific example

[1480] Example 1: Processing instructions for a new production line

[1481] The user creates instructions for a new production line on a terminal and uploads them to the system.

[1482] The server receives the instruction sheet and saves it to the database. Simultaneously with saving, it analyzes the contents and extracts information such as the product name, product code, and work procedure.

[1483] The server passes this information to a generative AI, which then creates a summary. The summary includes the key points of the instructions.

[1484] Example 2: Automatic generation of quotations

[1485] The user enters the price information agreed upon with the customer into the system.

[1486] The server uses that information to embed a quotation into a template and automatically generates it. The quotation will include the customer name, product name, price, delivery date, etc.

[1487] The server saves the generated quotation as a PDF and automatically sends it to the customer.

[1488] Example of a prompt

[1489] Examples of prompt statements to input into a generative AI model are as follows:

[1490] The project name is "Introduction of a New Manufacturing Line," the client is "Company A," the budget is "¥5,000,000," and the deadline is "December 31, 2023." Summarize the key points in three short sentences.

[1491] Using this prompt, GPT-3 generates a summary like this:

[1492] 1. A project to introduce a new manufacturing line.

[1493] 2. The customer is a general company A.

[1494] 3. The budget is ¥5,000,000, and the deadline is December 31, 2023.

[1495] In this way, the present invention can significantly improve document management and operational efficiency within a factory.

[1496] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1497] Step 1:

[1498] Upload document

[1499] Users create documents such as manufacturing instructions, inspection records, and delivery slips on their terminals and upload them to the system. At this time, the document files are read as input.

[1500] The uploaded document is sent to the server.

[1501] Step 2:

[1502] Save document

[1503] The server saves received documents to a database. During saving, a unique ID is assigned to each document, and its metadata (creation date, product name, etc.) is recorded. Input is the uploaded document file, and output is the document stored in the database.

[1504] Specifically, files are stored on a file server, and their metadata is registered in a relational database (such as MySQL).

[1505] Step 3:

[1506] Text conversion using Optical Character Recognition (OCR)

[1507] The server converts stored documents into text using OCR technology (Tesseract OCR). The input is a stored document file, and the output is text data.

[1508] In this step, text information is extracted from images and PDFs and taken out as structured text data.

[1509] Step 4:

[1510] Extracting important information using natural language processing

[1511] The server analyzes the text data obtained by OCR using natural language processing (using models such as spaCy or BERT) and extracts important information such as proper nouns, numerical data, and dates. The input is the text data obtained by OCR, and the output is the extracted important information.

[1512] Specifically, it analyzes text data to perform tasks such as entity identification, relationship extraction, and topic classification.

[1513] Step 5:

[1514] Summary generation

[1515] The server inputs the extracted key information into a generative AI (GPT-3) to generate a summary. An example of a prompt is: "The project name is 'Introduction of a new manufacturing line,' the customer name is 'General Company A,' the budget is '¥5,000,000,' and the deadline is 'December 31, 2023.' Please summarize the key points in three short sentences." The input is the extracted key information, and the output is a summary.

[1516] Send prompts to the generating AI to obtain an appropriate summary.

[1517] Step 6:

[1518] Automatic document generation

[1519] The server automatically generates quotations, delivery notes, invoices, etc., by embedding them into templates based on the generated summary text and extracted information. The input is the summary text and extracted information, and the output is the generated documents.

[1520] By embedding information into a template file, a formally correct document is generated.

[1521] Step 7:

[1522] Integration with accounting systems

[1523] The server automatically generates the necessary journal entries from the information in the generated invoices and reflects them in the accounting system. The input is the generated invoice, and the output is the journal entries reflected in the accounting system.

[1524] Accounts receivable and sales data are automatically registered in the accounting system via the accounting API.

[1525] Step 8:

[1526] Use of wearable devices

[1527] The user wears smart glasses to view information from manufacturing instructions and inspection records in real time. Input is the generated summary and extracted information, and output is the information displayed on the smart glasses.

[1528] It sends data to smart glasses (e.g., Google Glass) to display information with high visibility.

[1529] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1530] The system for implementing this invention combines functions for centrally managing various documents in sales activities, automatically performing analysis, summarization, document generation, and accounting processing, with an emotion engine that recognizes user emotions. The specific program processing and operation are described below in natural language.

[1531] System Configuration and Basic Operation

[1532] Information aggregation and emotion recognition

[1533] As a sales representative, the user creates documents such as proposals, contracts, and emails on their device and uploads them to the sales system.

[1534] The server receives uploaded documents in real time and saves them to the database. Upon saving, each document is assigned a unique ID, and its metadata (creation date, customer name, etc.) is recorded.

[1535] The server activates an emotion engine to recognize the user's emotions and acquire real-time emotion data from the user. For example, it acquires emotion data by integrating factors such as the speed of keyboard input when the user is writing, the tone of voice when using voice input, and facial expression analysis using a camera.

[1536] Document reading and analysis

[1537] The server sequentially reads documents from the database and uses optical character recognition (OCR) to convert PDF and image-based documents into text.

[1538] The server performs natural language processing on the converted text data to extract important information such as customer name, project name, contract amount, and delivery date. Here, sentiment information from the sentiment engine is used to prioritize extracting information that the user wants to emphasize or considers particularly important.

[1539] Summary generation

[1540] The server inputs the extracted key information into a generative artificial intelligence (AI) to generate a summary. This summary includes the project's key points and important matters. Furthermore, based on data from the emotion engine, it creates a summary that reflects the emotions the user wants to emphasize.

[1541] The server saves the generated summary text to a database, making it available for use in the document generation process.

[1542] Automatic document generation

[1543] The server automatically generates templates for quotations, delivery notes, invoices, approval forms, and stamping requests, embedding the information based on the summary and extracted data. Sentimental information is also used to adjust the tone and content of the documents to match the user's emotions.

[1544] The server saves each generated document in PDF format and automatically notifies the relevant parties. For example, a quotation would be sent to the customer, and a proposal would be sent to the internal approver.

[1545] Information sharing with accounting

[1546] The server automatically generates accounting journal entries from the information in the generated quotations and invoices. Specifically, it creates journal entries for accounts receivable and sales.

[1547] The server automatically reflects the generated journal entries into the accounting system. This eliminates the need for manual data entry for accounting staff.

[1548] Specific example

[1549] Example 1: Proposal for a new project

[1550] The user creates a proposal document for a new project on their device and uploads it to the system. The emotion engine analyzes the user's typing speed, facial expressions, and voice to determine how passionate the user is about the project.

[1551] The server receives the proposal documents and saves them to the database, also recording sentiment data. The server then analyzes the content and extracts information such as the project name, client name, proposal details, and budget.

[1552] The server then passes this information and sentiment data to a generative AI to create a summary. The summary includes the core points of the project and information that the user particularly wants to emphasize.

[1553] Example 2: Automatic generation of quotations

[1554] The user enters the price information agreed upon with the customer into the system. Here again, the emotion engine analyzes the user's emotions at the time of input.

[1555] The server uses that information to embed a quotation into a template and automatically generates it. The quotation includes the customer name, project name, price, delivery date, etc., and the content is adjusted to a tone that matches the user's sentiment.

[1556] The server saves the generated quotation as a PDF and automatically sends it to the customer.

[1557] Example 3: Automation of accounting processes

[1558] The server automatically generates sales entries and accounts receivable journal entries based on the generated invoices. During this process, sentiment information is used to prioritize accounting processes and display important notes.

[1559] The server automatically reflects the generated journal entries in the accounting system. Accounting staff are freed from manual entry and can review the entries with reference to notes based on sentiment data from the system.

[1560] The system implementing this invention streamlines the entire process from sales to accounting, and enables the generation of more personalized documents that reflect the user's emotions. As a result, overall business productivity improves, and more effective customer service is achieved.

[1561] The following describes the processing flow.

[1562] Step 1:

[1563] Users, acting as sales representatives, create documents such as proposals, contracts, and emails on their devices and upload them to the sales system.

[1564] Step 2:

[1565] The server receives uploaded documents in real time and saves them to the database. During saving, each document is assigned a unique ID, and metadata (creation date, customer name, etc.) is also recorded.

[1566] Step 3:

[1567] The server activates an emotion engine to acquire emotional data from the user's input speed, clicks, facial expressions, and voice input. This allows for the collection of real-time emotional data from the user.

[1568] Step 4:

[1569] The server sequentially reads documents from the database and uses optical character recognition (OCR) to convert PDF and image-based documents into text.

[1570] Step 5:

[1571] The server performs natural language processing on the converted text data to extract important information such as customer name, project name, contract amount, and delivery date. Here, sentiment data from the sentiment engine is used to prioritize the extraction of information that the user considers particularly important.

[1572] Step 6:

[1573] The server inputs the extracted key information into a generative artificial intelligence (AI) to generate a summary. The summary includes project points and important matters, and is further adjusted to reflect the emotions the user wants to emphasize based on sentiment data.

[1574] Step 7:

[1575] The server saves the generated summary text to a database, making it available for use in the document generation process.

[1576] Step 8:

[1577] The server automatically generates templates for quotations, delivery notes, invoices, approval forms, and stamping requests, embedding the information based on the summary text and extracted data. Sentiment data is also used here to adjust the tone and content of the documents to match the user's emotions.

[1578] Step 9:

[1579] The server saves each generated document in PDF format and automatically notifies the relevant parties. For example, a quotation is sent to the customer, and a proposal is sent to the internal approver.

[1580] Step 10:

[1581] The server automatically generates accounting journal entries from the information in the generated quotations and invoices. Specifically, it creates journal entries for accounts receivable and sales.

[1582] Step 11:

[1583] The server automatically reflects the generated journal entries into the accounting system. This eliminates the need for manual data entry for accounting staff.

[1584] Step 12:

[1585] The user, acting as an accountant, reviews automatically generated journal entries in the accounting system and makes corrections or approvals as needed. Sentimental data is also used here to highlight points that require particular attention.

[1586] In this way, user sentiment data is reflected throughout the entire process, from sales activities to accounting, enabling more effective and personalized responses. This improves operational efficiency and accuracy, and increases overall business productivity.

[1587] (Example 2)

[1588] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1589] Traditional systems struggled to efficiently manage sales activity documents, extract key information, and handle accounting processes consistently through automatically generated documents. Furthermore, generating documents that reflected user sentiment was difficult, resulting in insufficient individual support. This led to cumbersome sales and accounting processes and decreased productivity.

[1590] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1591] In this invention, the server includes means for receiving documents, database means for storing received documents, natural language processing means for analyzing stored documents and extracting important information, generative artificial intelligence means for generating summaries using the extracted information, emotion recognition means for acquiring and analyzing emotion data, means for adjusting the priority of information based on the acquired emotion data, document generation means for automatically generating quotations, delivery notes, invoices, approval documents, and stamp application forms based on the summarized information, accounting linkage means for generating journal entry information from the generated documents and reflecting it in the accounting system, and means for saving the generated various documents in PDF format and automatically notifying relevant parties. This streamlines the entire process from sales to accounting and enables the generation of personalized documents that reflect the user's emotions.

[1592] "Means for receiving documents" refers to the means by which the server receives documents uploaded by the user from their device.

[1593] "Database means for saving received documents" refers to database technology for recording and storing received documents along with a unique ID and metadata.

[1594] "Natural language processing means for analyzing saved documents and extracting important information" refers to natural language processing technology that analyzes text data within saved documents and extracts important information such as customer names, project names, contract amounts, and delivery dates.

[1595] "Generative artificial intelligence means for generating summaries using extracted information" refers to generative artificial intelligence technology for generating summary texts based on extracted important information.

[1596] "An emotion recognition method for acquiring and analyzing emotion data" refers to a technology for acquiring and analyzing a user's real-time emotion data (for example, keyboard input speed, voice tone, facial expressions, etc.).

[1597] "Means for adjusting information priority based on acquired sentiment data" refers to technologies that adjust the importance and priority of information based on users' sentiment data.

[1598] "Document generation means for automatically generating quotations, delivery notes, invoices, approval documents, and seal application forms based on summarized information" refers to technology for automatically generating various documents (quotations, delivery notes, invoices, approval documents, and seal application forms) based on summarized information.

[1599] "An accounting integration method for generating journal entries from generated documents and reflecting them in an accounting system" refers to an integration technology that automatically generates accounting journal entries from generated documents and reflects them in an accounting system.

[1600] "A means of saving generated documents in PDF format and automatically notifying relevant parties" refers to a means of saving automatically generated documents in PDF format and automatically notifying relevant parties (for example, customers or internal approvers) related to each document.

[1601] The system for implementing this invention has the functionality to centrally manage documents used in sales activities and to automatically perform analysis, summarization, document generation, and accounting processing. By incorporating an emotion engine that recognizes user emotions, this system enables more personalized document generation. The following describes a specific embodiment of this system.

[1602] System Configuration and Basic Operation

[1603] Information aggregation and emotion recognition

[1604] As a sales representative, the user creates documents such as proposals, contracts, and emails on their device and uploads them to the sales system. The devices used are PCs and mobile devices.

[1605] The server receives uploaded documents in real time and saves them to the database. Upon saving, each document is assigned a unique ID, and metadata (creation date, customer name, etc.) is recorded.

[1606] The server runs an emotion engine to acquire real-time emotion data from the user. Specifically, it acquires emotion data by integrating the user's keyboard input speed, tone of voice in the case of voice input, and facial expression analysis using a camera. Software used for emotion analysis includes TensorFlow.

[1607] Document reading and analysis

[1608] The server sequentially reads documents from the database and uses optical character recognition (OCR) to convert PDF and image-based documents into text. Specifically, Tesseract OCR is used.

[1609] The server performs natural language processing (NLP) on the converted text data to extract important information such as customer name, project name, contract amount, and delivery date. Sentimental information from an emotion engine is used to prioritize extracting information that the user wants to emphasize or considers particularly important. Examples of NLP engines used include spaCy and NLTK.

[1610] Summary generation

[1611] The server inputs the extracted key information into a generative artificial intelligence model to generate a summary. OpenAI's GPT-3 is used as the generative AI model. This summary includes project points and important details. Furthermore, based on sentiment engine data, it creates a summary that reflects the emotions the user wants to emphasize.

[1612] Example prompt: "Generate a summary based on the following information: Customer name is ABC Corporation, contract amount is 1 million yen, proposal details are xxxx"

[1613] The server saves the generated summary text to a database so that it can be used in subsequent document generation processes.

[1614] Automatic document generation

[1615] The server automatically generates templates for quotations, delivery notes, invoices, approval forms, and stamping requests, embedding the information based on the summary text and extracted data. Sentimental information is also used to adjust the tone and content of the documents to match the user's emotions.

[1616] The server saves each generated document in PDF format and automatically notifies the relevant parties. For example, a quotation would be sent to the customer, and a proposal would be sent to the internal approver.

[1617] Information sharing with accounting

[1618] The server automatically generates accounting journal entries from the information in the generated quotations and invoices. Specifically, it creates journal entries for accounts receivable and sales.

[1619] The server automatically reflects the generated journal entries into the accounting system. This eliminates the need for manual data entry for accounting staff. Specific accounting systems used include SAP and QuickBooks.

[1620] Specific example

[1621] Example 1: Proposal for a new project

[1622] Users create proposal documents for new projects on their devices and upload them to the system. The emotion engine analyzes the user's typing speed, facial expressions, and voice to determine how passionate the user is about the project.

[1623] The server receives the proposal documents and saves them to the database, also recording sentiment data. The server then analyzes the content and extracts information such as the project name, client name, proposal details, and budget.

[1624] The server then passes this information and sentiment data to a generative AI to create a summary. The summary includes the core points of the project and information that the user particularly wants to emphasize.

[1625] Example prompt: "Generate a summary based on the following information: Customer name is X Corporation, contract amount is 5 million yen, proposed content is the introduction of a new project."

[1626] Example 2: Automatic generation of quotations

[1627] The user inputs the price information agreed upon with the customer into the system. Here too, the emotion engine analyzes the user's emotions at the time of input.

[1628] The server uses that information to embed a quotation into a template and automatically generates it. The quotation includes the customer name, project name, price, delivery date, etc., and the content is adjusted to a tone that matches the user's mood.

[1629] The server saves the generated quotation as a PDF and automatically sends it to the customer.

[1630] Example 3: Automation of accounting processes

[1631] The server automatically generates sales entries and accounts receivable journal entries based on the generated invoices. During this process, sentiment information is used to prioritize accounting processes and display important notes.

[1632] The server automatically reflects the generated journal entries into the accounting system. Accounting staff are freed from manual entry and can review the entries while referring to notes based on sentiment data from the system.

[1633] The system implemented in this way streamlines the entire process from sales activities to accounting and enables the generation of personalized documents that reflect the user's emotions. As a result, overall business productivity improves, and the quality of customer service also improves.

[1634] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1635] Step 1:

[1636] The user uploads a document.

[1637] Input: Document files created by the user, such as proposals, contracts, and emails.

[1638] Operation: The user selects a document using the sales system's dedicated upload function and clicks the "Upload" button.

[1639] Output: The document file is sent to the server.

[1640] Step 2:

[1641] The server receives and saves the document.

[1642] Input: Uploaded document file.

[1643] Operation: The server receives uploaded documents in real time and saves them to the database. When saving, a unique ID is assigned to each document, and metadata (creation date, customer name, etc.) is recorded.

[1644] Output: Document files and associated metadata stored in the database.

[1645] Step 3:

[1646] The server retrieves user sentiment data.

[1647] Input: User input speed, voice tone, and facial expression data.

[1648] Operation: The emotion engine is activated, and emotion data is acquired by integrating keyboard input speed, voice input tone, and facial expression analysis from the camera.

[1649] Output: Real-time sentiment data.

[1650] Step 4:

[1651] The server converts the document to text.

[1652] Input: A saved document file.

[1653] Operation: Converts PDF and image-format documents into text data using Optical Character Recognition (OCR, e.g., Tesseract OCR).

[1654] Output: Text data of the document.

[1655] Step 5:

[1656] The server extracts important information.

[1657] Input: Text data and sentiment data.

[1658] Operation: Uses natural language processing (NLP, e.g., spaCy, NLTK) to extract important information such as customer name, project name, contract amount, and delivery date. Based on sentiment information, it prioritizes extracting information that the user wants to emphasize or considers particularly important.

[1659] Output: Extracted key information.

[1660] Step 6:

[1661] The server generates a summary.

[1662] Input: Extracted important information.

[1663] Operation: Sends a prompt to a generative artificial intelligence model (e.g., OpenAI GPT-3) to generate a summary. Example prompt: "Generate a summary based on the following information: Customer name is X Corporation, contract amount is 5 million yen, proposed content is the introduction of a new project."

[1664] Output: The generated summary.

[1665] Step 7:

[1666] The server saves the summary.

[1667] Input: Generated summary text.

[1668] Operation: The generated summary is saved to a database and made available for use in subsequent document generation processes.

[1669] Output: Summary text stored in the database.

[1670] Step 8:

[1671] The server automatically generates various documents.

[1672] Input: Summary text and extracted key information.

[1673] Operation: Based on the summary text and extracted information, it automatically generates templates for documents such as quotations, delivery notes, invoices, approval forms, and stamp application forms by embedding the information. It uses sentiment information to adjust the tone and content of the documents to match the user's emotions.

[1674] Output: Various generated documents.

[1675] Step 9:

[1676] The server sends the document.

[1677] Input: Various generated documents.

[1678] Operation: Each generated document is saved in PDF format, and notifications are automatically sent to the relevant parties. For example, a quotation is sent to the customer, and a proposal is sent to the internal approver.

[1679] Output: Sent PDF document and transmission log.

[1680] Step 10:

[1681] The server automatically generates accounting journal entries.

[1682] Input: Generated quotes and invoices.

[1683] Function: Automatically generates accounting journal entries (accounts receivable and sales) from quotations and invoices. Displays accounting processing priority and notes based on sentiment information.

[1684] Output: Generated journal entry information.

[1685] Step 11:

[1686] The server reflects the journal entry information in the accounting system.

[1687] Input: Generated journal entry information.

[1688] Operation: Uses the API of accounting systems (e.g., SAP, QuickBooks) to automatically register and update journal entry information.

[1689] Output: Journal entry information reflected in the accounting system.

[1690] Through the processing steps described above, this system streamlines the entire process from sales to accounting and enables the generation of personalized documents that reflect the user's emotions.

[1691] (Application Example 2)

[1692] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1693] In factory production management, the emotional state of workers often affects production efficiency and work quality, but conventional systems struggle to generate documents and process accounting that reflect emotional information. Furthermore, there is a lack of mechanisms to highlight highly important information in template-based document generation. A new system is needed to solve these problems and improve work efficiency and quality.

[1694] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for receiving documents, database means for storing the received documents, natural language processing means for analyzing the stored documents and extracting important information, generative artificial intelligence means for generating summaries using the extracted information, document generation means for automatically generating quotations, delivery notes, invoices, approval documents, and stamp application forms based on the summarized information, accounting linkage means for generating journal entry information from the generated documents and reflecting it in the accounting system, emotion recognition means for recognizing the emotional state of workers, means for adjusting the tone of documents using generative artificial intelligence based on emotion data, and means for highlighting particularly important information based on the emotional data of workers. This makes it possible to generate production management documents that reflect the emotional state of workers and to efficiently create documents by highlighting particularly important information.

[1695] "Means for receiving documents" refers to methods for electronically uploading various factory work data and reports to a server.

[1696] A "database system" is a system configured to securely store received documents and allow for quick access as needed.

[1697] "Natural language processing means" refers to technologies for analyzing text data from stored documents and extracting important information.

[1698] "Generative artificial intelligence means" refers to artificial intelligence technology for automatically generating summaries based on extracted information.

[1699] A "document generation method" is a method for automatically creating quotations, delivery notes, invoices, approval documents, and seal application forms based on summarized information.

[1700] "Accounting integration means" refers to technology that generates journal entry information from generated documents and automatically reflects it in the accounting system.

[1701] "Emotion recognition means" refers to methods for recognizing the emotional state of workers, and includes technologies such as facial expression analysis and voice analysis.

[1702] "Means for adjusting the tone of a document using generative artificial intelligence based on emotional data" refers to a technology for adjusting the expression and tone of a generated document based on emotional data acquired by an emotion recognition means.

[1703] "Methods for highlighting particularly important information based on worker sentiment data" refers to methods for taking sentiment data into consideration and making particularly important information stand out within the generated document.

[1704] The system for implementing this invention combines functions for centrally managing documents in factory production management, automatically analyzing, summarizing, generating documents, and performing accounting processing, with an emotion engine that recognizes the emotions of workers. The program of this system will now be described.

[1705] System Configuration and Basic Operation

[1706] Information aggregation and emotion recognition

[1707] The server receives documents from terminals used by workers. Workers upload documents such as work orders, reports, and inspection results to the server via smart glasses. The server stores the uploaded documents in a database and assigns a unique ID to each document. When saving, the document's metadata (creation date, project name, etc.) is also recorded.

[1708] The system uses a camera and microphone built into smart glasses as a means of emotion recognition. The server uses OpenCV and TensorFlow to analyze the worker's facial expressions and voice in real time, acquiring emotion data. This emotion data is recorded as metadata when the document is saved.

[1709] Document reading and analysis

[1710] The server sequentially reads documents from the database and converts PDF and image documents into text using an optical character recognition (OCR) engine. For example, Tesseract is used as the OCR engine. The converted text data is then analyzed using a natural language processing (NLP) library (e.g., spaCy, NLTK). This analysis extracts important information such as project name, materials, work procedures, and expected completion date.

[1711] Here, emotion data from emotion recognition tools is used. The server refers to the emotion data and prioritizes extracting information that the worker appears to have paid particular attention to.

[1712] Summary generation

[1713] The server inputs the extracted key information into a generative artificial intelligence (generative AI model) to generate a summary. Examples of such generative AI models include GPT-3 and BERT. The generated summary includes the core points and important details of the work. It also creates a summary text with a tone that reflects sentiment data.

[1714] Automatic document generation

[1715] The server automatically generates templates for work reports, inspection reports, material requirements, etc., by embedding the information based on the generated summary and extracted data. Sentimental information is also used to adjust the tone and content of the documents to match the workers' emotions. Each generated document is saved in PDF format and automatically notified to relevant stakeholders as needed.

[1716] Information sharing with accounting

[1717] The server automatically generates accounting entries from the information in the generated reports and material requirements documents. Specifically, it creates journal entries for material costs and labor costs. These journal entries are automatically reflected in the accounting system, saving accounting staff the trouble of manual data entry.

[1718] Specific example

[1719] Example 1: Starting a new project

[1720] Workers create instructions for new projects using smart glasses and upload them to the system. An emotion engine analyzes the worker's motivation based on their facial expressions and voice. A server receives the instructions and saves them to a database. The server analyzes the content and extracts information such as the project name, work procedures, and materials used.

[1721] Example 2: Automatic generation of work reports

[1722] Workers enter reports into the system upon completion of their tasks. The emotion engine analyzes the workers' emotions. The server automatically generates reports by embedding the report content into a template. The reports include the project name, work details, and time taken. The server saves the generated reports as PDFs and automatically sends them to the project manager.

[1723] Examples of prompts for generative AI models

[1724] "Obtain emotional data from the workers' facial expressions, extract key information for production planning, and generate a report that reflects those emotions."

[1725] This system enables efficient progress management and reporting of factory work, and allows for the generation of documents that reflect the emotional state of workers. This, in turn, increases worker motivation and improves overall work efficiency and quality.

[1726] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1727] Step 1: Receiving the document

[1728] Users upload documents such as work orders and reports to the system using smart glasses or a device. The uploaded documents are sent to the server. Inputs include the electronic file of the document, the user's ID, and the current project. The server receives these documents in real time and stores them in a database. An output notification is generated when the document has been successfully saved.

[1729] Step 2: Acquiring emotional data

[1730] The server uses the smart glasses' camera and microphone to acquire data in real time to recognize the user's emotional state. Input includes video and audio data. This data is analyzed using OpenCV and TensorFlow to recognize emotional states from facial expressions. Emotional data is generated as output and stored in a database as document metadata.

[1731] Step 3: Reading the document

[1732] The server sequentially reads the stored documents from the database. The input includes the electronic files and metadata of the stored documents. The server uses an OCR engine (e.g., Tesseract) to convert PDF and image-based documents into text. The output is the converted text data.

[1733] Step 4: Extracting important information

[1734] The server analyzes the converted text data using a natural language processing (NLP) library (e.g., spaCy, NLTK). Input includes text data and sentiment data. Based on the analysis results, the server extracts important information such as project name, materials, work procedures, and expected completion date. The extracted important information is then generated as output.

[1735] Step 5: Generating a summary

[1736] The server inputs the extracted key information into a generative artificial intelligence (generative AI model) to generate a summary. The input includes the extracted key information and sentiment data. The generative AI model (e.g., GPT-3, BERT) generates a summary based on this information. The output is the summary.

[1737] Step 6: Automatic document generation

[1738] The server embeds various reports into templates based on the generated summary and extracted information. Inputs include the summary, extracted information, and templates. The server utilizes sentiment data to adjust the document's tone and content to match the worker's emotions. The output is the completed report (in PDF format).

[1739] Step 7: Notify relevant parties

[1740] The server automatically sends the completed report to the project manager and other stakeholders. Input includes the completed report as a PDF file and the contact information of the stakeholders. The server sends the report via email or a notification system. Output is a notification confirming successful transmission.

[1741] Step 8: Generating and reflecting accounting information

[1742] The server automatically generates accounting journal entries based on the information in the generated report. Inputs include report data and accounting templates. The server generates journal entries for sales and material costs and automatically reflects them in the accounting system. The generated journal entries are output.

[1743] This series of processes enables the creation of documents that reflect the emotional state of workers in factory production management, and also streamlines accounting processes.

[1744] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1745] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1746] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[1747] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1748] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles...

Claims

1. Means for receiving documents, A database means for storing received documents, A natural language processing method for analyzing saved documents and extracting important information, A generative artificial intelligence means for generating a summary using extracted information, A document generation method for automatically generating quotations, delivery notes, invoices, approval documents, and seal application forms based on summarized information, An accounting integration method for generating journal entry information from the generated documents and reflecting it in the accounting system, A system that includes this.

2. The system according to claim 1, which analyzes a stored document using optical character recognition means and extracts text information.

3. The system according to claim 1, which generates various documents by embedding information into a template.

Images