system

Abstract

We provide the system. [Solution] A means of collecting internal company data and building a database, A means for training a generative AI model using the aforementioned database, A means of receiving emails, analyzing their content, and classifying them into categories, Means for generating an initial response email based on the aforementioned categories, A means to have the administrator perform a detailed analysis of the inquiry content as needed, A means of automatically generating and sending a final response email based on the administrator's feedback, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In a modern business environment, a large number of email exchanges are required, which causes problems such as employees spending a huge amount of time and being unable to concentrate on their original work. In particular, most emails are repetitive, such as internal consultations, reports, notifications, requirement confirmations in sales activities, and inquiries from customers. In such a situation, there is a strong demand for automating email responses to improve efficiency and productivity.

Means for Solving the Problems

[0005] The present invention solves the above problem with a system that includes means for collecting internal company data and building a database, means for training a generative AI model using the database, means for receiving emails, analyzing the content of the emails and classifying them into categories, means for generating an initial response email based on the categories, means for having an administrator perform a detailed analysis of the inquiry content as needed, and means for automatically generating and sending a final response email based on the administrator's feedback. This reduces the burden on employees to write emails and improves work efficiency.

[0006] "Internal company data" refers to all information generated or collected within a company, such as past email data, inquiry history, business processes, and decision-making criteria.

[0007] A "database" is a system that systematically stores collected company data, making it possible to efficiently search, query, and utilize it.

[0008] A "generative AI model" is an artificial intelligence that has the ability to learn patterns from large amounts of data and automatically generate text based on new data.

[0009] "Email" refers to digital text messages sent and received using communication networks such as the internet.

[0010] A "category" refers to a group of classifications based on the analyzed content of an email, and is divided according to specific characteristics or purposes.

[0011] An "initial response email" is the first reply email that a generative AI model automatically generates in response to an incoming email.

[0012] An "administrator" is a user responsible for operating and supervising the system, and for reviewing or correcting responses generated by the AI ​​model as needed.

[0013] "Feedback" refers to the act of an administrator reviewing the responses generated by an AI model and providing suggestions for corrections or other feedback.

[0014] A "final response email" is the final version of the reply email that is automatically generated to reflect the administrator's feedback.

[0015] A "system" is a collection of hardware and software components that work together to provide specific functions or services. [Brief explanation of the drawing]

[0016] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when combined with an emotion engine. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when combined with an emotion engine.

Mode for Carrying Out the Invention

[0017] Hereinafter, an example of an embodiment of the system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be one arithmetic unit or a combination of a plurality of arithmetic units. Also, the processor may be one type of arithmetic unit or a combination of a plurality of types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), APU (Accelerated Processing Unit), and the like.

[0020] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0021] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0022] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0024] [First Embodiment]

[0025] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0026] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0029] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0032] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0036] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0037] This invention relates to a system for automating internal data processing and email responses. This system functions collaboratively through the interaction of servers, terminals, and users. The program's processing is described below in natural language.

[0038] Database construction and AI model training

[0039] First, the server collects internal company data such as past email data, inquiry history, business processes, and decision criteria, and stores this information in a database. Next, the server uses this database to train a generative AI model. Specifically, by training the AI ​​with email content and appropriate response pairs, it becomes possible to generate natural-sounding responses.

[0040] Receiving and analyzing new emails

[0041] When a terminal receives a new email, it analyzes its content and categorizes it into categories such as "Internal Consultation," "Report / Communication," "Confirmation of Sales Activity Requirements," and "Customer Inquiry." To classify the email, natural language processing technology is used to understand its content and assign the appropriate category.

[0042] Generation of initial response

[0043] Next, the server generates an initial response email based on the category and content of the received email. In this process, it improves the quality of the response by referring to similar cases in the database and using appropriate templates (greeting, body, and closing). For example, if a customer inquires about product specifications, the server retrieves the relevant specification information from the product database and generates a response email.

[0044] Consult with the administrator (if necessary)

[0045] If a response is difficult or requires high accuracy, the terminal notifies the administrator to analyze the inquiry in detail. In this case, the server generates the most appropriate answer candidates and presents them to the user (administrator). The user reviews and modifies the answer candidates and provides feedback to the system.

[0046] Automated email creation and sending

[0047] Finally, the server receives feedback from the administrator and automatically generates a final response email. This email is checked for completeness and then automatically sent through the terminal.

[0048] Specific example: Request to arrange a visit

[0049] For example, if an email arrives from another department requesting to schedule a visit next week, the terminal receives the email and categorizes it as a "scheduling request." Next, the server checks the schedule database and automatically generates an email suggesting several possible dates for the visit. Finally, the terminal sends that email.

[0050] Specific example: Product inquiries from customers

[0051] Furthermore, when a customer sends an email requesting information about the specifications of product X, the terminal receives the email and identifies its category as "customer inquiry." The server retrieves the specifications of product X from the product database and generates an automated response email. If the response is incomplete, the terminal automatically consults with the relevant department to obtain a final answer. The server then creates a final email and sends it to the customer through the terminal.

[0052] By using the methods described above, internal email processing will be automated, significantly improving employee work efficiency.

[0053] The following describes the processing flow.

[0054] Step 1:

[0055] The server collects past email data, inquiry history, business processes, and decision-making criteria from within the company, and stores this information in a database. Specifically, it extracts necessary data from email sending and receiving logs and document management systems, organizes it, and registers it in the database.

[0056] Step 2:

[0057] The server uses information from the database to train a generative AI model (e.g., GPT-3®). This involves providing the AI ​​model with pairs of email content and corresponding replies, and repeatedly training it to improve its ability to generate natural-sounding text.

[0058] Step 3:

[0059] Each time the terminal receives a new email, it analyzes its content. Using natural language processing technology, it extracts keywords and important contextual information from the email body and classifies the emails into categories such as "internal consultation," "report / communication," "confirmation of sales activity requirements," and "customer inquiry."

[0060] Step 4:

[0061] The server generates an initial response email based on the aforementioned categories. It refers to similar cases and templates in the database to construct the most appropriate response. For example, in response to a customer inquiry, it prepares the answer by referring to relevant product information and FAQs.

[0062] Step 5:

[0063] The server provides the terminal with an initial response email. If the initial response is insufficient to resolve the issue, or if a high level of accuracy is required, the terminal will notify the appropriate administrator.

[0064] Step 6:

[0065] The user (administrator) receives a notification and reviews and modifies the suggested answers provided by the server. If necessary, they enter additional information or supplementary explanations. The administrator's feedback is reflected in the system.

[0066] Step 7:

[0067] The server automatically generates a final response email based on feedback from the administrator. It then incorporates any corrections or additional information from the administrator to create a complete response.

[0068] Step 8:

[0069] The terminal checks the final response email, performs error checks and final confirmations, and then automatically sends the email to the intended recipient.

[0070] This series of processing steps automates email processing within the company, significantly reducing the effort and time required from employees.

[0071] (Example 1)

[0072] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0073] This invention relates to a system for automating internal data processing and email responses. Conventional systems suffered from long response times from email reception to response, resulting in poor efficiency. Furthermore, manual response creation was prone to human error, often leading to a lack of consistency and accuracy in response content. This resulted in decreased operational efficiency and reduced customer satisfaction.

[0074] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0075] In this invention, the server includes means for collecting internal company data and building a database, means for training a generative AI model using the database, means for receiving emails, analyzing the content of the emails and classifying them into categories, means for generating an initial response email based on the categories, means for generating a response email by referring to similar cases in the database, means for having an administrator perform a detailed analysis of the inquiry content as needed, and means for automatically generating and sending a final response email based on the administrator's feedback. This makes email processing more efficient and improves the consistency and accuracy of response content.

[0076] A "server" is a computer system that provides data and services to multiple computers and terminals over a network.

[0077] A "database" is a systematically organized collection of data, stored in a way that allows for efficient searching and updating.

[0078] A "generative AI model" is a type of artificial intelligence that learns from large amounts of data and performs tasks such as generating natural language and automatically generating responses.

[0079] "Natural language processing technology" refers to a set of technologies and methods that enable computers to understand, generate, and engage in human dialogue.

[0080] "Email" refers to digital messages sent and received via the internet or other computer networks.

[0081] A "category" is a way of classifying and grouping data based on certain common characteristics or attributes.

[0082] A "template" is a document model with a specific format or pattern, and it is a document format that allows for the easy generation of standardized documents by filling in the content.

[0083] "Feedback" refers to evaluations and corrections provided by users or administrators regarding responses and results generated by a system.

[0084] "IMAP" is an abbreviation for Internet Mail Access Protocol, and it is a protocol for managing email on a server.

[0085] POP3 is version 3 of the postal protocol and is an internet standard protocol used to receive email.

[0086] "MIME format" is an abbreviation for Multipurpose Internet Mail Extensions format, and it is a format that allows emails to contain data other than text (such as images, audio, and video).

[0087] "JSON format" is an abbreviation for JavaScript (registered trademark) Object Notation format, and is a lightweight data description format primarily used for data exchange.

[0088] This invention relates to a system for automating internal data processing and email responses. This system functions collaboratively through the interaction of servers, terminals, and users.

[0089] First, the server collects data such as past email data, inquiry history, business processes, and decision criteria from within the company, and stores this information in a database. Possible email servers to use include Exchange Server and G Suite. The collected data is saved in CSV or JSON format, and database management systems such as MySQL® or MongoDB are used for the database. Based on this database, the server trains a generative AI model (for example, OpenAI®'s GPT-3). Email content and appropriate response pairs are used as training data.

[0090] Next, the device receives new emails. Protocols such as IMAP and POP3 are used to receive emails. The content of the received emails is analyzed using natural language processing technology (e.g., Google® Cloud Natural Language API), and the emails are categorized into "internal consultations," "reports and communications," "sales activity requirements confirmation," and "customer inquiries."

[0091] Based on the aforementioned category and email content, the server refers to similar cases in the database and generates an initial response email. The generated response email includes a greeting, body, and closing. For example, in response to an inquiry such as "Please tell me about the specifications of product X," the server retrieves the relevant specification information from the product database and generates a response. This response email is constructed in MIME format.

[0092] If a response is difficult or requires high accuracy, the terminal notifies the administrator for detailed analysis. The administrator is notified using tools such as Jira or Slack, and the server generates optimal response candidates and presents them to the user (administrator). The administrator reviews these response candidates, makes corrections as needed, and sends feedback back to the system in JSON format.

[0093] Based on this feedback, the server automatically generates a final response email. The generated email is then automatically sent through the device. Email sending APIs such as SendGrid and AWS® SES are used to send the email.

[0094] Specific example: Request to arrange a visit

[0095] For example, if an email arrives from another department requesting to schedule a visit next week, the device receives this email and interprets it as a "scheduling request." Next, the server checks its schedule database (e.g., Google Calendar) and automatically generates an email suggesting several possible dates for the visit. Finally, the device sends that email.

[0096] Specific example: Product inquiries from customers

[0097] Furthermore, when a customer sends an email requesting information about the specifications of product X, the terminal receives the email and identifies it as a customer inquiry. The server retrieves the specifications of product X from the product database and generates an automated response email. If the response is incomplete, the terminal automatically consults with the relevant department to obtain a final answer. Once this process is complete, the server creates a final email and sends it to the customer through the terminal.

[0098] Example of a prompt

[0099] "Inquiry: Please tell me about the specifications of product X."

[0100] "Category: Customer Inquiry"

[0101] By using the methods described above, this system can streamline internal email processing and maintain consistency and accuracy in responses.

[0102] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0103] Step 1:

[0104] The server collects internal company data. Specifically, it extracts past email data and inquiry history from mail servers such as Exchange Server and G Suite in CSV or JSON format. The input is data from the mail server, and the output is a structured data file.

[0105] Step 2:

[0106] The server stores the collected data in a database. The collected email data and inquiry history are saved in a database management system such as MySQL or MongoDB. The input is the data file generated in step 1, and the output is the records in the database.

[0107] Step 3:

[0108] The server uses a database to train a generative AI model. It uses pre-processed text data (e.g., OpenAI's GPT-3) to train the generative AI model. The input is text data from the database, and the output is the trained AI model.

[0109] Step 4:

[0110] The device receives a new email. Protocols such as IMAP and POP3 are used to receive the email. The input is the new email from the mail server, and the output is the email data stored on the device's local system.

[0111] Step 5:

[0112] The terminal analyzes the content of emails and categorizes them. Using natural language processing technologies such as the Google Cloud Natural Language API, emails are classified into categories such as "Internal Consultation," "Report / Communication," "Confirmation of Sales Activity Requirements," and "Customer Inquiry." The input is the email data received in step 4, and the output is the analysis results including category information.

[0113] Step 6:

[0114] The server generates an initial response email based on the aforementioned categories. It references similar cases from the database and constructs the response email using an appropriate template. The input is the analysis results from step 5 and similar case data from the database, and the output is the initial response email.

[0115] Step 7:

[0116] If the terminal has difficulty responding or if high accuracy is required, the administrator will be asked to perform a detailed analysis of the inquiry. A notification will be sent to the administrator using tools such as Jira or Slack. The input is the initial response email and its difficulty assessment result, and the output is the notification to the administrator.

[0117] Step 8:

[0118] The server generates the most suitable answer candidates for the administrator. The generated answers are displayed in an HTML email or a dedicated dashboard. The input is the notification content from step 7, and the output is the answer candidates presented to the administrator.

[0119] Step 9:

[0120] The user (administrator) reviews the suggested answers and sends back feedback. The feedback is sent back to the system in JSON format. The input is the suggested answers presented in step 8, and the output is the administrator's feedback.

[0121] Step 10:

[0122] The server automatically generates a final response email based on feedback from the administrator. The generated email is constructed in MIME format. The input is the feedback obtained in step 9, and the output is the final response email.

[0123] Step 11:

[0124] The terminal automatically sends the final response email that was generated. Email sending APIs such as SendGrid or AWS SES are used to send the email. The input is the final response email generated in step 10, and the output is the sent email.

[0125] (Application Example 1)

[0126] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0127] In conventional factories, managers had to manually check and respond to work instructions and reports via email, a process that wasted time and effort. Furthermore, it was difficult to process abnormal reports and adjustment requests in a timely manner, leading to decreased operational efficiency and delays in responses. This invention aims to solve these problems and improve operational efficiency by automating data processing and email responses within the factory.

[0128] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0129] In this invention, the server includes means for collecting internal company data and building a database, means for training a generation AI model using the database, means for receiving emails, analyzing the content of the emails and classifying them into categories, means for a robot terminal to receive new emails, analyze their content and classify them into work instructions or report categories, means for having an administrator perform a detailed analysis of the inquiry content as needed, and means for automatically generating and sending a final response email based on the administrator's feedback. This automates the processing of work instructions and report emails and enables rapid response to anomaly reports and adjustment requests.

[0130] "Internal company data" refers to all information generated within a company, including past email data, inquiry history, business processes, decision-making criteria, and so on.

[0131] A "database" is a collection of electronic information that systematically organizes and stores company data, and manages it in a way that allows access as needed.

[0132] A "generative AI model" refers to an artificial intelligence model that uses information stored in a database to produce appropriate responses and processes, utilizing machine learning and deep learning.

[0133] "Email" refers to electronic messages sent and received via the internet or other electronic communication networks.

[0134] A "category" refers to a group or type of email classified based on its content or nature, and examples include "internal consultation" and "anomaly report."

[0135] An "initial response email" refers to the first response email automatically generated based on an AI model or template in response to a newly received email.

[0136] A "robot terminal" is a device that performs actual physical tasks in factories and other facilities, and has the ability to receive and analyze emails.

[0137] An "administrator" is a person or role responsible for overseeing and coordinating the system, and provides detailed analysis of email content and feedback as needed.

[0138] "Work instructions" are documents that describe the procedures and instructions for specific tasks or operations, and are provided to robot terminals or employees.

[0139] A "progress report" refers to a report that explains the status of ongoing work or projects, and is sent to the administrator via email or other means.

[0140] A "request for adjustment" refers to a request for changes or adjustments to the schedule or working conditions.

[0141] An "anomaly report" refers to a report of a problem or abnormal event that occurred within a factory or system, and it provides detailed information about the incident.

[0142] This invention relates to a system for automating data processing and email responses within a factory. In this system, servers, terminals, and users work together. Specific embodiments of the invention are described in detail below.

[0143] First, the server collects internal company data generated within the factory and builds a database. This data includes past email data, inquiry history, business processes, and decision-making criteria. A database like PostgreSQL might be used.

[0144] Next, the server uses this database to train a generative AI model. Machine learning and deep learning techniques are used for training, such as OpenAI's GPT-3.5. The AI ​​model learns past email content and appropriate response pairs to enable it to generate natural-sounding responses.

[0145] When the terminal receives a new email, it analyzes its content and classifies it into categories such as "internal consultation," "report / communication," "confirmation of sales activity requirements," "customer inquiry," or "work instructions," "progress report," "coordination request," or "anomaly report." Natural language processing technology (e.g., SpaCy) is used for the analysis.

[0146] The server generates an initial response email based on the category and content of the received email. In doing so, it improves the quality of the response by referring to similar cases in the database and using appropriate templates (greeting, body, and closing). For example, for scheduling requests, it refers to the schedule database and automatically generates an email suggesting several possible dates for a visit.

[0147] If a response is difficult or requires high accuracy, the terminal notifies the administrator to analyze the inquiry in detail. In this case, the server generates the most appropriate answer candidates and presents them to the user (administrator). The user reviews and modifies the answer candidates and provides feedback to the system.

[0148] The server receives feedback from the administrator and automatically generates a final response email. This email is checked for completeness and then automatically sent through the terminal.

[0149] As a concrete example, the following is a case of an abnormal report.

[0150] The database stores a history of abnormal occurrences, past response results, and detailed information about the work area. When the robot terminal detects an abnormality in the equipment, it automatically generates the following prompt message and sends it to the server.

[0151] Prompt message:

[0152] Generate an anomaly detection report email.

[0153] Location of abnormality: "Robot arm A3"

[0154] Problem description: "Motor overheating"

[0155] Date of occurrence: YYYY / MM / DD

[0156] Required action: "Prompt inspection and replacement"

[0157] Based on past cases of handling anomalies, generate a detailed report email.

[0158] The server uses this prompt to generate a detailed report email from the AI ​​model. As a result, an email like the following is sent to the administrator.

[0159] Result email:

[0160] Subject: Regarding motor overheating in robot arm A3

[0161] Main text:

[0162] Dear Administrator,

[0163] An overheating issue has been detected in the A3 robot arm. Please see the details below.

[0164] Location of abnormality: Robot arm A3

[0165] Problem description: Motor overheating

[0166] Date of occurrence: YYYY / MM / DD

[0167] Required action: Prompt inspection and replacement

[0168] Based on similar cases in the past, prompt action is required. Please review the matter.

[0169] From factory automation systems

[0170] In this way, data processing and email responses within the factory are automated, significantly improving operational efficiency.

[0171] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0172] Step 1:

[0173] The server collects internal company data generated within the factory, such as past email data, inquiry history, business processes, and decision-making criteria. A database is built to integrate and manage this data. Various internal company data is provided as input, and an integrated database is generated as output. This database is implemented using, for example, PostgreSQL.

[0174] Step 2:

[0175] The server uses the constructed database to train a generative AI model. This training uses email content and appropriate response pairs, for example, OpenAI's GPT-3.5 protocol. The input consists of past email data and example responses from the database, and the output is a generative AI model. This model is used to generate natural-sounding email responses.

[0176] Step 3:

[0177] The terminal receives a new email and analyzes its content. Using natural language processing technology (e.g., SpaCy), it analyzes the email content and classifies it into categories such as "Internal Consultation," "Report / Communication," "Confirmation of Sales Activity Requirements," "Customer Inquiry," "Work Instructions," "Progress Report," "Adjustment Request," and "Anomaly Report." A new email is provided as input, and a category is assigned as output.

[0178] Step 4:

[0179] The server generates an initial response email based on the category and content of the received email. In doing so, it refers to similar cases in the database and uses the appropriate template. Given the email category and its content as input, the initial response email is generated as output. For example, for a scheduling request email, it refers to the schedule database and generates an email suggesting available dates for a visit.

[0180] Step 5:

[0181] If the terminal has difficulty responding or requires high accuracy, it notifies the administrator to perform a detailed analysis of the inquiry. In this case, the server generates the most suitable answer candidates and presents them to the user (administrator). The input is an email requiring detailed analysis, and the output is the generation of answer candidates to present to the administrator.

[0182] Step 6:

[0183] The user (administrator) reviews the suggested answer options and makes corrections as needed. The corrections are fed back into the system. Feedback from the administrator is provided as input, and the corrected answer is obtained as output.

[0184] Step 7:

[0185] The server receives feedback from the administrator and automatically generates a final response email. This email is checked for completeness and automatically sent through the terminal. The corrected response is given as input, and the final response email is generated as output.

[0186] Step 8:

[0187] When the terminal detects an anomaly, it generates a prompt message and sends it to the server. The server uses the prompt message to create a detailed report email from the AI ​​model and sends it to the administrator. As a specific example, the prompt message when the robot terminal detects an anomaly is as follows:

[0188] Prompt message:

[0189] Generate an anomaly detection report email.

[0190] Location of abnormality: "Robot arm A3"

[0191] Problem description: "Motor overheating"

[0192] Date of occurrence: YYYY / MM / DD

[0193] Required action: "Prompt inspection and replacement"

[0194] Based on past cases of handling anomalies, generate a detailed report email.

[0195] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0196] This invention relates to a system that collects internal company data, builds a database, and trains a generative AI model. Furthermore, this system incorporates an emotion engine that recognizes user emotions, adding a function to reflect user emotions in email content and generate more appropriate initial responses. The specific processing of the program of this system will be described below.

[0197] Database construction and AI model training

[0198] The server acquires new data and integrates it into the existing database. This includes historical email data, inquiry history, business processes, and decision criteria. The constructed database is used to train a generative AI model. The training dataset includes email content and corresponding replies, and the AI ​​model is trained to generate natural-sounding responses based on this data.

[0199] Emotional engine integration and emotion recognition

[0200] When a device receives a new email, it analyzes its content. During this process, the emotion engine analyzes the text to determine positive, negative, or neutral emotions and generates a corresponding emotion score. For example, in the case of a customer complaint email, the emotion engine identifies negative emotions and incorporates this result into the subsequent response generation process.

[0201] Category classification and generation of initial responses

[0202] Based on the analysis results of the emotion engine, the terminal classifies the email content into categories such as "internal consultation," "report / communication," "confirmation of sales activity requirements," and "customer inquiry." Based on this classification, the server refers to similar cases and templates in the database and generates an initial response email that reflects the emotion score.

[0203] Notification to the administrator and confirmation of response.

[0204] If a response is difficult or negative emotions are detected, the terminal automatically notifies the administrator. The server then sends an initial response email to the administrator, clearly indicating that negative emotions are present. This allows the user (administrator) to provide appropriate feedback on the negative emotions and modify the initial response.

[0205] Generation and transmission of the final response

[0206] Once administrator feedback is provided, the server automatically generates a final response email based on it. This final email is reviewed and revised by the administrator, and the system ensures it is sent to the recipient without fail. For example, if negative sentiment is detected in an email from a customer regarding a product defect, it provides guidance for the administrator to respond more carefully and courteously.

[0207] Specific example

[0208] For example, if an email arrives from another department requesting to schedule a visit next week, the terminal receives the email and categorizes it as a "scheduling request." If the sentiment analysis result is neutral, the server checks the schedule database and automatically generates an email suggesting several possible dates for the visit. If the sentiment engine detects a positive sentiment, the response email can include more friendly language.

[0209] Furthermore, if a customer sends an email asking for information about the specifications of product X, the terminal receives the email, determines its category to "customer inquiry," and the sentiment engine analyzes the message to see if it contains any negative sentiment. If negative sentiment is detected, the server automatically consults with the relevant department and prepares a detailed response. The user (administrator) reviews and revises the response options, and then sends the final answer to the customer based on that feedback.

[0210] This system significantly improves the efficiency of email correspondence and enables better communication by taking user emotions into consideration.

[0211] The following describes the processing flow.

[0212] Step 1:

[0213] The server collects internal company data such as past email data, inquiry history, business processes, and decision-making criteria, and stores this information in a database. Specifically, it extracts necessary information from internal document management systems and email servers, enabling centralized management of this data.

[0214] Step 2:

[0215] The server uses information from the database to train a generative AI model. This process involves providing the AI ​​model with pairs of email content and corresponding replies, training it to generate natural-sounding responses. This allows the model to acquire the ability to generate appropriate responses for various situations.

[0216] Step 3:

[0217] Each time the terminal receives a new email, it analyzes its contents. Using natural language processing technology, it extracts keywords and important contextual information from the email body and classifies the email into categories such as "internal consultation," "report / communication," "sales activity requirements confirmation," and "customer inquiry." This allows the appropriate response to be selected in the next processing step.

[0218] Step 4:

[0219] The device inputs the analyzed email content into an emotion engine, which determines whether the emotion is positive, negative, or neutral. The emotion engine calculates an emotion score from the text and incorporates the result into the next response generation process. Specifically, if the negative emotion is high, a response that takes this into account is generated.

[0220] Step 5:

[0221] The server generates an initial response email based on the aforementioned categories and sentiment engine results. It refers to similar cases and templates in the database, creating the initial response email using more friendly language for positive responses and polite and careful language for negative responses.

[0222] Step 6:

[0223] The server provides the terminal with an initial response email and notifies the administrator as needed. If the email content is complex or the emotion engine identifies negative emotions, the terminal automatically notifies the administrator.

[0224] Step 7:

[0225] The user (administrator) receives the notification and provides necessary corrections or additional information based on the initial response email and sentiment analysis results provided by the server. For emails containing negative sentiment, the administrator will create a more careful and appropriate response.

[0226] Step 8:

[0227] The server automatically generates a final response email based on the administrator's feedback and provides it to the terminal. Once the email content reflects the administrator's confirmation, a final check is performed.

[0228] Step 9:

[0229] The terminal performs a final check, including error checking and final verification, before automatically sending the email to the intended recipient.

[0230] Through this series of steps, the system automatically generates high-quality email responses that reflect the user's emotions, streamlining internal and external communication.

[0231] (Example 2)

[0232] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0233] Traditional email response systems often fail to consider user emotions, making it difficult to provide appropriate responses, particularly in handling emails containing negative emotions quickly and effectively. Furthermore, the time and effort required to manually categorize email content was a significant problem.

[0234] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0235] In this invention, the server includes means for collecting internal company data and building a database, means for training a generative AI model using the database, means for receiving emails, analyzing the content of the emails and generating sentiment scores, means for classifying email categories based on the sentiment scores, means for generating initial response emails based on the categories, means for notifying the administrator if the initial response email contains negative sentiments, and means for automatically generating and sending a final response email based on the administrator's feedback. This enables quick and appropriate email responses that take into account the user's sentiments.

[0236] "Internal data" refers to information generated or collected within a company or organization, and includes email data, inquiry history, business processes, decision-making criteria, etc.

[0237] A "database" refers to a system for systematically storing collected company data and for efficiently accessing and managing it.

[0238] A "generative AI model" refers to an artificial intelligence model that uses machine learning algorithms to generate natural-sounding responses based on the content of an email.

[0239] "Email" refers to digital messages exchanged over the internet, and can include text and attachments.

[0240] "Sentiment score" is a numerical or categorical representation of the user's emotions (positive, negative, neutral) as expressed in the content of an email.

[0241] A "category" is a group of emails classified based on their content, and examples include "internal consultation," "reporting / communication," "confirmation of sales activity requirements," and "customer inquiry."

[0242] An "initial response email" refers to the first reply email automatically generated by the system in response to an email that has been received.

[0243] An "administrator" refers to someone responsible for making important decisions and corrections in system operation, and is responsible for reviewing and correcting response emails automatically generated by the system.

[0244] A "final response email" refers to the final reply email generated based on the administrator's feedback.

[0245] This invention aims to realize an efficient email response system by collecting internal company data to build a database and training a generative AI model to provide an automated email response function that takes user sentiment into consideration. This system consists of a server, terminals, and users.

[0246] Data collection and database construction

[0247] The server aggregates data collected from various departments within the company, including email data, inquiry history, business processes, and decision-making criteria. The data is first saved to the local disk and then inserted into a database. This involves using the Python pandas library to read data in CSV format and then using SQLAlchemy to insert it into a PostgreSQL database. Finally, it executes queries necessary to ensure data integrity.

[0248] AI model training

[0249] The server trains a generative AI model using an integrated database. This AI model uses the Hugging Face transformers library to load a pre-prepared language model (e.g., BERT). Then, it feeds the training dataset extracted from the integrated database to the model and optimizes it using the PyTorch library.

[0250] Emotion analysis and emotion score generation

[0251] When the device receives a new email, it analyzes and understands its content. The email content is then analyzed using the TextBlob library to generate sentiment scores (positive, negative, or neutral). For example, if the prompt "I am very disappointed with this product" is analyzed, a negative sentiment score will be assigned.

[0252] Email category classification

[0253] Upon receiving an email, the terminal categorizes it based on its sentiment score and content. This classification is performed using Scikit-learn's Naive Bayes classifier, and the results are stored in a local database. Categories include "Internal Consultation," "Reporting / Communication," "Confirmation of Sales Activity Requirements," and "Customer Inquiry."

[0254] Generating an initial response email

[0255] The server generates an initial response email by referencing similar templates in the database based on the classified category and sentiment score. This process uses SQL queries to find the appropriate template and the Jinja2 template engine to populate it with sentiment score and category information.

[0256] Notification to the administrator

[0257] If the initial response email contains negative sentiment, the device automatically notifies the administrator. This notification sends the initial response email and the sentiment analysis results to the administrator via the SMTP server.

[0258] Administrator's feedback

[0259] The user (administrator) receives a notification, reviews the initial response email, and makes corrections as needed. These corrections are made through a web interface (e.g., a Django application), and feedback is sent to the server.

[0260] Generating and sending the final response email

[0261] Based on the administrator's feedback, the server generates a final response email. This generated email is automatically sent via the SMTP server.

[0262] Specific example

[0263] For example, if an email arrives from another department requesting to schedule a visit next week, the terminal receives the email and categorizes it as a "scheduling request." If the sentiment analysis is neutral, the server checks the schedule database and automatically generates an email suggesting several possible visit dates. Similarly, for an email requesting information about product X, the category is determined to be "customer inquiry." If the sentiment engine detects negative emotions, the server consults with the relevant department and prepares a detailed response. The user (administrator) reviews and revises the response options, and the final answer is sent to the customer based on their feedback.

[0264] This system significantly improves the efficiency of email correspondence and enables better communication by taking user emotions into consideration.

[0265] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0266] Step 1:

[0267] The server collects data from various departments within the company, including email data, inquiry history, business processes, and decision-making criteria. Specifically, it uses the Python pandas library to read data in CSV file format and saves it to the local disk.

[0268] Input: Data in CSV file format collected from each department.

[0269] Data processing: Read the CSV file using pandas and convert it to a DataFrame.

[0270] Output: Data stored in memory as a pandas dataframe

[0271] Step 2:

[0272] The server inserts the collected data into a PostgreSQL database using SQLAlchemy. During this process, it executes queries to maintain data integrity and sets necessary indexes.

[0273] Input: Data in pandas DataFrame format

[0274] Data processing: Inserting dataframes into a PostgreSQL database using SQLAlchemy.

[0275] Output: Data integrated into a PostgreSQL database

[0276] Step 3:

[0277] The server extracts training datasets from an integrated database and trains a generative AI model. Here, it uses the Hugging Face transformers library to load a pre-prepared BERT model. The PyTorch library is then used to optimize the model.

[0278] Input: Training dataset extracted from a PostgreSQL database

[0279] Data Calculation: Training an AI model using Hugging Face transformers and PyTorch

[0280] Output: Optimized generative AI model

[0281] Step 4:

[0282] The terminal receives new emails. Using a mail server (e.g., IMAP server), the received emails are analyzed with the TextBlob library to generate positive, negative, and neutral sentiment scores.

[0283] Input: Received email

[0284] Data processing: Analyze the sentiment of the email content using the TextBlob library

[0285] Output: Sentiment score

[0286] Step 5:

[0287] Based on the content and sentiment score of the received email, the terminal classifies the email into categories using the Naive Bayes classifier in Scikit-learn. This classification is saved in the local database.

[0288] Input: Sentiment score and email content

[0289] Data processing: Categorize using the Naive Bayes classifier in Scikit-learn

[0290] Output: Categorization result

[0291] Step 6:

[0292] Based on the classified category and sentiment score, the server searches for similar templates in the database and uses the Jinja2 template engine to generate an initial response email.

[0293] Input: Categorization result and sentiment score

[0294] Data processing: Search for templates with SQL queries and embed information in the templates with Jinja2

[0295] Output: Initial response email

[0296] Step 7:

[0297] If the terminal detects negative emotions in the initial response email, it automatically notifies the administrator. The notification is sent using the SMTP server and includes the initial response email and the result of the sentiment analysis.

[0298] Input: Initial response email

[0299] Data processing: Send a notification email using the SMTP server

[0300] Output: Notification email to the administrator

[0301] Step 8:

[0302] The user (administrator) receives the notification, checks the initial response email, and makes corrections through the web interface (e.g., Django application) if necessary. The correction results are sent to the server.

[0303] Input: Notification email to the administrator and web interface

[0304] Data processing: Administrator's corrections and feedback submission

[0305] Output: Corrected response email feedback

[0306] Step 9:

[0307] The server generates the final response email based on the administrator's feedback. The generated email is automatically sent to the recipient through the SMTP server.

[0308] Input: Administrator's feedback

[0309] Data processing: Generate the final response email using the Jinja2 template engine

[0310] Output: Sending of the final response email

[0311] (Application Example 2)

[0312] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0313] In modern businesses, there is a need to process emails and other digital communications quickly and appropriately to improve the efficiency of customer service and internal communication. Furthermore, in the security field, it is necessary to utilize real-time sentiment recognition to detect abnormal behavior early and take swift, appropriate action. However, conventional systems struggle to address these challenges simultaneously, particularly lacking in the detection of abnormal behavior and rapid notification to administrators using sentiment recognition technology. Therefore, there is a need for the development of a system that comprehensively achieves both efficient email response and enhanced security monitoring.

[0314] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0315] In this invention, the server includes means for collecting internal company data and building a database, means for training a generative AI model using the database, means for receiving emails, analyzing the content of the emails and classifying them into categories, means for generating an initial response email based on the categories, means for having an administrator perform a detailed analysis of the inquiry content as needed, means for automatically generating and sending a final response email based on the administrator's feedback, means for detecting faces in real time and generating an emotion score using an emotion engine, means for suggesting countermeasures when abnormal emotions are detected, and means for detecting abnormal behavior based on the emotion score and providing notifications. This enables more efficient email handling and enhanced security monitoring.

[0316] "Internal company data" refers to various types of information and data generated within a company, including business processes, inquiry history, past email data, and decision-making criteria.

[0317] A "database" is a structured collection of data that systematically stores collected company data, making it easy to search and manipulate.

[0318] A "generative AI model" is an artificial intelligence model that learns from collected data and provides natural responses and inferences to new data.

[0319] "Email" refers to digital messages sent and received using computer networks such as the internet.

[0320] "Means of categorization" refers to algorithms or devices that analyze the content of received emails and sort them into predefined categories (e.g., internal consultation, report / communication).

[0321] An "initial response email" is the first reply email that is automatically generated in response to an email that has been received.

[0322] An "administrator" is a person responsible for the operation and supervision of a system, and is also responsible for analyzing system inquiries and providing feedback as needed.

[0323] A "final response email" is the final reply email that is automatically generated based on the administrator's feedback and sent to the recipient.

[0324] "Means for detecting faces" refers to technologies or devices that use cameras or other imaging devices to identify and recognize human faces within an image.

[0325] An "emotion engine" is an algorithm or system that analyzes emotions from detected facial expressions and generates emotion scores such as positive, negative, or neutral.

[0326] An "emotion score" is a numerical representation of the degree of emotion analyzed by the emotion engine.

[0327] "Abnormal emotions" refer to negative emotions that exceed certain standards, such as anger, fear, or disgust.

[0328] "Means of suggesting countermeasures" refers to a mechanism or system that, when abnormal emotions are detected, indicates appropriate actions or responses in accordance with the situation.

[0329] "Means of notification" refers to a system for promptly informing security personnel and administrators of any detected abnormal emotions or behaviors.

[0330] To implement this invention, the system is constructed according to the following steps.

[0331] Database construction and AI model training

[0332] The server collects internal company data and builds a database. This internal data includes past email data, inquiry history, business processes, and decision-making criteria. Based on this data, a generative AI model is trained. The training dataset includes email content and corresponding replies, and the AI ​​model is trained to generate natural-sounding responses based on this data.

[0333] Emotional engine integration and emotion recognition

[0334] When a device receives a new email, it analyzes its content. During this process, the emotion engine analyzes the text to determine positive, negative, or neutral emotions and generates an emotion score. For example, in the case of a "customer complaint email," negative emotions are identified, and this result is reflected in the subsequent response generation process.

[0335] Face detection and real-time emotion score generation

[0336] The device uses a camera to detect faces in its surroundings in real time. Face detection uses face recognition libraries such as OpenCV. The detected face images are analyzed by an emotion engine, and an emotion score is generated. This allows for suggested actions if abnormal emotions are detected.

[0337] Generation of initial response email and notification to administrator

[0338] The device categorizes emails based on their sentiment score and content, and generates an initial response email. This response email is automatically generated using a template. If an abnormal sentiment is detected, the device notifies the administrator, and if the situation is complex or negative sentiment is identified, the administrator is instructed to perform a detailed analysis.

[0339] Generating and sending the final response email

[0340] Based on administrator feedback, the server automatically generates a final response email. This final email is reviewed and corrected by the administrator, and the system sends it to the recipient.

[0341] Hardware and software to be used

[0342] Hardware: Camera devices, servers, terminals

[0343] Software: OpenCV (face detection), emotion engine (emotion analysis), database management system (management of internal company data)

[0344] Specific example:

[0345] Smart glasses are used as a security system in supermarkets and shopping malls to detect individuals exhibiting unusual emotions in real time. Alerts are sent to security guards, enabling a rapid response.

[0346] Example of a prompt:

[0347] Premise: In a supermarket, security guards are using smart glasses. A real-time emotion recognition and facial recognition system is needed to detect unusual emotions (anger, fear, disgust) and immediately notify the security team.

[0348] prompt:

[0349] 1. The facial recognition system detects customers' faces in the supermarket in real time and performs sentiment analysis.

[0350] 2. The emotion engine analyzes each customer's facial expressions and identifies customers who are experiencing emotions such as anger, fear, or disgust.

[0351] 3. If abnormal emotions are detected, the notification system will send an alert to the security team along with the log and facial image.

[0352] 4. The security team will use this information to respond quickly and ensure security.

[0353] Please create Python code to implement the process described above.

[0354] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0355] Step 1:

[0356] The server collects internal company data. It retrieves data such as past email data, inquiry history, business processes, and decision criteria, and stores it in a database. This database is used to build a training dataset for generative AI models. The input is internal company data, and the output is the constructed database.

[0357] Step 2:

[0358] The server uses a database to train a generative AI model. Email content and corresponding replies are used as the training dataset to train the AI ​​model to generate natural-sounding responses. The input is the training dataset in the database, and the output is the trained generative AI model.

[0359] Step 3:

[0360] The device receives a new email. It analyzes the email's content, uses an emotion engine to identify positive, negative, and neutral emotions, and generates an emotion score. The input is the received email, and the output is the emotion score. Specifically, it analyzes the email text and calculates the emotion score using the emotion engine.

[0361] Step 4:

[0362] The device categorizes emails based on their sentiment score and content. For example, it might categorize them as "Internal Consultation," "Report / Communication," "Confirmation of Sales Activity Requirements," or "Customer Inquiry." The input is the sentiment score and email content, and the output is the categorized email.

[0363] Step 5:

[0364] The terminal generates an initial response email based on the category. Using a template, it generates an initial response that includes appropriate wording according to the sentiment score. The input is the category and sentiment score, and the output is the initial response email.

[0365] Step 6:

[0366] If deemed necessary by the administrator, the terminal will prompt the administrator to perform a detailed analysis of the inquiry. The administrator will then provide feedback and optimize the content of the initial response email. The input is the initial response email, and the output is the administrator's feedback.

[0367] Step 7:

[0368] The server automatically generates a final response email based on the administrator's feedback. This final email is then reviewed and revised by the administrator before being sent to the recipient. The input is the administrator's feedback, and the output is the final response email.

[0369] Step 8:

[0370] The device uses a camera to detect faces in its surroundings in real time. The detected face images are analyzed by an emotion engine, and an emotion score is generated. The input is an image from the camera device, and the output is the face image and the emotion score.

[0371] Step 9:

[0372] Based on the emotion score, if abnormal emotions are detected, the device will suggest actions and send a notification. The notification will be sent to a designated security officer or administrator, requiring a prompt response. The input is the emotion score, and the output is a notification message. Specifically, it evaluates the emotion score and sends a notification according to the level of risk.

[0373] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0374] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0375] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0376] [Second Embodiment]

[0377] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0378] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0379] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0380] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0381] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0382] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0383] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0384] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0385] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0386] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0387] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0388] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0389] This invention relates to a system for automating internal data processing and email responses. This system functions collaboratively through the interaction of servers, terminals, and users. The program's processing is described below in natural language.

[0390] Database construction and AI model training

[0391] First, the server collects internal company data such as past email data, inquiry history, business processes, and decision criteria, and stores this information in a database. Next, the server uses this database to train a generative AI model. Specifically, by training the AI ​​with email content and appropriate response pairs, it becomes possible to generate natural-sounding responses.

[0392] Receiving and analyzing new emails

[0393] When a terminal receives a new email, it analyzes its content and categorizes it into categories such as "Internal Consultation," "Report / Communication," "Confirmation of Sales Activity Requirements," and "Customer Inquiry." To classify the email, natural language processing technology is used to understand its content and assign the appropriate category.

[0394] Generation of initial response

[0395] Next, the server generates an initial response email based on the category and content of the received email. In this process, it improves the quality of the response by referring to similar cases in the database and using appropriate templates (greeting, body, and closing). For example, if a customer inquires about product specifications, the server retrieves the relevant specification information from the product database and generates a response email.

[0396] Consult with the administrator (if necessary)

[0397] If a response is difficult or requires high accuracy, the terminal notifies the administrator to analyze the inquiry in detail. In this case, the server generates the most appropriate answer candidates and presents them to the user (administrator). The user reviews and modifies the answer candidates and provides feedback to the system.

[0398] Automated email creation and sending

[0399] Finally, the server receives feedback from the administrator and automatically generates a final response email. This email is checked for completeness and then automatically sent through the terminal.

[0400] Specific example: Request to arrange a visit

[0401] For example, if an email arrives from another department requesting to schedule a visit next week, the terminal receives the email and categorizes it as a "scheduling request." Next, the server checks the schedule database and automatically generates an email suggesting several possible dates for the visit. Finally, the terminal sends that email.

[0402] Specific example: Product inquiries from customers

[0403] Furthermore, when a customer sends an email requesting information about the specifications of product X, the terminal receives the email and identifies its category as "customer inquiry." The server retrieves the specifications of product X from the product database and generates an automated response email. If the response is incomplete, the terminal automatically consults with the relevant department to obtain a final answer. The server then creates a final email and sends it to the customer through the terminal.

[0404] By using the methods described above, internal email processing will be automated, significantly improving employee work efficiency.

[0405] The following describes the processing flow.

[0406] Step 1:

[0407] The server collects past email data, inquiry history, business processes, and decision-making criteria from within the company, and stores this information in a database. Specifically, it extracts necessary data from email sending and receiving logs and document management systems, organizes it, and registers it in the database.

[0408] Step 2:

[0409] The server uses information from the database to train a generative AI model (such as GPT-3). This involves providing the AI ​​model with pairs of email content and corresponding replies, and repeatedly training it to improve its ability to generate natural-sounding text.

[0410] Step 3:

[0411] Each time the terminal receives a new email, it analyzes its content. Using natural language processing technology, it extracts keywords and important contextual information from the email body and classifies the emails into categories such as "internal consultation," "report / communication," "confirmation of sales activity requirements," and "customer inquiry."

[0412] Step 4:

[0413] The server generates an initial response email based on the aforementioned categories. It refers to similar cases and templates in the database to construct the most appropriate response. For example, in response to a customer inquiry, it prepares the answer by referring to relevant product information and FAQs.

[0414] Step 5:

[0415] The server provides the terminal with an initial response email. If the initial response is insufficient to resolve the issue, or if a high level of accuracy is required, the terminal will notify the appropriate administrator.

[0416] Step 6:

[0417] The user (administrator) receives a notification and reviews and modifies the suggested answers provided by the server. If necessary, they enter additional information or supplementary explanations. The administrator's feedback is reflected in the system.

[0418] Step 7:

[0419] The server automatically generates a final response email based on feedback from the administrator. It then incorporates any corrections or additional information from the administrator to create a complete response.

[0420] Step 8:

[0421] The terminal checks the final response email, performs error checks and final confirmations, and then automatically sends the email to the intended recipient.

[0422] This series of processing steps automates email processing within the company, significantly reducing the effort and time required from employees.

[0423] (Example 1)

[0424] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0425] This invention relates to a system for automating internal data processing and email responses. Conventional systems suffered from long response times from email reception to response, resulting in poor efficiency. Furthermore, manual response creation was prone to human error, often leading to a lack of consistency and accuracy in response content. This resulted in decreased operational efficiency and reduced customer satisfaction.

[0426] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0427] In this invention, the server includes means for collecting internal company data and building a database, means for training a generative AI model using the database, means for receiving emails, analyzing the content of the emails and classifying them into categories, means for generating an initial response email based on the categories, means for generating a response email by referring to similar cases in the database, means for having an administrator perform a detailed analysis of the inquiry content as needed, and means for automatically generating and sending a final response email based on the administrator's feedback. This makes email processing more efficient and improves the consistency and accuracy of response content.

[0428] A "server" is a computer system that provides data and services to multiple computers and terminals over a network.

[0429] A "database" is a systematically organized collection of data, stored in a way that allows for efficient searching and updating.

[0430] A "generative AI model" is a type of artificial intelligence that learns from large amounts of data and performs tasks such as generating natural language and automatically generating responses.

[0431] "Natural language processing technology" refers to a set of technologies and methods that enable computers to understand, generate, and engage in human dialogue.

[0432] "Email" refers to digital messages sent and received via the internet or other computer networks.

[0433] A "category" is a way of classifying and grouping data based on certain common characteristics or attributes.

[0434] A "template" is a document model with a specific format or pattern, and it is a document format that allows for the easy generation of standardized documents by filling in the content.

[0435] "Feedback" refers to evaluations and corrections provided by users or administrators regarding responses and results generated by a system.

[0436] "IMAP" is an abbreviation for Internet Mail Access Protocol, and it is a protocol for managing email on a server.

[0437] POP3 is version 3 of the postal protocol and is an internet standard protocol used to receive email.

[0438] "MIME format" is an abbreviation for Multipurpose Internet Mail Extensions format, and it is a format that allows emails to contain data other than text (such as images, audio, and video).

[0439] "JSON format" is an abbreviation for JavaScript Object Notation format, and is a lightweight data description format primarily used for data exchange.

[0440] This invention relates to a system for automating internal data processing and email responses. This system functions collaboratively through the interaction of servers, terminals, and users.

[0441] First, the server collects data such as past email data, inquiry history, business processes, and decision criteria from within the company, and stores this information in a database. Possible email servers to use include Exchange Server and G Suite. The collected data is saved in CSV or JSON format, and database management systems such as MySQL or MongoDB are used for the database. Based on this database, the server trains a generative AI model (for example, OpenAI's GPT-3). Email content and appropriate response pairs are used as training data.

[0442] Next, the device receives new emails. Protocols such as IMAP and POP3 are used to receive emails. The content of the received emails is analyzed using natural language processing technology (e.g., Google Cloud Natural Language API), and the emails are categorized into categories such as "internal consultation," "report / communication," "sales activity requirements confirmation," and "customer inquiry."

[0443] Based on the aforementioned category and email content, the server refers to similar cases in the database and generates an initial response email. The generated response email includes a greeting, body, and closing. For example, in response to an inquiry such as "Please tell me about the specifications of product X," the server retrieves the relevant specification information from the product database and generates a response. This response email is constructed in MIME format.

[0444] If a response is difficult or requires high accuracy, the terminal notifies the administrator for detailed analysis. The administrator is notified using tools such as Jira or Slack, and the server generates optimal response candidates and presents them to the user (administrator). The administrator reviews these response candidates, makes corrections as needed, and sends feedback back to the system in JSON format.

[0445] Based on this feedback, the server automatically generates a final response email. The generated email is then automatically sent through the device. Email sending APIs such as SendGrid and AWS SES are used to send the email.

[0446] Specific example: Request to arrange a visit

[0447] For example, if an email arrives from another department requesting to schedule a visit next week, the device receives this email and interprets it as a "scheduling request." Next, the server checks its schedule database (e.g., Google Calendar) and automatically generates an email suggesting several possible dates for the visit. Finally, the device sends that email.

[0448] Specific example: Product inquiries from customers

[0449] Furthermore, when a customer sends an email requesting information about the specifications of product X, the terminal receives the email and identifies it as a customer inquiry. The server retrieves the specifications of product X from the product database and generates an automated response email. If the response is incomplete, the terminal automatically consults with the relevant department to obtain a final answer. Once this process is complete, the server creates a final email and sends it to the customer through the terminal.

[0450] Example of a prompt

[0451] "Inquiry: Please tell me about the specifications of product X."

[0452] "Category: Customer Inquiry"

[0453] By using the methods described above, this system can streamline internal email processing and maintain consistency and accuracy in responses.

[0454] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0455] Step 1:

[0456] The server collects internal company data. Specifically, it extracts past email data and inquiry history from mail servers such as Exchange Server and G Suite in CSV or JSON format. The input is data from the mail server, and the output is a structured data file.

[0457] Step 2:

[0458] The server stores the collected data in a database. The collected email data and inquiry history are saved in a database management system such as MySQL or MongoDB. The input is the data file generated in step 1, and the output is the records in the database.

[0459] Step 3:

[0460] The server uses a database to train a generative AI model. It uses pre-processed text data (e.g., OpenAI's GPT-3) to train the generative AI model. The input is text data from the database, and the output is the trained AI model.

[0461] Step 4:

[0462] The device receives a new email. Protocols such as IMAP and POP3 are used to receive the email. The input is the new email from the mail server, and the output is the email data stored on the device's local system.

[0463] Step 5:

[0464] The terminal analyzes the content of emails and categorizes them. Using natural language processing technologies such as the Google Cloud Natural Language API, emails are classified into categories such as "Internal Consultation," "Report / Communication," "Confirmation of Sales Activity Requirements," and "Customer Inquiry." The input is the email data received in step 4, and the output is the analysis results including category information.

[0465] Step 6:

[0466] The server generates an initial response email based on the aforementioned categories. It references similar cases from the database and constructs the response email using an appropriate template. The input is the analysis results from step 5 and similar case data from the database, and the output is the initial response email.

[0467] Step 7:

[0468] If the terminal has difficulty responding or if high accuracy is required, the administrator will be asked to perform a detailed analysis of the inquiry. A notification will be sent to the administrator using tools such as Jira or Slack. The input is the initial response email and its difficulty assessment result, and the output is the notification to the administrator.

[0469] Step 8:

[0470] The server generates the most suitable answer candidates for the administrator. The generated answers are displayed in an HTML email or a dedicated dashboard. The input is the notification content from step 7, and the output is the answer candidates presented to the administrator.

[0471] Step 9:

[0472] The user (administrator) reviews the suggested answers and sends back feedback. The feedback is sent back to the system in JSON format. The input is the suggested answers presented in step 8, and the output is the administrator's feedback.

[0473] Step 10:

[0474] The server automatically generates a final response email based on feedback from the administrator. The generated email is constructed in MIME format. The input is the feedback obtained in step 9, and the output is the final response email.

[0475] Step 11:

[0476] The terminal automatically sends the final response email that was generated. Email sending APIs such as SendGrid or AWS SES are used to send the email. The input is the final response email generated in step 10, and the output is the sent email.

[0477] (Application Example 1)

[0478] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0479] In conventional factories, managers had to manually check and respond to work instructions and reports via email, a process that wasted time and effort. Furthermore, it was difficult to process abnormal reports and adjustment requests in a timely manner, leading to decreased operational efficiency and delays in responses. This invention aims to solve these problems and improve operational efficiency by automating data processing and email responses within the factory.

[0480] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0481] In this invention, the server includes means for collecting internal company data and building a database, means for training a generation AI model using the database, means for receiving emails, analyzing the content of the emails and classifying them into categories, means for a robot terminal to receive new emails, analyze their content and classify them into work instructions or report categories, means for having an administrator perform a detailed analysis of the inquiry content as needed, and means for automatically generating and sending a final response email based on the administrator's feedback. This automates the processing of work instructions and report emails and enables rapid response to anomaly reports and adjustment requests.

[0482] "Internal company data" refers to all information generated within a company, including past email data, inquiry history, business processes, decision-making criteria, and so on.

[0483] A "database" is a collection of electronic information that systematically organizes and stores company data, and manages it in a way that allows access as needed.

[0484] A "generative AI model" refers to an artificial intelligence model that uses information stored in a database to produce appropriate responses and processes, utilizing machine learning and deep learning.

[0485] "Email" refers to electronic messages sent and received via the internet or other electronic communication networks.

[0486] A "category" refers to a group or type of email classified based on its content or nature, and examples include "internal consultation" and "anomaly report."

[0487] An "initial response email" refers to the first response email automatically generated based on an AI model or template in response to a newly received email.

[0488] A "robot terminal" is a device that performs actual physical tasks in factories and other facilities, and has the ability to receive and analyze emails.

[0489] An "administrator" is a person or role responsible for overseeing and coordinating the system, and provides detailed analysis of email content and feedback as needed.

[0490] "Work instructions" are documents that describe the procedures and instructions for specific tasks or operations, and are provided to robot terminals or employees.

[0491] A "progress report" refers to a report that explains the status of ongoing work or projects, and is sent to the administrator via email or other means.

[0492] A "request for adjustment" refers to a request for changes or adjustments to the schedule or working conditions.

[0493] An "anomaly report" refers to a report of a problem or abnormal event that occurred within a factory or system, and it provides detailed information about the incident.

[0494] This invention relates to a system for automating data processing and email responses within a factory. In this system, servers, terminals, and users work together. Specific embodiments of the invention are described in detail below.

[0495] First, the server collects internal company data generated within the factory and builds a database. This data includes past email data, inquiry history, business processes, and decision-making criteria. A database like PostgreSQL might be used.

[0496] Next, the server uses this database to train a generative AI model. Machine learning and deep learning techniques are used for training, such as OpenAI's GPT-3.5. The AI ​​model learns past email content and appropriate response pairs to enable it to generate natural-sounding responses.

[0497] When the terminal receives a new email, it analyzes its content and classifies it into categories such as "internal consultation," "report / communication," "confirmation of sales activity requirements," "customer inquiry," or "work instructions," "progress report," "coordination request," or "anomaly report." Natural language processing technology (e.g., SpaCy) is used for the analysis.

[0498] The server generates an initial response email based on the category and content of the received email. In doing so, it improves the quality of the response by referring to similar cases in the database and using appropriate templates (greeting, body, and closing). For example, for scheduling requests, it refers to the schedule database and automatically generates an email suggesting several possible dates for a visit.

[0499] If a response is difficult or requires high accuracy, the terminal notifies the administrator to analyze the inquiry in detail. In this case, the server generates the most appropriate answer candidates and presents them to the user (administrator). The user reviews and modifies the answer candidates and provides feedback to the system.

[0500] The server receives feedback from the administrator and automatically generates a final response email. This email is checked for completeness and then automatically sent through the terminal.

[0501] As a concrete example, the following is a case of an abnormal report.

[0502] The database stores a history of abnormal occurrences, past response results, and detailed information about the work area. When the robot terminal detects an abnormality in the equipment, it automatically generates the following prompt message and sends it to the server.

[0503] Prompt message:

[0504] Generate an anomaly detection report email.

[0505] Location of abnormality: "Robot arm A3"

[0506] Problem description: "Motor overheating"

[0507] Date of occurrence: YYYY / MM / DD

[0508] Required action: "Prompt inspection and replacement"

[0509] Based on past cases of handling anomalies, generate a detailed report email.

[0510] The server uses this prompt to generate a detailed report email from the AI ​​model. As a result, an email like the following is sent to the administrator.

[0511] Result email:

[0512] Subject: Regarding motor overheating in robot arm A3

[0513] Main text:

[0514] Dear Administrator,

[0515] An overheating issue has been detected in the A3 robot arm. Please see the details below.

[0516] Location of abnormality: Robot arm A3

[0517] Problem description: Motor overheating

[0518] Date of occurrence: YYYY / MM / DD

[0519] Required action: Prompt inspection and replacement

[0520] Based on similar cases in the past, prompt action is required. Please review the matter.

[0521] From factory automation systems

[0522] In this way, data processing and email responses within the factory are automated, significantly improving operational efficiency.

[0523] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0524] Step 1:

[0525] The server collects internal company data generated within the factory, such as past email data, inquiry history, business processes, and decision-making criteria. A database is built to integrate and manage this data. Various internal company data is provided as input, and an integrated database is generated as output. This database is implemented using, for example, PostgreSQL.

[0526] Step 2:

[0527] The server uses the constructed database to train a generative AI model. This training uses email content and appropriate response pairs, for example, OpenAI's GPT-3.5 protocol. The input consists of past email data and example responses from the database, and the output is a generative AI model. This model is used to generate natural-sounding email responses.

[0528] Step 3:

[0529] The terminal receives a new email and analyzes its content. Using natural language processing technology (e.g., SpaCy), it analyzes the email content and classifies it into categories such as "Internal Consultation," "Report / Communication," "Confirmation of Sales Activity Requirements," "Customer Inquiry," "Work Instructions," "Progress Report," "Adjustment Request," and "Anomaly Report." A new email is provided as input, and a category is assigned as output.

[0530] Step 4:

[0531] The server generates an initial response email based on the category and content of the received email. In doing so, it refers to similar cases in the database and uses the appropriate template. Given the email category and its content as input, the initial response email is generated as output. For example, for a scheduling request email, it refers to the schedule database and generates an email suggesting available dates for a visit.

[0532] Step 5:

[0533] If the terminal has difficulty responding or requires high accuracy, it notifies the administrator to perform a detailed analysis of the inquiry. In this case, the server generates the most suitable answer candidates and presents them to the user (administrator). The input is an email requiring detailed analysis, and the output is the generation of answer candidates to present to the administrator.

[0534] Step 6:

[0535] The user (administrator) reviews the suggested answer options and makes corrections as needed. The corrections are fed back into the system. Feedback from the administrator is provided as input, and the corrected answer is obtained as output.

[0536] Step 7:

[0537] The server receives feedback from the administrator and automatically generates a final response email. This email is checked for completeness and automatically sent through the terminal. The corrected response is given as input, and the final response email is generated as output.

[0538] Step 8:

[0539] When the terminal detects an anomaly, it generates a prompt message and sends it to the server. The server uses the prompt message to create a detailed report email from the AI ​​model and sends it to the administrator. As a specific example, the prompt message when the robot terminal detects an anomaly is as follows:

[0540] Prompt message:

[0541] Generate an anomaly detection report email.

[0542] Location of abnormality: "Robot arm A3"

[0543] Problem description: "Motor overheating"

[0544] Date of occurrence: YYYY / MM / DD

[0545] Required action: "Prompt inspection and replacement"

[0546] Based on past cases of handling anomalies, generate a detailed report email.

[0547] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0548] This invention relates to a system that collects internal company data, builds a database, and trains a generative AI model. Furthermore, this system incorporates an emotion engine that recognizes user emotions, adding a function to reflect user emotions in email content and generate more appropriate initial responses. The specific processing of the program of this system will be described below.

[0549] Database construction and AI model training

[0550] The server acquires new data and integrates it into the existing database. This includes historical email data, inquiry history, business processes, and decision criteria. The constructed database is used to train a generative AI model. The training dataset includes email content and corresponding replies, and the AI ​​model is trained to generate natural-sounding responses based on this data.

[0551] Emotional engine integration and emotion recognition

[0552] When a device receives a new email, it analyzes its content. During this process, the emotion engine analyzes the text to determine positive, negative, or neutral emotions and generates a corresponding emotion score. For example, in the case of a customer complaint email, the emotion engine identifies negative emotions and incorporates this result into the subsequent response generation process.

[0553] Category classification and generation of initial responses

[0554] Based on the analysis results of the emotion engine, the terminal classifies the email content into categories such as "internal consultation," "report / communication," "confirmation of sales activity requirements," and "customer inquiry." Based on this classification, the server refers to similar cases and templates in the database and generates an initial response email that reflects the emotion score.

[0555] Notification to the administrator and confirmation of response.

[0556] If a response is difficult or negative emotions are detected, the terminal automatically notifies the administrator. The server then sends an initial response email to the administrator, clearly indicating that negative emotions are present. This allows the user (administrator) to provide appropriate feedback on the negative emotions and modify the initial response.

[0557] Generation and transmission of the final response

[0558] Once administrator feedback is provided, the server automatically generates a final response email based on it. This final email is reviewed and revised by the administrator, and the system ensures it is sent to the recipient without fail. For example, if negative sentiment is detected in an email from a customer regarding a product defect, it provides guidance for the administrator to respond more carefully and courteously.

[0559] Specific example

[0560] For example, if an email arrives from another department requesting to schedule a visit next week, the terminal receives the email and categorizes it as a "scheduling request." If the sentiment analysis result is neutral, the server checks the schedule database and automatically generates an email suggesting several possible dates for the visit. If the sentiment engine detects a positive sentiment, the response email can include more friendly language.

[0561] Furthermore, if a customer sends an email asking for information about the specifications of product X, the terminal receives the email, determines its category to "customer inquiry," and the sentiment engine analyzes the message to see if it contains any negative sentiment. If negative sentiment is detected, the server automatically consults with the relevant department and prepares a detailed response. The user (administrator) reviews and revises the response options, and then sends the final answer to the customer based on that feedback.

[0562] This system significantly improves the efficiency of email correspondence and enables better communication by taking user emotions into consideration.

[0563] The following describes the processing flow.

[0564] Step 1:

[0565] The server collects internal company data such as past email data, inquiry history, business processes, and decision-making criteria, and stores this information in a database. Specifically, it extracts necessary information from internal document management systems and email servers, enabling centralized management of this data.

[0566] Step 2:

[0567] The server uses information from the database to train a generative AI model. This process involves providing the AI ​​model with pairs of email content and corresponding replies, training it to generate natural-sounding responses. This allows the model to acquire the ability to generate appropriate responses for various situations.

[0568] Step 3:

[0569] Each time the terminal receives a new email, it analyzes its contents. Using natural language processing technology, it extracts keywords and important contextual information from the email body and classifies the email into categories such as "internal consultation," "report / communication," "sales activity requirements confirmation," and "customer inquiry." This allows the appropriate response to be selected in the next processing step.

[0570] Step 4:

[0571] The device inputs the analyzed email content into an emotion engine, which determines whether the emotion is positive, negative, or neutral. The emotion engine calculates an emotion score from the text and incorporates the result into the next response generation process. Specifically, if the negative emotion is high, a response that takes this into account is generated.

[0572] Step 5:

[0573] The server generates an initial response email based on the aforementioned categories and sentiment engine results. It refers to similar cases and templates in the database, creating the initial response email using more friendly language for positive responses and polite and careful language for negative responses.

[0574] Step 6:

[0575] The server provides the terminal with an initial response email and notifies the administrator as needed. If the email content is complex or the emotion engine identifies negative emotions, the terminal automatically notifies the administrator.

[0576] Step 7:

[0577] The user (administrator) receives the notification and provides necessary corrections or additional information based on the initial response email and sentiment analysis results provided by the server. For emails containing negative sentiment, the administrator will create a more careful and appropriate response.

[0578] Step 8:

[0579] The server automatically generates a final response email based on the administrator's feedback and provides it to the terminal. Once the email content reflects the administrator's confirmation, a final check is performed.

[0580] Step 9:

[0581] The terminal performs a final check, including error checking and final verification, before automatically sending the email to the intended recipient.

[0582] Through this series of steps, the system automatically generates high-quality email responses that reflect the user's emotions, streamlining internal and external communication.

[0583] (Example 2)

[0584] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0585] Traditional email response systems often fail to consider user emotions, making it difficult to provide appropriate responses, particularly in handling emails containing negative emotions quickly and effectively. Furthermore, the time and effort required to manually categorize email content was a significant problem.

[0586] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0587] In this invention, the server includes means for collecting internal company data and building a database, means for training a generative AI model using the database, means for receiving emails, analyzing the content of the emails and generating sentiment scores, means for classifying email categories based on the sentiment scores, means for generating initial response emails based on the categories, means for notifying the administrator if the initial response email contains negative sentiments, and means for automatically generating and sending a final response email based on the administrator's feedback. This enables quick and appropriate email responses that take into account the user's sentiments.

[0588] "Internal data" refers to information generated or collected within a company or organization, and includes email data, inquiry history, business processes, decision-making criteria, etc.

[0589] A "database" refers to a system for systematically storing collected company data and for efficiently accessing and managing it.

[0590] A "generative AI model" refers to an artificial intelligence model that uses machine learning algorithms to generate natural-sounding responses based on the content of an email.

[0591] "Email" refers to digital messages exchanged over the internet, and can include text and attachments.

[0592] "Sentiment score" is a numerical or categorical representation of the user's emotions (positive, negative, neutral) as expressed in the content of an email.

[0593] A "category" is a group of emails classified based on their content, and examples include "internal consultation," "reporting / communication," "confirmation of sales activity requirements," and "customer inquiry."

[0594] An "initial response email" refers to the first reply email automatically generated by the system in response to an email that has been received.

[0595] An "administrator" refers to someone responsible for making important decisions and corrections in system operation, and is responsible for reviewing and correcting response emails automatically generated by the system.

[0596] A "final response email" refers to the final reply email generated based on the administrator's feedback.

[0597] This invention aims to realize an efficient email response system by collecting internal company data to build a database and training a generative AI model to provide an automated email response function that takes user sentiment into consideration. This system consists of a server, terminals, and users.

[0598] Data collection and database construction

[0599] The server aggregates data collected from various departments within the company, including email data, inquiry history, business processes, and decision-making criteria. The data is first saved to the local disk and then inserted into a database. This involves using the Python pandas library to read data in CSV format and then using SQLAlchemy to insert it into a PostgreSQL database. Finally, it executes queries necessary to ensure data integrity.

[0600] AI model training

[0601] The server trains a generative AI model using an integrated database. This AI model uses the Hugging Face transformers library to load a pre-prepared language model (e.g., BERT). Then, it feeds the training dataset extracted from the integrated database to the model and optimizes it using the PyTorch library.

[0602] Emotion analysis and emotion score generation

[0603] When the device receives a new email, it analyzes and understands its content. The email content is then analyzed using the TextBlob library to generate sentiment scores (positive, negative, or neutral). For example, if the prompt "I am very disappointed with this product" is analyzed, a negative sentiment score will be assigned.

[0604] Email category classification

[0605] Upon receiving an email, the terminal categorizes it based on its sentiment score and content. This classification is performed using Scikit-learn's Naive Bayes classifier, and the results are stored in a local database. Categories include "Internal Consultation," "Reporting / Communication," "Confirmation of Sales Activity Requirements," and "Customer Inquiry."

[0606] Generating an initial response email

[0607] The server generates an initial response email by referencing similar templates in the database based on the classified category and sentiment score. This process uses SQL queries to find the appropriate template and the Jinja2 template engine to populate it with sentiment score and category information.

[0608] Notification to the administrator

[0609] If the initial response email contains negative sentiment, the device automatically notifies the administrator. This notification sends the initial response email and the sentiment analysis results to the administrator via the SMTP server.

[0610] Administrator's feedback

[0611] The user (administrator) receives a notification, reviews the initial response email, and makes corrections as needed. These corrections are made through a web interface (e.g., a Django application), and feedback is sent to the server.

[0612] Generating and sending the final response email

[0613] Based on the administrator's feedback, the server generates a final response email. This generated email is automatically sent via the SMTP server.

[0614] Specific example

[0615] For example, if an email arrives from another department requesting to schedule a visit next week, the terminal receives the email and categorizes it as a "scheduling request." If the sentiment analysis is neutral, the server checks the schedule database and automatically generates an email suggesting several possible visit dates. Similarly, for an email requesting information about product X, the category is determined to be "customer inquiry." If the sentiment engine detects negative emotions, the server consults with the relevant department and prepares a detailed response. The user (administrator) reviews and revises the response options, and the final answer is sent to the customer based on their feedback.

[0616] This system significantly improves the efficiency of email correspondence and enables better communication by taking user emotions into consideration.

[0617] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0618] Step 1:

[0619] The server collects data from various departments within the company, including email data, inquiry history, business processes, and decision-making criteria. Specifically, it uses the Python pandas library to read data in CSV file format and saves it to the local disk.

[0620] Input: Data in CSV file format collected from each department.

[0621] Data processing: Read the CSV file using pandas and convert it to a DataFrame.

[0622] Output: Data stored in memory as a pandas dataframe

[0623] Step 2:

[0624] The server inserts the collected data into a PostgreSQL database using SQLAlchemy. During this process, it executes queries to maintain data integrity and sets necessary indexes.

[0625] Input: Data in pandas DataFrame format

[0626] Data processing: Inserting dataframes into a PostgreSQL database using SQLAlchemy.

[0627] Output: Data integrated into a PostgreSQL database

[0628] Step 3:

[0629] The server extracts training datasets from an integrated database and trains a generative AI model. Here, it uses the Hugging Face transformers library to load a pre-prepared BERT model. The PyTorch library is then used to optimize the model.

[0630] Input: Training dataset extracted from a PostgreSQL database

[0631] Data Calculation: Training an AI model using Hugging Face transformers and PyTorch

[0632] Output: Optimized generative AI model

[0633] Step 4:

[0634] The device receives a new email. Using the mail server (e.g., an IMAP server), it parses the received email with the TextBlob library and generates positive, negative, and neutral sentiment scores.

[0635] Input: Received email

[0636] Data processing: Sentiment analysis of email content using the TextBlob library

[0637] Output: Emotion score

[0638] Step 5:

[0639] The device categorizes received emails using a Naive Bayes classifier in Scikit-learn, based on their content and sentiment score. These categories are stored in a local database.

[0640] Input: Sentiment score and email content

[0641] Data processing: Categorical classification using Scikit-learn's Naive Bayes classifier.

[0642] Output: Category classification results

[0643] Step 6:

[0644] The server searches for similar templates in the database based on the classified category and sentiment score, and generates an initial response email using the Jinja2 template engine.

[0645] Input: Category classification results and sentiment score

[0646] Data processing: Search for templates using SQL queries and embed the information into the templates using Jinja2.

[0647] Output: Initial response email

[0648] Step 7:

[0649] If the initial response email contains negative sentiment, the device automatically notifies the administrator. The notification is sent using an SMTP server and includes the initial response email and the sentiment analysis results.

[0650] Input: Initial response email

[0651] Data processing: Send notification emails using an SMTP server.

[0652] Output: Notification email to administrator

[0653] Step 8:

[0654] The user (administrator) receives a notification, checks the initial response email, and makes corrections as needed via the web interface (e.g., a Django application). The corrected results are sent to the server.

[0655] Input: Notification email and web interface to administrator

[0656] Data processing: Administrator corrections and feedback submission

[0657] Output: Corrected response email feedback

[0658] Step 9:

[0659] The server generates a final response email based on the administrator's feedback. The generated email is automatically sent to the recipient via the SMTP server.

[0660] Input: Administrator's feedback

[0661] Data processing: Generate the final response email using the Jinja2 template engine.

[0662] Output: Sending of final response email

[0663] (Application Example 2)

[0664] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0665] In modern businesses, there is a need to process emails and other digital communications quickly and appropriately to improve the efficiency of customer service and internal communication. Furthermore, in the security field, it is necessary to utilize real-time sentiment recognition to detect abnormal behavior early and take swift, appropriate action. However, conventional systems struggle to address these challenges simultaneously, particularly lacking in the detection of abnormal behavior and rapid notification to administrators using sentiment recognition technology. Therefore, there is a need for the development of a system that comprehensively achieves both efficient email response and enhanced security monitoring.

[0666] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0667] In this invention, the server includes means for collecting internal company data and building a database, means for training a generative AI model using the database, means for receiving emails, analyzing the content of the emails and classifying them into categories, means for generating an initial response email based on the categories, means for having an administrator perform a detailed analysis of the inquiry content as needed, means for automatically generating and sending a final response email based on the administrator's feedback, means for detecting faces in real time and generating an emotion score using an emotion engine, means for suggesting countermeasures when abnormal emotions are detected, and means for detecting abnormal behavior based on the emotion score and providing notifications. This enables more efficient email handling and enhanced security monitoring.

[0668] "Internal company data" refers to various types of information and data generated within a company, including business processes, inquiry history, past email data, and decision-making criteria.

[0669] A "database" is a structured collection of data that systematically stores collected company data, making it easy to search and manipulate.

[0670] A "generative AI model" is an artificial intelligence model that learns from collected data and provides natural responses and inferences to new data.

[0671] "Email" refers to digital messages sent and received using computer networks such as the internet.

[0672] "Means of categorization" refers to algorithms or devices that analyze the content of received emails and sort them into predefined categories (e.g., internal consultation, report / communication).

[0673] An "initial response email" is the first reply email that is automatically generated in response to an email that has been received.

[0674] An "administrator" is a person responsible for the operation and supervision of a system, and is also responsible for analyzing system inquiries and providing feedback as needed.

[0675] A "final response email" is the final reply email that is automatically generated based on the administrator's feedback and sent to the recipient.

[0676] "Means for detecting faces" refers to technologies or devices that use cameras or other imaging devices to identify and recognize human faces within an image.

[0677] An "emotion engine" is an algorithm or system that analyzes emotions from detected facial expressions and generates emotion scores such as positive, negative, or neutral.

[0678] An "emotion score" is a numerical representation of the degree of emotion analyzed by the emotion engine.

[0679] "Abnormal emotions" refer to negative emotions that exceed certain standards, such as anger, fear, or disgust.

[0680] "Means of suggesting countermeasures" refers to a mechanism or system that, when abnormal emotions are detected, indicates appropriate actions or responses in accordance with the situation.

[0681] "Means of notification" refers to a system for promptly informing security personnel and administrators of any detected abnormal emotions or behaviors.

[0682] To implement this invention, the system is constructed according to the following steps.

[0683] Database construction and AI model training

[0684] The server collects internal company data and builds a database. This internal data includes past email data, inquiry history, business processes, and decision-making criteria. Based on this data, a generative AI model is trained. The training dataset includes email content and corresponding replies, and the AI ​​model is trained to generate natural-sounding responses based on this data.

[0685] Emotional engine integration and emotion recognition

[0686] When a device receives a new email, it analyzes its content. During this process, the emotion engine analyzes the text to determine positive, negative, or neutral emotions and generates an emotion score. For example, in the case of a "customer complaint email," negative emotions are identified, and this result is reflected in the subsequent response generation process.

[0687] Face detection and real-time emotion score generation

[0688] The device uses a camera to detect faces in its surroundings in real time. Face detection uses face recognition libraries such as OpenCV. The detected face images are analyzed by an emotion engine, and an emotion score is generated. This allows for suggested actions if abnormal emotions are detected.

[0689] Generation of initial response email and notification to administrator

[0690] The device categorizes emails based on their sentiment score and content, and generates an initial response email. This response email is automatically generated using a template. If an abnormal sentiment is detected, the device notifies the administrator, and if the situation is complex or negative sentiment is identified, the administrator is instructed to perform a detailed analysis.

[0691] Generating and sending the final response email

[0692] Based on administrator feedback, the server automatically generates a final response email. This final email is reviewed and corrected by the administrator, and the system sends it to the recipient.

[0693] Hardware and software to be used

[0694] Hardware: Camera devices, servers, terminals

[0695] Software: OpenCV (face detection), emotion engine (emotion analysis), database management system (management of internal company data)

[0696] Specific example:

[0697] Smart glasses are used as a security system in supermarkets and shopping malls to detect individuals exhibiting unusual emotions in real time. Alerts are sent to security guards, enabling a rapid response.

[0698] Example of a prompt:

[0699] Premise: In a supermarket, security guards are using smart glasses. A real-time emotion recognition and facial recognition system is needed to detect unusual emotions (anger, fear, disgust) and immediately notify the security team.

[0700] prompt:

[0701] 1. The facial recognition system detects customers' faces in the supermarket in real time and performs sentiment analysis.

[0702] 2. The emotion engine analyzes each customer's facial expressions and identifies customers who are experiencing emotions such as anger, fear, or disgust.

[0703] 3. If abnormal emotions are detected, the notification system will send an alert to the security team along with the log and facial image.

[0704] 4. The security team will use this information to respond quickly and ensure security.

[0705] Please create Python code to implement the process described above.

[0706] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0707] Step 1:

[0708] The server collects internal company data. It retrieves data such as past email data, inquiry history, business processes, and decision criteria, and stores it in a database. This database is used to build a training dataset for generative AI models. The input is internal company data, and the output is the constructed database.

[0709] Step 2:

[0710] The server uses a database to train a generative AI model. Email content and corresponding replies are used as the training dataset to train the AI ​​model to generate natural-sounding responses. The input is the training dataset in the database, and the output is the trained generative AI model.

[0711] Step 3:

[0712] The device receives a new email. It analyzes the email's content, uses an emotion engine to identify positive, negative, and neutral emotions, and generates an emotion score. The input is the received email, and the output is the emotion score. Specifically, it analyzes the email text and calculates the emotion score using the emotion engine.

[0713] Step 4:

[0714] The device categorizes emails based on their sentiment score and content. For example, it might categorize them as "Internal Consultation," "Report / Communication," "Confirmation of Sales Activity Requirements," or "Customer Inquiry." The input is the sentiment score and email content, and the output is the categorized email.

[0715] Step 5:

[0716] The terminal generates an initial response email based on the category. Using a template, it generates an initial response that includes appropriate wording according to the sentiment score. The input is the category and sentiment score, and the output is the initial response email.

[0717] Step 6:

[0718] If deemed necessary by the administrator, the terminal will prompt the administrator to perform a detailed analysis of the inquiry. The administrator will then provide feedback and optimize the content of the initial response email. The input is the initial response email, and the output is the administrator's feedback.

[0719] Step 7:

[0720] The server automatically generates a final response email based on the administrator's feedback. This final email is then reviewed and revised by the administrator before being sent to the recipient. The input is the administrator's feedback, and the output is the final response email.

[0721] Step 8:

[0722] The device uses a camera to detect faces in its surroundings in real time. The detected face images are analyzed by an emotion engine, and an emotion score is generated. The input is an image from the camera device, and the output is the face image and the emotion score.

[0723] Step 9:

[0724] Based on the emotion score, if abnormal emotions are detected, the device will suggest actions and send a notification. The notification will be sent to a designated security officer or administrator, requiring a prompt response. The input is the emotion score, and the output is a notification message. Specifically, it evaluates the emotion score and sends a notification according to the level of risk.

[0725] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0726] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0727] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0728] [Third Embodiment]

[0729] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0730] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0731] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0732] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0733] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0734] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0735] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0736] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0737] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0738] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0739] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0740] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0741] This invention relates to a system for automating internal data processing and email responses. This system functions collaboratively through the interaction of servers, terminals, and users. The program's processing is described below in natural language.

[0742] Database construction and AI model training

[0743] First, the server collects internal company data such as past email data, inquiry history, business processes, and decision criteria, and stores this information in a database. Next, the server uses this database to train a generative AI model. Specifically, by training the AI ​​with email content and appropriate response pairs, it becomes possible to generate natural-sounding responses.

[0744] Receiving and analyzing new emails

[0745] When a terminal receives a new email, it analyzes its content and categorizes it into categories such as "Internal Consultation," "Report / Communication," "Confirmation of Sales Activity Requirements," and "Customer Inquiry." To classify the email, natural language processing technology is used to understand its content and assign the appropriate category.

[0746] Generation of initial response

[0747] Next, the server generates an initial response email based on the category and content of the received email. In this process, it improves the quality of the response by referring to similar cases in the database and using appropriate templates (greeting, body, and closing). For example, if a customer inquires about product specifications, the server retrieves the relevant specification information from the product database and generates a response email.

[0748] Consult with the administrator (if necessary)

[0749] If a response is difficult or requires high accuracy, the terminal notifies the administrator to analyze the inquiry in detail. In this case, the server generates the most appropriate answer candidates and presents them to the user (administrator). The user reviews and modifies the answer candidates and provides feedback to the system.

[0750] Automated email creation and sending

[0751] Finally, the server receives feedback from the administrator and automatically generates a final response email. This email is checked for completeness and then automatically sent through the terminal.

[0752] Specific example: Request to arrange a visit

[0753] For example, if an email arrives from another department requesting to schedule a visit next week, the terminal receives the email and categorizes it as a "scheduling request." Next, the server checks the schedule database and automatically generates an email suggesting several possible dates for the visit. Finally, the terminal sends that email.

[0754] Specific example: Product inquiries from customers

[0755] Furthermore, when a customer sends an email requesting information about the specifications of product X, the terminal receives the email and identifies its category as "customer inquiry." The server retrieves the specifications of product X from the product database and generates an automated response email. If the response is incomplete, the terminal automatically consults with the relevant department to obtain a final answer. The server then creates a final email and sends it to the customer through the terminal.

[0756] By using the methods described above, internal email processing will be automated, significantly improving employee work efficiency.

[0757] The following describes the processing flow.

[0758] Step 1:

[0759] The server collects past email data, inquiry history, business processes, and decision-making criteria from within the company, and stores this information in a database. Specifically, it extracts necessary data from email sending and receiving logs and document management systems, organizes it, and registers it in the database.

[0760] Step 2:

[0761] The server uses information from the database to train a generative AI model (such as GPT-3). This involves providing the AI ​​model with pairs of email content and corresponding replies, and repeatedly training it to improve its ability to generate natural-sounding text.

[0762] Step 3:

[0763] Each time the terminal receives a new email, it analyzes its content. Using natural language processing technology, it extracts keywords and important contextual information from the email body and classifies the emails into categories such as "internal consultation," "report / communication," "confirmation of sales activity requirements," and "customer inquiry."

[0764] Step 4:

[0765] The server generates an initial response email based on the aforementioned categories. It refers to similar cases and templates in the database to construct the most appropriate response. For example, in response to a customer inquiry, it prepares the answer by referring to relevant product information and FAQs.

[0766] Step 5:

[0767] The server provides the terminal with an initial response email. If the initial response is insufficient to resolve the issue, or if a high level of accuracy is required, the terminal will notify the appropriate administrator.

[0768] Step 6:

[0769] The user (administrator) receives a notification and reviews and modifies the suggested answers provided by the server. If necessary, they enter additional information or supplementary explanations. The administrator's feedback is reflected in the system.

[0770] Step 7:

[0771] The server automatically generates a final response email based on feedback from the administrator. It then incorporates any corrections or additional information from the administrator to create a complete response.

[0772] Step 8:

[0773] The terminal checks the final response email, performs error checks and final confirmations, and then automatically sends the email to the intended recipient.

[0774] This series of processing steps automates email processing within the company, significantly reducing the effort and time required from employees.

[0775] (Example 1)

[0776] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0777] This invention relates to a system for automating internal data processing and email responses. Conventional systems suffered from long response times from email reception to response, resulting in poor efficiency. Furthermore, manual response creation was prone to human error, often leading to a lack of consistency and accuracy in response content. This resulted in decreased operational efficiency and reduced customer satisfaction.

[0778] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0779] In this invention, the server includes means for collecting internal company data and building a database, means for training a generative AI model using the database, means for receiving emails, analyzing the content of the emails and classifying them into categories, means for generating an initial response email based on the categories, means for generating a response email by referring to similar cases in the database, means for having an administrator perform a detailed analysis of the inquiry content as needed, and means for automatically generating and sending a final response email based on the administrator's feedback. This makes email processing more efficient and improves the consistency and accuracy of response content.

[0780] A "server" is a computer system that provides data and services to multiple computers and terminals over a network.

[0781] A "database" is a systematically organized collection of data, stored in a way that allows for efficient searching and updating.

[0782] A "generative AI model" is a type of artificial intelligence that learns from large amounts of data and performs tasks such as generating natural language and automatically generating responses.

[0783] "Natural language processing technology" refers to a set of technologies and methods that enable computers to understand, generate, and engage in human dialogue.

[0784] "Email" refers to digital messages sent and received via the internet or other computer networks.

[0785] A "category" is a way of classifying and grouping data based on certain common characteristics or attributes.

[0786] A "template" is a document model with a specific format or pattern, and it is a document format that allows for the easy generation of standardized documents by filling in the content.

[0787] "Feedback" refers to evaluations and corrections provided by users or administrators regarding responses and results generated by a system.

[0788] "IMAP" is an abbreviation for Internet Mail Access Protocol, and it is a protocol for managing email on a server.

[0789] POP3 is version 3 of the postal protocol and is an internet standard protocol used to receive email.

[0790] "MIME format" is an abbreviation for Multipurpose Internet Mail Extensions format, and it is a format that allows emails to contain data other than text (such as images, audio, and video).

[0791] "JSON format" is an abbreviation for JavaScript Object Notation format, and is a lightweight data description format primarily used for data exchange.

[0792] This invention relates to a system for automating internal data processing and email responses. This system functions collaboratively through the interaction of servers, terminals, and users.

[0793] First, the server collects data such as past email data, inquiry history, business processes, and decision criteria from within the company, and stores this information in a database. Possible email servers to use include Exchange Server and G Suite. The collected data is saved in CSV or JSON format, and database management systems such as MySQL or MongoDB are used for the database. Based on this database, the server trains a generative AI model (for example, OpenAI's GPT-3). Email content and appropriate response pairs are used as training data.

[0794] Next, the device receives new emails. Protocols such as IMAP and POP3 are used to receive emails. The content of the received emails is analyzed using natural language processing technology (e.g., Google Cloud Natural Language API), and the emails are categorized into categories such as "internal consultation," "report / communication," "sales activity requirements confirmation," and "customer inquiry."

[0795] Based on the aforementioned category and email content, the server refers to similar cases in the database and generates an initial response email. The generated response email includes a greeting, body, and closing. For example, in response to an inquiry such as "Please tell me about the specifications of product X," the server retrieves the relevant specification information from the product database and generates a response. This response email is constructed in MIME format.

[0796] If a response is difficult or requires high accuracy, the terminal notifies the administrator for detailed analysis. The administrator is notified using tools such as Jira or Slack, and the server generates optimal response candidates and presents them to the user (administrator). The administrator reviews these response candidates, makes corrections as needed, and sends feedback back to the system in JSON format.

[0797] Based on this feedback, the server automatically generates a final response email. The generated email is then automatically sent through the device. Email sending APIs such as SendGrid and AWS SES are used to send the email.

[0798] Specific example: Request to arrange a visit

[0799] For example, if an email arrives from another department requesting to schedule a visit next week, the device receives this email and interprets it as a "scheduling request." Next, the server checks its schedule database (e.g., Google Calendar) and automatically generates an email suggesting several possible dates for the visit. Finally, the device sends that email.

[0800] Specific example: Product inquiries from customers

[0801] Furthermore, when a customer sends an email requesting information about the specifications of product X, the terminal receives the email and identifies it as a customer inquiry. The server retrieves the specifications of product X from the product database and generates an automated response email. If the response is incomplete, the terminal automatically consults with the relevant department to obtain a final answer. Once this process is complete, the server creates a final email and sends it to the customer through the terminal.

[0802] Example of a prompt

[0803] "Inquiry: Please tell me about the specifications of product X."

[0804] "Category: Customer Inquiry"

[0805] By using the methods described above, this system can streamline internal email processing and maintain consistency and accuracy in responses.

[0806] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0807] Step 1:

[0808] The server collects internal company data. Specifically, it extracts past email data and inquiry history from mail servers such as Exchange Server and G Suite in CSV or JSON format. The input is data from the mail server, and the output is a structured data file.

[0809] Step 2:

[0810] The server stores the collected data in a database. The collected email data and inquiry history are saved in a database management system such as MySQL or MongoDB. The input is the data file generated in step 1, and the output is the records in the database.

[0811] Step 3:

[0812] The server uses a database to train a generative AI model. It uses pre-processed text data (e.g., OpenAI's GPT-3) to train the generative AI model. The input is text data from the database, and the output is the trained AI model.

[0813] Step 4:

[0814] The device receives a new email. Protocols such as IMAP and POP3 are used to receive the email. The input is the new email from the mail server, and the output is the email data stored on the device's local system.

[0815] Step 5:

[0816] The terminal analyzes the content of emails and categorizes them. Using natural language processing technologies such as the Google Cloud Natural Language API, emails are classified into categories such as "Internal Consultation," "Report / Communication," "Confirmation of Sales Activity Requirements," and "Customer Inquiry." The input is the email data received in step 4, and the output is the analysis results including category information.

[0817] Step 6:

[0818] The server generates an initial response email based on the aforementioned categories. It references similar cases from the database and constructs the response email using an appropriate template. The input is the analysis results from step 5 and similar case data from the database, and the output is the initial response email.

[0819] Step 7:

[0820] If the terminal has difficulty responding or if high accuracy is required, the administrator will be asked to perform a detailed analysis of the inquiry. A notification will be sent to the administrator using tools such as Jira or Slack. The input is the initial response email and its difficulty assessment result, and the output is the notification to the administrator.

[0821] Step 8:

[0822] The server generates the most suitable answer candidates for the administrator. The generated answers are displayed in an HTML email or a dedicated dashboard. The input is the notification content from step 7, and the output is the answer candidates presented to the administrator.

[0823] Step 9:

[0824] The user (administrator) reviews the suggested answers and sends back feedback. The feedback is sent back to the system in JSON format. The input is the suggested answers presented in step 8, and the output is the administrator's feedback.

[0825] Step 10:

[0826] The server automatically generates a final response email based on feedback from the administrator. The generated email is constructed in MIME format. The input is the feedback obtained in step 9, and the output is the final response email.

[0827] Step 11:

[0828] The terminal automatically sends the final response email that was generated. Email sending APIs such as SendGrid or AWS SES are used to send the email. The input is the final response email generated in step 10, and the output is the sent email.

[0829] (Application Example 1)

[0830] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0831] In conventional factories, managers had to manually check and respond to work instructions and reports via email, a process that wasted time and effort. Furthermore, it was difficult to process abnormal reports and adjustment requests in a timely manner, leading to decreased operational efficiency and delays in responses. This invention aims to solve these problems and improve operational efficiency by automating data processing and email responses within the factory.

[0832] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0833] In this invention, the server includes means for collecting internal company data and building a database, means for training a generation AI model using the database, means for receiving emails, analyzing the content of the emails and classifying them into categories, means for a robot terminal to receive new emails, analyze their content and classify them into work instructions or report categories, means for having an administrator perform a detailed analysis of the inquiry content as needed, and means for automatically generating and sending a final response email based on the administrator's feedback. This automates the processing of work instructions and report emails and enables rapid response to anomaly reports and adjustment requests.

[0834] "Internal company data" refers to all information generated within a company, including past email data, inquiry history, business processes, decision-making criteria, and so on.

[0835] A "database" is a collection of electronic information that systematically organizes and stores company data, and manages it in a way that allows access as needed.

[0836] A "generative AI model" refers to an artificial intelligence model that uses information stored in a database to produce appropriate responses and processes, utilizing machine learning and deep learning.

[0837] "Email" refers to electronic messages sent and received via the internet or other electronic communication networks.

[0838] A "category" refers to a group or type of email classified based on its content or nature, and examples include "internal consultation" and "anomaly report."

[0839] An "initial response email" refers to the first response email automatically generated based on an AI model or template in response to a newly received email.

[0840] A "robot terminal" is a device that performs actual physical tasks in factories and other facilities, and has the ability to receive and analyze emails.

[0841] An "administrator" is a person or role responsible for overseeing and coordinating the system, and provides detailed analysis of email content and feedback as needed.

[0842] "Work instructions" are documents that describe the procedures and instructions for specific tasks or operations, and are provided to robot terminals or employees.

[0843] A "progress report" refers to a report that explains the status of ongoing work or projects, and is sent to the administrator via email or other means.

[0844] A "request for adjustment" refers to a request for changes or adjustments to the schedule or working conditions.

[0845] An "anomaly report" refers to a report of a problem or abnormal event that occurred within a factory or system, and it provides detailed information about the incident.

[0846] This invention relates to a system for automating data processing and email responses within a factory. In this system, servers, terminals, and users work together. Specific embodiments of the invention are described in detail below.

[0847] First, the server collects internal company data generated within the factory and builds a database. This data includes past email data, inquiry history, business processes, and decision-making criteria. A database like PostgreSQL might be used.

[0848] Next, the server uses this database to train a generative AI model. Machine learning and deep learning techniques are used for training, such as OpenAI's GPT-3.5. The AI ​​model learns past email content and appropriate response pairs to enable it to generate natural-sounding responses.

[0849] When the terminal receives a new email, it analyzes its content and classifies it into categories such as "internal consultation," "report / communication," "confirmation of sales activity requirements," "customer inquiry," or "work instructions," "progress report," "coordination request," or "anomaly report." Natural language processing technology (e.g., SpaCy) is used for the analysis.

[0850] The server generates an initial response email based on the category and content of the received email. In doing so, it improves the quality of the response by referring to similar cases in the database and using appropriate templates (greeting, body, and closing). For example, for scheduling requests, it refers to the schedule database and automatically generates an email suggesting several possible dates for a visit.

[0851] If a response is difficult or requires high accuracy, the terminal notifies the administrator to analyze the inquiry in detail. In this case, the server generates the most appropriate answer candidates and presents them to the user (administrator). The user reviews and modifies the answer candidates and provides feedback to the system.

[0852] The server receives feedback from the administrator and automatically generates a final response email. This email is checked for completeness and then automatically sent through the terminal.

[0853] As a concrete example, the following is a case of an abnormal report.

[0854] The database stores a history of abnormal occurrences, past response results, and detailed information about the work area. When the robot terminal detects an abnormality in the equipment, it automatically generates the following prompt message and sends it to the server.

[0855] Prompt message:

[0856] Generate an anomaly detection report email.

[0857] Location of abnormality: "Robot arm A3"

[0858] Problem description: "Motor overheating"

[0859] Date of occurrence: YYYY / MM / DD

[0860] Required action: "Prompt inspection and replacement"

[0861] Based on past cases of handling anomalies, generate a detailed report email.

[0862] The server uses this prompt to generate a detailed report email from the AI ​​model. As a result, an email like the following is sent to the administrator.

[0863] Result email:

[0864] Subject: Regarding motor overheating in robot arm A3

[0865] Main text:

[0866] Dear Administrator,

[0867] An overheating issue has been detected in the A3 robot arm. Please see the details below.

[0868] Location of abnormality: Robot arm A3

[0869] Problem description: Motor overheating

[0870] Date of occurrence: YYYY / MM / DD

[0871] Required action: Prompt inspection and replacement

[0872] Based on similar cases in the past, prompt action is required. Please review the matter.

[0873] From factory automation systems

[0874] In this way, data processing and email responses within the factory are automated, significantly improving operational efficiency.

[0875] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0876] Step 1:

[0877] The server collects internal company data generated within the factory, such as past email data, inquiry history, business processes, and decision-making criteria. A database is built to integrate and manage this data. Various internal company data is provided as input, and an integrated database is generated as output. This database is implemented using, for example, PostgreSQL.

[0878] Step 2:

[0879] The server uses the constructed database to train a generative AI model. This training uses email content and appropriate response pairs, for example, OpenAI's GPT-3.5 protocol. The input consists of past email data and example responses from the database, and the output is a generative AI model. This model is used to generate natural-sounding email responses.

[0880] Step 3:

[0881] The terminal receives a new email and analyzes its content. Using natural language processing technology (e.g., SpaCy), it analyzes the email content and classifies it into categories such as "Internal Consultation," "Report / Communication," "Confirmation of Sales Activity Requirements," "Customer Inquiry," "Work Instructions," "Progress Report," "Adjustment Request," and "Anomaly Report." A new email is provided as input, and a category is assigned as output.

[0882] Step 4:

[0883] The server generates an initial response email based on the category and content of the received email. In doing so, it refers to similar cases in the database and uses the appropriate template. Given the email category and its content as input, the initial response email is generated as output. For example, for a scheduling request email, it refers to the schedule database and generates an email suggesting available dates for a visit.

[0884] Step 5:

[0885] If the terminal has difficulty responding or requires high accuracy, it notifies the administrator to perform a detailed analysis of the inquiry. In this case, the server generates the most suitable answer candidates and presents them to the user (administrator). The input is an email requiring detailed analysis, and the output is the generation of answer candidates to present to the administrator.

[0886] Step 6:

[0887] The user (administrator) reviews the suggested answer options and makes corrections as needed. The corrections are fed back into the system. Feedback from the administrator is provided as input, and the corrected answer is obtained as output.

[0888] Step 7:

[0889] The server receives feedback from the administrator and automatically generates a final response email. This email is checked for completeness and automatically sent through the terminal. The corrected response is given as input, and the final response email is generated as output.

[0890] Step 8:

[0891] When the terminal detects an anomaly, it generates a prompt message and sends it to the server. The server uses the prompt message to create a detailed report email from the AI ​​model and sends it to the administrator. As a specific example, the prompt message when the robot terminal detects an anomaly is as follows:

[0892] Prompt message:

[0893] Generate an anomaly detection report email.

[0894] Location of abnormality: "Robot arm A3"

[0895] Problem description: "Motor overheating"

[0896] Date of occurrence: YYYY / MM / DD

[0897] Required action: "Prompt inspection and replacement"

[0898] Based on past cases of handling anomalies, generate a detailed report email.

[0899] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0900] This invention relates to a system that collects internal company data, builds a database, and trains a generative AI model. Furthermore, this system incorporates an emotion engine that recognizes user emotions, adding a function to reflect user emotions in email content and generate more appropriate initial responses. The specific processing of the program of this system will be described below.

[0901] Database construction and AI model training

[0902] The server acquires new data and integrates it into the existing database. This includes historical email data, inquiry history, business processes, and decision criteria. The constructed database is used to train a generative AI model. The training dataset includes email content and corresponding replies, and the AI ​​model is trained to generate natural-sounding responses based on this data.

[0903] Emotional engine integration and emotion recognition

[0904] When a device receives a new email, it analyzes its content. During this process, the emotion engine analyzes the text to determine positive, negative, or neutral emotions and generates a corresponding emotion score. For example, in the case of a customer complaint email, the emotion engine identifies negative emotions and incorporates this result into the subsequent response generation process.

[0905] Category classification and generation of initial responses

[0906] Based on the analysis results of the emotion engine, the terminal classifies the email content into categories such as "internal consultation," "report / communication," "confirmation of sales activity requirements," and "customer inquiry." Based on this classification, the server refers to similar cases and templates in the database and generates an initial response email that reflects the emotion score.

[0907] Notification to the administrator and confirmation of response.

[0908] If a response is difficult or negative emotions are detected, the terminal automatically notifies the administrator. The server then sends an initial response email to the administrator, clearly indicating that negative emotions are present. This allows the user (administrator) to provide appropriate feedback on the negative emotions and modify the initial response.

[0909] Generation and transmission of the final response

[0910] Once administrator feedback is provided, the server automatically generates a final response email based on it. This final email is reviewed and revised by the administrator, and the system ensures it is sent to the recipient without fail. For example, if negative sentiment is detected in an email from a customer regarding a product defect, it provides guidance for the administrator to respond more carefully and courteously.

[0911] Specific example

[0912] For example, if an email arrives from another department requesting to schedule a visit next week, the terminal receives the email and categorizes it as a "scheduling request." If the sentiment analysis result is neutral, the server checks the schedule database and automatically generates an email suggesting several possible dates for the visit. If the sentiment engine detects a positive sentiment, the response email can include more friendly language.

[0913] Furthermore, if a customer sends an email asking for information about the specifications of product X, the terminal receives the email, determines its category to "customer inquiry," and the sentiment engine analyzes the message to see if it contains any negative sentiment. If negative sentiment is detected, the server automatically consults with the relevant department and prepares a detailed response. The user (administrator) reviews and revises the response options, and then sends the final answer to the customer based on that feedback.

[0914] This system significantly improves the efficiency of email correspondence and enables better communication by taking user emotions into consideration.

[0915] The following describes the processing flow.

[0916] Step 1:

[0917] The server collects internal company data such as past email data, inquiry history, business processes, and decision-making criteria, and stores this information in a database. Specifically, it extracts necessary information from internal document management systems and email servers, enabling centralized management of this data.

[0918] Step 2:

[0919] The server uses information from the database to train a generative AI model. This process involves providing the AI ​​model with pairs of email content and corresponding replies, training it to generate natural-sounding responses. This allows the model to acquire the ability to generate appropriate responses for various situations.

[0920] Step 3:

[0921] Each time the terminal receives a new email, it analyzes its contents. Using natural language processing technology, it extracts keywords and important contextual information from the email body and classifies the email into categories such as "internal consultation," "report / communication," "sales activity requirements confirmation," and "customer inquiry." This allows the appropriate response to be selected in the next processing step.

[0922] Step 4:

[0923] The device inputs the analyzed email content into an emotion engine, which determines whether the emotion is positive, negative, or neutral. The emotion engine calculates an emotion score from the text and incorporates the result into the next response generation process. Specifically, if the negative emotion is high, a response that takes this into account is generated.

[0924] Step 5:

[0925] The server generates an initial response email based on the aforementioned categories and sentiment engine results. It refers to similar cases and templates in the database, creating the initial response email using more friendly language for positive responses and polite and careful language for negative responses.

[0926] Step 6:

[0927] The server provides the terminal with an initial response email and notifies the administrator as needed. If the email content is complex or the emotion engine identifies negative emotions, the terminal automatically notifies the administrator.

[0928] Step 7:

[0929] The user (administrator) receives the notification and provides necessary corrections or additional information based on the initial response email and sentiment analysis results provided by the server. For emails containing negative sentiment, the administrator will create a more careful and appropriate response.

[0930] Step 8:

[0931] The server automatically generates a final response email based on the administrator's feedback and provides it to the terminal. Once the email content reflects the administrator's confirmation, a final check is performed.

[0932] Step 9:

[0933] The terminal performs a final check, including error checking and final verification, before automatically sending the email to the intended recipient.

[0934] Through this series of steps, the system automatically generates high-quality email responses that reflect the user's emotions, streamlining internal and external communication.

[0935] (Example 2)

[0936] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0937] Traditional email response systems often fail to consider user emotions, making it difficult to provide appropriate responses, particularly in handling emails containing negative emotions quickly and effectively. Furthermore, the time and effort required to manually categorize email content was a significant problem.

[0938] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0939] In this invention, the server includes means for collecting internal company data and building a database, means for training a generative AI model using the database, means for receiving emails, analyzing the content of the emails and generating sentiment scores, means for classifying email categories based on the sentiment scores, means for generating initial response emails based on the categories, means for notifying the administrator if the initial response email contains negative sentiments, and means for automatically generating and sending a final response email based on the administrator's feedback. This enables quick and appropriate email responses that take into account the user's sentiments.

[0940] "Internal data" refers to information generated or collected within a company or organization, and includes email data, inquiry history, business processes, decision-making criteria, etc.

[0941] A "database" refers to a system for systematically storing collected company data and for efficiently accessing and managing it.

[0942] A "generative AI model" refers to an artificial intelligence model that uses machine learning algorithms to generate natural-sounding responses based on the content of an email.

[0943] "Email" refers to digital messages exchanged over the internet, and can include text and attachments.

[0944] "Sentiment score" is a numerical or categorical representation of the user's emotions (positive, negative, neutral) as expressed in the content of an email.

[0945] A "category" is a group of emails classified based on their content, and examples include "internal consultation," "reporting / communication," "confirmation of sales activity requirements," and "customer inquiry."

[0946] An "initial response email" refers to the first reply email automatically generated by the system in response to an email that has been received.

[0947] An "administrator" refers to someone responsible for making important decisions and corrections in system operation, and is responsible for reviewing and correcting response emails automatically generated by the system.

[0948] A "final response email" refers to the final reply email generated based on the administrator's feedback.

[0949] This invention aims to realize an efficient email response system by collecting internal company data to build a database and training a generative AI model to provide an automated email response function that takes user sentiment into consideration. This system consists of a server, terminals, and users.

[0950] Data collection and database construction

[0951] The server aggregates data collected from various departments within the company, including email data, inquiry history, business processes, and decision-making criteria. The data is first saved to the local disk and then inserted into a database. This involves using the Python pandas library to read data in CSV format and then using SQLAlchemy to insert it into a PostgreSQL database. Finally, it executes queries necessary to ensure data integrity.

[0952] AI model training

[0953] The server trains a generative AI model using an integrated database. This AI model uses the Hugging Face transformers library to load a pre-prepared language model (e.g., BERT). Then, it feeds the training dataset extracted from the integrated database to the model and optimizes it using the PyTorch library.

[0954] Emotion analysis and emotion score generation

[0955] When the device receives a new email, it analyzes and understands its content. The email content is then analyzed using the TextBlob library to generate sentiment scores (positive, negative, or neutral). For example, if the prompt "I am very disappointed with this product" is analyzed, a negative sentiment score will be assigned.

[0956] Email category classification

[0957] Upon receiving an email, the terminal categorizes it based on its sentiment score and content. This classification is performed using Scikit-learn's Naive Bayes classifier, and the results are stored in a local database. Categories include "Internal Consultation," "Reporting / Communication," "Confirmation of Sales Activity Requirements," and "Customer Inquiry."

[0958] Generating an initial response email

[0959] The server generates an initial response email by referencing similar templates in the database based on the classified category and sentiment score. This process uses SQL queries to find the appropriate template and the Jinja2 template engine to populate it with sentiment score and category information.

[0960] Notification to the administrator

[0961] If the initial response email contains negative sentiment, the device automatically notifies the administrator. This notification sends the initial response email and the sentiment analysis results to the administrator via the SMTP server.

[0962] Administrator's feedback

[0963] The user (administrator) receives a notification, reviews the initial response email, and makes corrections as needed. These corrections are made through a web interface (e.g., a Django application), and feedback is sent to the server.

[0964] Generating and sending the final response email

[0965] Based on the administrator's feedback, the server generates a final response email. This generated email is automatically sent via the SMTP server.

[0966] Specific example

[0967] For example, if an email arrives from another department requesting to schedule a visit next week, the terminal receives the email and categorizes it as a "scheduling request." If the sentiment analysis is neutral, the server checks the schedule database and automatically generates an email suggesting several possible visit dates. Similarly, for an email requesting information about product X, the category is determined to be "customer inquiry." If the sentiment engine detects negative emotions, the server consults with the relevant department and prepares a detailed response. The user (administrator) reviews and revises the response options, and the final answer is sent to the customer based on their feedback.

[0968] This system significantly improves the efficiency of email correspondence and enables better communication by taking user emotions into consideration.

[0969] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0970] Step 1:

[0971] The server collects data from various departments within the company, including email data, inquiry history, business processes, and decision-making criteria. Specifically, it uses the Python pandas library to read data in CSV file format and saves it to the local disk.

[0972] Input: Data in CSV file format collected from each department.

[0973] Data processing: Read the CSV file using pandas and convert it to a DataFrame.

[0974] Output: Data stored in memory as a pandas dataframe

[0975] Step 2:

[0976] The server inserts the collected data into a PostgreSQL database using SQLAlchemy. During this process, it executes queries to maintain data integrity and sets necessary indexes.

[0977] Input: Data in pandas DataFrame format

[0978] Data processing: Inserting dataframes into a PostgreSQL database using SQLAlchemy.

[0979] Output: Data integrated into a PostgreSQL database

[0980] Step 3:

[0981] The server extracts training datasets from an integrated database and trains a generative AI model. Here, it uses the Hugging Face transformers library to load a pre-prepared BERT model. The PyTorch library is then used to optimize the model.

[0982] Input: Training dataset extracted from a PostgreSQL database

[0983] Data Calculation: Training an AI model using Hugging Face transformers and PyTorch

[0984] Output: Optimized generative AI model

[0985] Step 4:

[0986] The device receives a new email. Using the mail server (e.g., an IMAP server), it parses the received email with the TextBlob library and generates positive, negative, and neutral sentiment scores.

[0987] Input: Received email

[0988] Data processing: Sentiment analysis of email content using the TextBlob library

[0989] Output: Emotion score

[0990] Step 5:

[0991] The device categorizes received emails using a Naive Bayes classifier in Scikit-learn, based on their content and sentiment score. These categories are stored in a local database.

[0992] Input: Sentiment score and email content

[0993] Data processing: Categorical classification using Scikit-learn's Naive Bayes classifier.

[0994] Output: Category classification results

[0995] Step 6:

[0996] The server searches for similar templates in the database based on the classified category and sentiment score, and generates an initial response email using the Jinja2 template engine.

[0997] Input: Category classification results and sentiment score

[0998] Data processing: Search for templates using SQL queries and embed the information into the templates using Jinja2.

[0999] Output: Initial response email

[1000] Step 7:

[1001] If the initial response email contains negative sentiment, the device automatically notifies the administrator. The notification is sent using an SMTP server and includes the initial response email and the sentiment analysis results.

[1002] Input: Initial response email

[1003] Data processing: Send notification emails using an SMTP server.

[1004] Output: Notification email to administrator

[1005] Step 8:

[1006] The user (administrator) receives a notification, checks the initial response email, and makes corrections as needed via the web interface (e.g., a Django application). The corrected results are sent to the server.

[1007] Input: Notification email and web interface to administrator

[1008] Data processing: Administrator corrections and feedback submission

[1009] Output: Corrected response email feedback

[1010] Step 9:

[1011] The server generates a final response email based on the administrator's feedback. The generated email is automatically sent to the recipient via the SMTP server.

[1012] Input: Administrator's feedback

[1013] Data processing: Generate the final response email using the Jinja2 template engine.

[1014] Output: Sending of final response email

[1015] (Application Example 2)

[1016] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1017] In modern businesses, there is a need to process emails and other digital communications quickly and appropriately to improve the efficiency of customer service and internal communication. Furthermore, in the security field, it is necessary to utilize real-time sentiment recognition to detect abnormal behavior early and take swift, appropriate action. However, conventional systems struggle to address these challenges simultaneously, particularly lacking in the detection of abnormal behavior and rapid notification to administrators using sentiment recognition technology. Therefore, there is a need for the development of a system that comprehensively achieves both efficient email response and enhanced security monitoring.

[1018] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1019] In this invention, the server includes means for collecting internal company data and building a database, means for training a generative AI model using the database, means for receiving emails, analyzing the content of the emails and classifying them into categories, means for generating an initial response email based on the categories, means for having an administrator perform a detailed analysis of the inquiry content as needed, means for automatically generating and sending a final response email based on the administrator's feedback, means for detecting faces in real time and generating an emotion score using an emotion engine, means for suggesting countermeasures when abnormal emotions are detected, and means for detecting abnormal behavior based on the emotion score and providing notifications. This enables more efficient email handling and enhanced security monitoring.

[1020] "Internal company data" refers to various types of information and data generated within a company, including business processes, inquiry history, past email data, and decision-making criteria.

[1021] A "database" is a structured collection of data that systematically stores collected company data, making it easy to search and manipulate.

[1022] A "generative AI model" is an artificial intelligence model that learns from collected data and provides natural responses and inferences to new data.

[1023] "Email" refers to digital messages sent and received using computer networks such as the internet.

[1024] "Means of categorization" refers to algorithms or devices that analyze the content of received emails and sort them into predefined categories (e.g., internal consultation, report / communication).

[1025] An "initial response email" is the first reply email that is automatically generated in response to an email that has been received.

[1026] An "administrator" is a person responsible for the operation and supervision of a system, and is also responsible for analyzing system inquiries and providing feedback as needed.

[1027] A "final response email" is the final reply email that is automatically generated based on the administrator's feedback and sent to the recipient.

[1028] "Means for detecting faces" refers to technologies or devices that use cameras or other imaging devices to identify and recognize human faces within an image.

[1029] An "emotion engine" is an algorithm or system that analyzes emotions from detected facial expressions and generates emotion scores such as positive, negative, or neutral.

[1030] An "emotion score" is a numerical representation of the degree of emotion analyzed by the emotion engine.

[1031] "Abnormal emotions" refer to negative emotions that exceed certain standards, such as anger, fear, or disgust.

[1032] "Means of suggesting countermeasures" refers to a mechanism or system that, when abnormal emotions are detected, indicates appropriate actions or responses in accordance with the situation.

[1033] "Means of notification" refers to a system for promptly informing security personnel and administrators of any detected abnormal emotions or behaviors.

[1034] To implement this invention, the system is constructed according to the following steps.

[1035] Database construction and AI model training

[1036] The server collects internal company data and builds a database. This internal data includes past email data, inquiry history, business processes, and decision-making criteria. Based on this data, a generative AI model is trained. The training dataset includes email content and corresponding replies, and the AI ​​model is trained to generate natural-sounding responses based on this data.

[1037] Emotional engine integration and emotion recognition

[1038] When a device receives a new email, it analyzes its content. During this process, the emotion engine analyzes the text to determine positive, negative, or neutral emotions and generates an emotion score. For example, in the case of a "customer complaint email," negative emotions are identified, and this result is reflected in the subsequent response generation process.

[1039] Face detection and real-time emotion score generation

[1040] The device uses a camera to detect faces in its surroundings in real time. Face detection uses face recognition libraries such as OpenCV. The detected face images are analyzed by an emotion engine, and an emotion score is generated. This allows for suggested actions if abnormal emotions are detected.

[1041] Generation of initial response email and notification to administrator

[1042] The device categorizes emails based on their sentiment score and content, and generates an initial response email. This response email is automatically generated using a template. If an abnormal sentiment is detected, the device notifies the administrator, and if the situation is complex or negative sentiment is identified, the administrator is instructed to perform a detailed analysis.

[1043] Generating and sending the final response email

[1044] Based on administrator feedback, the server automatically generates a final response email. This final email is reviewed and corrected by the administrator, and the system sends it to the recipient.

[1045] Hardware and software to be used

[1046] Hardware: Camera devices, servers, terminals

[1047] Software: OpenCV (face detection), emotion engine (emotion analysis), database management system (management of internal company data)

[1048] Specific example:

[1049] Smart glasses are used as a security system in supermarkets and shopping malls to detect individuals exhibiting unusual emotions in real time. Alerts are sent to security guards, enabling a rapid response.

[1050] Example of a prompt:

[1051] Premise: In a supermarket, security guards are using smart glasses. A real-time emotion recognition and facial recognition system is needed to detect unusual emotions (anger, fear, disgust) and immediately notify the security team.

[1052] prompt:

[1053] 1. The facial recognition system detects customers' faces in the supermarket in real time and performs sentiment analysis.

[1054] 2. The emotion engine analyzes each customer's facial expressions and identifies customers who are experiencing emotions such as anger, fear, or disgust.

[1055] 3. If abnormal emotions are detected, the notification system will send an alert to the security team along with the log and facial image.

[1056] 4. The security team will use this information to respond quickly and ensure security.

[1057] Please create Python code to implement the process described above.

[1058] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1059] Step 1:

[1060] The server collects internal company data. It retrieves data such as past email data, inquiry history, business processes, and decision criteria, and stores it in a database. This database is used to build a training dataset for generative AI models. The input is internal company data, and the output is the constructed database.

[1061] Step 2:

[1062] The server uses a database to train a generative AI model. Email content and corresponding replies are used as the training dataset to train the AI ​​model to generate natural-sounding responses. The input is the training dataset in the database, and the output is the trained generative AI model.

[1063] Step 3:

[1064] The device receives a new email. It analyzes the email's content, uses an emotion engine to identify positive, negative, and neutral emotions, and generates an emotion score. The input is the received email, and the output is the emotion score. Specifically, it analyzes the email text and calculates the emotion score using the emotion engine.

[1065] Step 4:

[1066] The device categorizes emails based on their sentiment score and content. For example, it might categorize them as "Internal Consultation," "Report / Communication," "Confirmation of Sales Activity Requirements," or "Customer Inquiry." The input is the sentiment score and email content, and the output is the categorized email.

[1067] Step 5:

[1068] The terminal generates an initial response email based on the category. Using a template, it generates an initial response that includes appropriate wording according to the sentiment score. The input is the category and sentiment score, and the output is the initial response email.

[1069] Step 6:

[1070] If deemed necessary by the administrator, the terminal will prompt the administrator to perform a detailed analysis of the inquiry. The administrator will then provide feedback and optimize the content of the initial response email. The input is the initial response email, and the output is the administrator's feedback.

[1071] Step 7:

[1072] The server automatically generates a final response email based on the administrator's feedback. This final email is then reviewed and revised by the administrator before being sent to the recipient. The input is the administrator's feedback, and the output is the final response email.

[1073] Step 8:

[1074] The device uses a camera to detect faces in its surroundings in real time. The detected face images are analyzed by an emotion engine, and an emotion score is generated. The input is an image from the camera device, and the output is the face image and the emotion score.

[1075] Step 9:

[1076] Based on the emotion score, if abnormal emotions are detected, the device will suggest actions and send a notification. The notification will be sent to a designated security officer or administrator, requiring a prompt response. The input is the emotion score, and the output is a notification message. Specifically, it evaluates the emotion score and sends a notification according to the level of risk.

[1077] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1078] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1079] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1080] [Fourth Embodiment]

[1081] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1082] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1083] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1084] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1085] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1086] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1087] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1088] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1089] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1090] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1091] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1092] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1093] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1094] This invention relates to a system for automating internal data processing and email responses. This system functions collaboratively through the interaction of servers, terminals, and users. The program's processing is described below in natural language.

[1095] Database construction and AI model training

[1096] First, the server collects internal company data such as past email data, inquiry history, business processes, and decision criteria, and stores this information in a database. Next, the server uses this database to train a generative AI model. Specifically, by training the AI ​​with email content and appropriate response pairs, it becomes possible to generate natural-sounding responses.

[1097] Receiving and analyzing new emails

[1098] When a terminal receives a new email, it analyzes its content and categorizes it into categories such as "Internal Consultation," "Report / Communication," "Confirmation of Sales Activity Requirements," and "Customer Inquiry." To classify the email, natural language processing technology is used to understand its content and assign the appropriate category.

[1099] Generation of initial response

[1100] Next, the server generates an initial response email based on the category and content of the received email. In this process, it improves the quality of the response by referring to similar cases in the database and using appropriate templates (greeting, body, and closing). For example, if a customer inquires about product specifications, the server retrieves the relevant specification information from the product database and generates a response email.

[1101] Consult with the administrator (if necessary)

[1102] If a response is difficult or requires high accuracy, the terminal notifies the administrator to analyze the inquiry in detail. In this case, the server generates the most appropriate answer candidates and presents them to the user (administrator). The user reviews and modifies the answer candidates and provides feedback to the system.

[1103] Automated email creation and sending

[1104] Finally, the server receives feedback from the administrator and automatically generates a final response email. This email is checked for completeness and then automatically sent through the terminal.

[1105] Specific example: Request to arrange a visit

[1106] For example, if an email arrives from another department requesting to schedule a visit next week, the terminal receives the email and categorizes it as a "scheduling request." Next, the server checks the schedule database and automatically generates an email suggesting several possible dates for the visit. Finally, the terminal sends that email.

[1107] Specific example: Product inquiries from customers

[1108] Furthermore, when a customer sends an email requesting information about the specifications of product X, the terminal receives the email and identifies its category as "customer inquiry." The server retrieves the specifications of product X from the product database and generates an automated response email. If the response is incomplete, the terminal automatically consults with the relevant department to obtain a final answer. The server then creates a final email and sends it to the customer through the terminal.

[1109] By using the methods described above, internal email processing will be automated, significantly improving employee work efficiency.

[1110] The following describes the processing flow.

[1111] Step 1:

[1112] The server collects past email data, inquiry history, business processes, and decision-making criteria from within the company, and stores this information in a database. Specifically, it extracts necessary data from email sending and receiving logs and document management systems, organizes it, and registers it in the database.

[1113] Step 2:

[1114] The server uses information from the database to train a generative AI model (such as GPT-3). This involves providing the AI ​​model with pairs of email content and corresponding replies, and repeatedly training it to improve its ability to generate natural-sounding text.

[1115] Step 3:

[1116] Each time the terminal receives a new email, it analyzes its content. Using natural language processing technology, it extracts keywords and important contextual information from the email body and classifies the emails into categories such as "internal consultation," "report / communication," "confirmation of sales activity requirements," and "customer inquiry."

[1117] Step 4:

[1118] The server generates an initial response email based on the aforementioned categories. It refers to similar cases and templates in the database to construct the most appropriate response. For example, in response to a customer inquiry, it prepares the answer by referring to relevant product information and FAQs.

[1119] Step 5:

[1120] The server provides the terminal with an initial response email. If the initial response is insufficient to resolve the issue, or if a high level of accuracy is required, the terminal will notify the appropriate administrator.

[1121] Step 6:

[1122] The user (administrator) receives a notification and reviews and modifies the suggested answers provided by the server. If necessary, they enter additional information or supplementary explanations. The administrator's feedback is reflected in the system.

[1123] Step 7:

[1124] The server automatically generates a final response email based on feedback from the administrator. It then incorporates any corrections or additional information from the administrator to create a complete response.

[1125] Step 8:

[1126] The terminal checks the final response email, performs error checks and final confirmations, and then automatically sends the email to the intended recipient.

[1127] This series of processing steps automates email processing within the company, significantly reducing the effort and time required from employees.

[1128] (Example 1)

[1129] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1130] This invention relates to a system for automating internal data processing and email responses. Conventional systems suffered from long response times from email reception to response, resulting in poor efficiency. Furthermore, manual response creation was prone to human error, often leading to a lack of consistency and accuracy in response content. This resulted in decreased operational efficiency and reduced customer satisfaction.

[1131] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1132] In this invention, the server includes means for collecting internal company data and building a database, means for training a generative AI model using the database, means for receiving emails, analyzing the content of the emails and classifying them into categories, means for generating an initial response email based on the categories, means for generating a response email by referring to similar cases in the database, means for having an administrator perform a detailed analysis of the inquiry content as needed, and means for automatically generating and sending a final response email based on the administrator's feedback. This makes email processing more efficient and improves the consistency and accuracy of response content.

[1133] A "server" is a computer system that provides data and services to multiple computers and terminals over a network.

[1134] A "database" is a systematically organized collection of data, stored in a way that allows for efficient searching and updating.

[1135] A "generative AI model" is a type of artificial intelligence that learns from large amounts of data and performs tasks such as generating natural language and automatically generating responses.

[1136] "Natural language processing technology" refers to a set of technologies and methods that enable computers to understand, generate, and engage in human dialogue.

[1137] "Email" refers to digital messages sent and received via the internet or other computer networks.

[1138] A "category" is a way of classifying and grouping data based on certain common characteristics or attributes.

[1139] A "template" is a document model with a specific format or pattern, and it is a document format that allows for the easy generation of standardized documents by filling in the content.

[1140] "Feedback" refers to evaluations and corrections provided by users or administrators regarding responses and results generated by a system.

[1141] "IMAP" is an abbreviation for Internet Mail Access Protocol, and it is a protocol for managing email on a server.

[1142] POP3 is version 3 of the postal protocol and is an internet standard protocol used to receive email.

[1143] "MIME format" is an abbreviation for Multipurpose Internet Mail Extensions format, and it is a format that allows emails to contain data other than text (such as images, audio, and video).

[1144] "JSON format" is an abbreviation for JavaScript Object Notation format, and is a lightweight data description format primarily used for data exchange.

[1145] This invention relates to a system for automating internal data processing and email responses. This system functions collaboratively through the interaction of servers, terminals, and users.

[1146] First, the server collects data such as past email data, inquiry history, business processes, and decision criteria from within the company, and stores this information in a database. Possible email servers to use include Exchange Server and G Suite. The collected data is saved in CSV or JSON format, and database management systems such as MySQL or MongoDB are used for the database. Based on this database, the server trains a generative AI model (for example, OpenAI's GPT-3). Email content and appropriate response pairs are used as training data.

[1147] Next, the device receives new emails. Protocols such as IMAP and POP3 are used to receive emails. The content of the received emails is analyzed using natural language processing technology (e.g., Google Cloud Natural Language API), and the emails are categorized into categories such as "internal consultation," "report / communication," "sales activity requirements confirmation," and "customer inquiry."

[1148] Based on the aforementioned category and email content, the server refers to similar cases in the database and generates an initial response email. The generated response email includes a greeting, body, and closing. For example, in response to an inquiry such as "Please tell me about the specifications of product X," the server retrieves the relevant specification information from the product database and generates a response. This response email is constructed in MIME format.

[1149] If a response is difficult or requires high accuracy, the terminal notifies the administrator for detailed analysis. The administrator is notified using tools such as Jira or Slack, and the server generates optimal response candidates and presents them to the user (administrator). The administrator reviews these response candidates, makes corrections as needed, and sends feedback back to the system in JSON format.

[1150] Based on this feedback, the server automatically generates a final response email. The generated email is then automatically sent through the device. Email sending APIs such as SendGrid and AWS SES are used to send the email.

[1151] Specific example: Request to arrange a visit

[1152] For example, if an email arrives from another department requesting to schedule a visit next week, the device receives this email and interprets it as a "scheduling request." Next, the server checks its schedule database (e.g., Google Calendar) and automatically generates an email suggesting several possible dates for the visit. Finally, the device sends that email.

[1153] Specific example: Product inquiries from customers

[1154] Furthermore, when a customer sends an email requesting information about the specifications of product X, the terminal receives the email and identifies it as a customer inquiry. The server retrieves the specifications of product X from the product database and generates an automated response email. If the response is incomplete, the terminal automatically consults with the relevant department to obtain a final answer. Once this process is complete, the server creates a final email and sends it to the customer through the terminal.

[1155] Example of a prompt

[1156] "Inquiry: Please tell me about the specifications of product X."

[1157] "Category: Customer Inquiry"

[1158] By using the methods described above, this system can streamline internal email processing and maintain consistency and accuracy in responses.

[1159] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1160] Step 1:

[1161] The server collects internal company data. Specifically, it extracts past email data and inquiry history from mail servers such as Exchange Server and G Suite in CSV or JSON format. The input is data from the mail server, and the output is a structured data file.

[1162] Step 2:

[1163] The server stores the collected data in a database. The collected email data and inquiry history are saved in a database management system such as MySQL or MongoDB. The input is the data file generated in step 1, and the output is the records in the database.

[1164] Step 3:

[1165] The server uses a database to train a generative AI model. It uses pre-processed text data (e.g., OpenAI's GPT-3) to train the generative AI model. The input is text data from the database, and the output is the trained AI model.

[1166] Step 4:

[1167] The device receives a new email. Protocols such as IMAP and POP3 are used to receive the email. The input is the new email from the mail server, and the output is the email data stored on the device's local system.

[1168] Step 5:

[1169] The terminal analyzes the content of emails and categorizes them. Using natural language processing technologies such as the Google Cloud Natural Language API, emails are classified into categories such as "Internal Consultation," "Report / Communication," "Confirmation of Sales Activity Requirements," and "Customer Inquiry." The input is the email data received in step 4, and the output is the analysis results including category information.

[1170] Step 6:

[1171] The server generates an initial response email based on the aforementioned categories. It references similar cases from the database and constructs the response email using an appropriate template. The input is the analysis results from step 5 and similar case data from the database, and the output is the initial response email.

[1172] Step 7:

[1173] If the terminal has difficulty responding or if high accuracy is required, the administrator will be asked to perform a detailed analysis of the inquiry. A notification will be sent to the administrator using tools such as Jira or Slack. The input is the initial response email and its difficulty assessment result, and the output is the notification to the administrator.

[1174] Step 8:

[1175] The server generates the most suitable answer candidates for the administrator. The generated answers are displayed in an HTML email or a dedicated dashboard. The input is the notification content from step 7, and the output is the answer candidates presented to the administrator.

[1176] Step 9:

[1177] The user (administrator) reviews the suggested answers and sends back feedback. The feedback is sent back to the system in JSON format. The input is the suggested answers presented in step 8, and the output is the administrator's feedback.

[1178] Step 10:

[1179] The server automatically generates a final response email based on feedback from the administrator. The generated email is constructed in MIME format. The input is the feedback obtained in step 9, and the output is the final response email.

[1180] Step 11:

[1181] The terminal automatically sends the final response email that was generated. Email sending APIs such as SendGrid or AWS SES are used to send the email. The input is the final response email generated in step 10, and the output is the sent email.

[1182] (Application Example 1)

[1183] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1184] In conventional factories, managers had to manually check and respond to work instructions and reports via email, a process that wasted time and effort. Furthermore, it was difficult to process abnormal reports and adjustment requests in a timely manner, leading to decreased operational efficiency and delays in responses. This invention aims to solve these problems and improve operational efficiency by automating data processing and email responses within the factory.

[1185] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1186] In this invention, the server includes means for collecting internal company data and building a database, means for training a generation AI model using the database, means for receiving emails, analyzing the content of the emails and classifying them into categories, means for a robot terminal to receive new emails, analyze their content and classify them into work instructions or report categories, means for having an administrator perform a detailed analysis of the inquiry content as needed, and means for automatically generating and sending a final response email based on the administrator's feedback. This automates the processing of work instructions and report emails and enables rapid response to anomaly reports and adjustment requests.

[1187] "Internal company data" refers to all information generated within a company, including past email data, inquiry history, business processes, decision-making criteria, and so on.

[1188] A "database" is a collection of electronic information that systematically organizes and stores company data, and manages it in a way that allows access as needed.

[1189] A "generative AI model" refers to an artificial intelligence model that uses information stored in a database to produce appropriate responses and processes, utilizing machine learning and deep learning.

[1190] "Email" refers to electronic messages sent and received via the internet or other electronic communication networks.

[1191] A "category" refers to a group or type of email classified based on its content or nature, and examples include "internal consultation" and "anomaly report."

[1192] An "initial response email" refers to the first response email automatically generated based on an AI model or template in response to a newly received email.

[1193] A "robot terminal" is a device that performs actual physical tasks in factories and other facilities, and has the ability to receive and analyze emails.

[1194] An "administrator" is a person or role responsible for overseeing and coordinating the system, and provides detailed analysis of email content and feedback as needed.

[1195] "Work instructions" are documents that describe the procedures and instructions for specific tasks or operations, and are provided to robot terminals or employees.

[1196] A "progress report" refers to a report that explains the status of ongoing work or projects, and is sent to the administrator via email or other means.

[1197] A "request for adjustment" refers to a request for changes or adjustments to the schedule or working conditions.

[1198] An "anomaly report" refers to a report of a problem or abnormal event that occurred within a factory or system, and it provides detailed information about the incident.

[1199] This invention relates to a system for automating data processing and email responses within a factory. In this system, servers, terminals, and users work together. Specific embodiments of the invention are described in detail below.

[1200] First, the server collects internal company data generated within the factory and builds a database. This data includes past email data, inquiry history, business processes, and decision-making criteria. A database like PostgreSQL might be used.

[1201] Next, the server uses this database to train a generative AI model. Machine learning and deep learning techniques are used for training, such as OpenAI's GPT-3.5. The AI ​​model learns past email content and appropriate response pairs to enable it to generate natural-sounding responses.

[1202] When the terminal receives a new email, it analyzes its content and classifies it into categories such as "internal consultation," "report / communication," "confirmation of sales activity requirements," "customer inquiry," or "work instructions," "progress report," "coordination request," or "anomaly report." Natural language processing technology (e.g., SpaCy) is used for the analysis.

[1203] The server generates an initial response email based on the category and content of the received email. In doing so, it improves the quality of the response by referring to similar cases in the database and using appropriate templates (greeting, body, and closing). For example, for scheduling requests, it refers to the schedule database and automatically generates an email suggesting several possible dates for a visit.

[1204] If a response is difficult or requires high accuracy, the terminal notifies the administrator to analyze the inquiry in detail. In this case, the server generates the most appropriate answer candidates and presents them to the user (administrator). The user reviews and modifies the answer candidates and provides feedback to the system.

[1205] The server receives feedback from the administrator and automatically generates a final response email. This email is checked for completeness and then automatically sent through the terminal.

[1206] As a concrete example, the following is a case of an abnormal report.

[1207] The database stores a history of abnormal occurrences, past response results, and detailed information about the work area. When the robot terminal detects an abnormality in the equipment, it automatically generates the following prompt message and sends it to the server.

[1208] Prompt message:

[1209] Generate an anomaly detection report email.

[1210] Location of abnormality: "Robot arm A3"

[1211] Problem description: "Motor overheating"

[1212] Date of occurrence: YYYY / MM / DD

[1213] Required action: "Prompt inspection and replacement"

[1214] Based on past cases of handling anomalies, generate a detailed report email.

[1215] The server uses this prompt to generate a detailed report email from the AI ​​model. As a result, an email like the following is sent to the administrator.

[1216] Result email:

[1217] Subject: Regarding motor overheating in robot arm A3

[1218] Main text:

[1219] Dear Administrator,

[1220] An overheating issue has been detected in the A3 robot arm. Please see the details below.

[1221] Location of abnormality: Robot arm A3

[1222] Problem description: Motor overheating

[1223] Date of occurrence: YYYY / MM / DD

[1224] Required action: Prompt inspection and replacement

[1225] Based on similar cases in the past, prompt action is required. Please review the matter.

[1226] From factory automation systems

[1227] In this way, data processing and email responses within the factory are automated, significantly improving operational efficiency.

[1228] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1229] Step 1:

[1230] The server collects internal company data generated within the factory, such as past email data, inquiry history, business processes, and decision-making criteria. A database is built to integrate and manage this data. Various internal company data is provided as input, and an integrated database is generated as output. This database is implemented using, for example, PostgreSQL.

[1231] Step 2:

[1232] The server uses the constructed database to train a generative AI model. This training uses email content and appropriate response pairs, for example, OpenAI's GPT-3.5 protocol. The input consists of past email data and example responses from the database, and the output is a generative AI model. This model is used to generate natural-sounding email responses.

[1233] Step 3:

[1234] The terminal receives a new email and analyzes its content. Using natural language processing technology (e.g., SpaCy), it analyzes the email content and classifies it into categories such as "Internal Consultation," "Report / Communication," "Confirmation of Sales Activity Requirements," "Customer Inquiry," "Work Instructions," "Progress Report," "Adjustment Request," and "Anomaly Report." A new email is provided as input, and a category is assigned as output.

[1235] Step 4:

[1236] The server generates an initial response email based on the category and content of the received email. In doing so, it refers to similar cases in the database and uses the appropriate template. Given the email category and its content as input, the initial response email is generated as output. For example, for a scheduling request email, it refers to the schedule database and generates an email suggesting available dates for a visit.

[1237] Step 5:

[1238] If the terminal has difficulty responding or requires high accuracy, it notifies the administrator to perform a detailed analysis of the inquiry. In this case, the server generates the most suitable answer candidates and presents them to the user (administrator). The input is an email requiring detailed analysis, and the output is the generation of answer candidates to present to the administrator.

[1239] Step 6:

[1240] The user (administrator) reviews the suggested answer options and makes corrections as needed. The corrections are fed back into the system. Feedback from the administrator is provided as input, and the corrected answer is obtained as output.

[1241] Step 7:

[1242] The server receives feedback from the administrator and automatically generates a final response email. This email is checked for completeness and automatically sent through the terminal. The corrected response is given as input, and the final response email is generated as output.

[1243] Step 8:

[1244] When the terminal detects an anomaly, it generates a prompt message and sends it to the server. The server uses the prompt message to create a detailed report email from the AI ​​model and sends it to the administrator. As a specific example, the prompt message when the robot terminal detects an anomaly is as follows:

[1245] Prompt message:

[1246] Generate an anomaly detection report email.

[1247] Location of abnormality: "Robot arm A3"

[1248] Problem description: "Motor overheating"

[1249] Date of occurrence: YYYY / MM / DD

[1250] Required action: "Prompt inspection and replacement"

[1251] Based on past cases of handling anomalies, generate a detailed report email.

[1252] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1253] This invention relates to a system that collects internal company data, builds a database, and trains a generative AI model. Furthermore, this system incorporates an emotion engine that recognizes user emotions, adding a function to reflect user emotions in email content and generate more appropriate initial responses. The specific processing of the program of this system will be described below.

[1254] Database construction and AI model training

[1255] The server acquires new data and integrates it into the existing database. This includes historical email data, inquiry history, business processes, and decision criteria. The constructed database is used to train a generative AI model. The training dataset includes email content and corresponding replies, and the AI ​​model is trained to generate natural-sounding responses based on this data.

[1256] Emotional engine integration and emotion recognition

[1257] When a device receives a new email, it analyzes its content. During this process, the emotion engine analyzes the text to determine positive, negative, or neutral emotions and generates a corresponding emotion score. For example, in the case of a customer complaint email, the emotion engine identifies negative emotions and incorporates this result into the subsequent response generation process.

[1258] Category classification and generation of initial responses

[1259] Based on the analysis results of the emotion engine, the terminal classifies the email content into categories such as "internal consultation," "report / communication," "confirmation of sales activity requirements," and "customer inquiry." Based on this classification, the server refers to similar cases and templates in the database and generates an initial response email that reflects the emotion score.

[1260] Notification to the administrator and confirmation of response.

[1261] If a response is difficult or negative emotions are detected, the terminal automatically notifies the administrator. The server then sends an initial response email to the administrator, clearly indicating that negative emotions are present. This allows the user (administrator) to provide appropriate feedback on the negative emotions and modify the initial response.

[1262] Generation and transmission of the final response

[1263] Once administrator feedback is provided, the server automatically generates a final response email based on it. This final email is reviewed and revised by the administrator, and the system ensures it is sent to the recipient without fail. For example, if negative sentiment is detected in an email from a customer regarding a product defect, it provides guidance for the administrator to respond more carefully and courteously.

[1264] Specific example

[1265] For example, if an email arrives from another department requesting to schedule a visit next week, the terminal receives the email and categorizes it as a "scheduling request." If the sentiment analysis result is neutral, the server checks the schedule database and automatically generates an email suggesting several possible dates for the visit. If the sentiment engine detects a positive sentiment, the response email can include more friendly language.

[1266] Furthermore, if a customer sends an email asking for information about the specifications of product X, the terminal receives the email, determines its category to "customer inquiry," and the sentiment engine analyzes the message to see if it contains any negative sentiment. If negative sentiment is detected, the server automatically consults with the relevant department and prepares a detailed response. The user (administrator) reviews and revises the response options, and then sends the final answer to the customer based on that feedback.

[1267] This system significantly improves the efficiency of email correspondence and enables better communication by taking user emotions into consideration.

[1268] The following describes the processing flow.

[1269] Step 1:

[1270] The server collects internal company data such as past email data, inquiry history, business processes, and decision-making criteria, and stores this information in a database. Specifically, it extracts necessary information from internal document management systems and email servers, enabling centralized management of this data.

[1271] Step 2:

[1272] The server uses information from the database to train a generative AI model. This process involves providing the AI ​​model with pairs of email content and corresponding replies, training it to generate natural-sounding responses. This allows the model to acquire the ability to generate appropriate responses for various situations.

[1273] Step 3:

[1274] Each time the terminal receives a new email, it analyzes its contents. Using natural language processing technology, it extracts keywords and important contextual information from the email body and classifies the email into categories such as "internal consultation," "report / communication," "sales activity requirements confirmation," and "customer inquiry." This allows the appropriate response to be selected in the next processing step.

[1275] Step 4:

[1276] The device inputs the analyzed email content into an emotion engine, which determines whether the emotion is positive, negative, or neutral. The emotion engine calculates an emotion score from the text and incorporates the result into the next response generation process. Specifically, if the negative emotion is high, a response that takes this into account is generated.

[1277] Step 5:

[1278] The server generates an initial response email based on the aforementioned categories and sentiment engine results. It refers to similar cases and templates in the database, creating the initial response email using more friendly language for positive responses and polite and careful language for negative responses.

[1279] Step 6:

[1280] The server provides the terminal with an initial response email and notifies the administrator as needed. If the email content is complex or the emotion engine identifies negative emotions, the terminal automatically notifies the administrator.

[1281] Step 7:

[1282] The user (administrator) receives the notification and provides necessary corrections or additional information based on the initial response email and sentiment analysis results provided by the server. For emails containing negative sentiment, the administrator will create a more careful and appropriate response.

[1283] Step 8:

[1284] The server automatically generates a final response email based on the administrator's feedback and provides it to the terminal. Once the email content reflects the administrator's confirmation, a final check is performed.

[1285] Step 9:

[1286] The terminal performs a final check, including error checking and final verification, before automatically sending the email to the intended recipient.

[1287] Through this series of steps, the system automatically generates high-quality email responses that reflect the user's emotions, streamlining internal and external communication.

[1288] (Example 2)

[1289] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1290] Traditional email response systems often fail to consider user emotions, making it difficult to provide appropriate responses, particularly in handling emails containing negative emotions quickly and effectively. Furthermore, the time and effort required to manually categorize email content was a significant problem.

[1291] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1292] In this invention, the server includes means for collecting internal company data and building a database, means for training a generative AI model using the database, means for receiving emails, analyzing the content of the emails and generating sentiment scores, means for classifying email categories based on the sentiment scores, means for generating initial response emails based on the categories, means for notifying the administrator if the initial response email contains negative sentiments, and means for automatically generating and sending a final response email based on the administrator's feedback. This enables quick and appropriate email responses that take into account the user's sentiments.

[1293] "Internal data" refers to information generated or collected within a company or organization, and includes email data, inquiry history, business processes, decision-making criteria, etc.

[1294] A "database" refers to a system for systematically storing collected company data and for efficiently accessing and managing it.

[1295] A "generative AI model" refers to an artificial intelligence model that uses machine learning algorithms to generate natural-sounding responses based on the content of an email.

[1296] "Email" refers to digital messages exchanged over the internet, and can include text and attachments.

[1297] "Sentiment score" is a numerical or categorical representation of the user's emotions (positive, negative, neutral) as expressed in the content of an email.

[1298] A "category" is a group of emails classified based on their content, and examples include "internal consultation," "reporting / communication," "confirmation of sales activity requirements," and "customer inquiry."

[1299] An "initial response email" refers to the first reply email automatically generated by the system in response to an email that has been received.

[1300] An "administrator" refers to someone responsible for making important decisions and corrections in system operation, and is responsible for reviewing and correcting response emails automatically generated by the system.

[1301] A "final response email" refers to the final reply email generated based on the administrator's feedback.

[1302] This invention aims to realize an efficient email response system by collecting internal company data to build a database and training a generative AI model to provide an automated email response function that takes user sentiment into consideration. This system consists of a server, terminals, and users.

[1303] Data collection and database construction

[1304] The server aggregates data collected from various departments within the company, including email data, inquiry history, business processes, and decision-making criteria. The data is first saved to the local disk and then inserted into a database. This involves using the Python pandas library to read data in CSV format and then using SQLAlchemy to insert it into a PostgreSQL database. Finally, it executes queries necessary to ensure data integrity.

[1305] AI model training

[1306] The server trains a generative AI model using an integrated database. This AI model uses the Hugging Face transformers library to load a pre-prepared language model (e.g., BERT). Then, it feeds the training dataset extracted from the integrated database to the model and optimizes it using the PyTorch library.

[1307] Emotion analysis and emotion score generation

[1308] When the device receives a new email, it analyzes and understands its content. The email content is then analyzed using the TextBlob library to generate sentiment scores (positive, negative, or neutral). For example, if the prompt "I am very disappointed with this product" is analyzed, a negative sentiment score will be assigned.

[1309] Email category classification

[1310] Upon receiving an email, the terminal categorizes it based on its sentiment score and content. This classification is performed using Scikit-learn's Naive Bayes classifier, and the results are stored in a local database. Categories include "Internal Consultation," "Reporting / Communication," "Confirmation of Sales Activity Requirements," and "Customer Inquiry."

[1311] Generating an initial response email

[1312] The server generates an initial response email by referencing similar templates in the database based on the classified category and sentiment score. This process uses SQL queries to find the appropriate template and the Jinja2 template engine to populate it with sentiment score and category information.

[1313] Notification to the administrator

[1314] If the initial response email contains negative sentiment, the device automatically notifies the administrator. This notification sends the initial response email and the sentiment analysis results to the administrator via the SMTP server.

[1315] Administrator's feedback

[1316] The user (administrator) receives a notification, reviews the initial response email, and makes corrections as needed. These corrections are made through a web interface (e.g., a Django application), and feedback is sent to the server.

[1317] Generating and sending the final response email

[1318] Based on the administrator's feedback, the server generates a final response email. This generated email is automatically sent via the SMTP server.

[1319] Specific example

[1320] For example, if an email arrives from another department requesting to schedule a visit next week, the terminal receives the email and categorizes it as a "scheduling request." If the sentiment analysis is neutral, the server checks the schedule database and automatically generates an email suggesting several possible visit dates. Similarly, for an email requesting information about product X, the category is determined to be "customer inquiry." If the sentiment engine detects negative emotions, the server consults with the relevant department and prepares a detailed response. The user (administrator) reviews and revises the response options, and the final answer is sent to the customer based on their feedback.

[1321] This system significantly improves the efficiency of email correspondence and enables better communication by taking user emotions into consideration.

[1322] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1323] Step 1:

[1324] The server collects data from various departments within the company, including email data, inquiry history, business processes, and decision-making criteria. Specifically, it uses the Python pandas library to read data in CSV file format and saves it to the local disk.

[1325] Input: Data in CSV file format collected from each department.

[1326] Data processing: Read the CSV file using pandas and convert it to a DataFrame.

[1327] Output: Data stored in memory as a pandas dataframe

[1328] Step 2:

[1329] The server inserts the collected data into a PostgreSQL database using SQLAlchemy. During this process, it executes queries to maintain data integrity and sets necessary indexes.

[1330] Input: Data in pandas DataFrame format

[1331] Data processing: Inserting dataframes into a PostgreSQL database using SQLAlchemy.

[1332] Output: Data integrated into a PostgreSQL database

[1333] Step 3:

[1334] The server extracts training datasets from an integrated database and trains a generative AI model. Here, it uses the Hugging Face transformers library to load a pre-prepared BERT model. The PyTorch library is then used to optimize the model.

[1335] Input: Training dataset extracted from a PostgreSQL database

[1336] Data Calculation: Training an AI model using Hugging Face transformers and PyTorch

[1337] Output: Optimized generative AI model

[1338] Step 4:

[1339] The device receives a new email. Using the mail server (e.g., an IMAP server), it parses the received email with the TextBlob library and generates positive, negative, and neutral sentiment scores.

[1340] Input: Received email

[1341] Data processing: Sentiment analysis of email content using the TextBlob library

[1342] Output: Emotion score

[1343] Step 5:

[1344] The device categorizes received emails using a Naive Bayes classifier in Scikit-learn, based on their content and sentiment score. These categories are stored in a local database.

[1345] Input: Sentiment score and email content

[1346] Data processing: Categorical classification using Scikit-learn's Naive Bayes classifier.

[1347] Output: Category classification results

[1348] Step 6:

[1349] The server searches for similar templates in the database based on the classified category and sentiment score, and generates an initial response email using the Jinja2 template engine.

[1350] Input: Category classification results and sentiment score

[1351] Data processing: Search for templates using SQL queries and embed the information into the templates using Jinja2.

[1352] Output: Initial response email

[1353] Step 7:

[1354] If the initial response email contains negative sentiment, the device automatically notifies the administrator. The notification is sent using an SMTP server and includes the initial response email and the sentiment analysis results.

[1355] Input: Initial response email

[1356] Data processing: Send notification emails using an SMTP server.

[1357] Output: Notification email to administrator

[1358] Step 8:

[1359] The user (administrator) receives a notification, checks the initial response email, and makes corrections as needed via the web interface (e.g., a Django application). The corrected results are sent to the server.

[1360] Input: Notification email and web interface to administrator

[1361] Data processing: Administrator corrections and feedback submission

[1362] Output: Corrected response email feedback

[1363] Step 9:

[1364] The server generates a final response email based on the administrator's feedback. The generated email is automatically sent to the recipient via the SMTP server.

[1365] Input: Administrator's feedback

[1366] Data processing: Generate the final response email using the Jinja2 template engine.

[1367] Output: Sending of final response email

[1368] (Application Example 2)

[1369] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1370] In modern businesses, there is a need to process emails and other digital communications quickly and appropriately to improve the efficiency of customer service and internal communication. Furthermore, in the security field, it is necessary to utilize real-time sentiment recognition to detect abnormal behavior early and take swift, appropriate action. However, conventional systems struggle to address these challenges simultaneously, particularly lacking in the detection of abnormal behavior and rapid notification to administrators using sentiment recognition technology. Therefore, there is a need for the development of a system that comprehensively achieves both efficient email response and enhanced security monitoring.

[1371] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1372] In this invention, the server includes means for collecting internal company data and building a database, means for training a generative AI model using the database, means for receiving emails, analyzing the content of the emails and classifying them into categories, means for generating an initial response email based on the categories, means for having an administrator perform a detailed analysis of the inquiry content as needed, means for automatically generating and sending a final response email based on the administrator's feedback, means for detecting faces in real time and generating an emotion score using an emotion engine, means for suggesting countermeasures when abnormal emotions are detected, and means for detecting abnormal behavior based on the emotion score and providing notifications. This enables more efficient email handling and enhanced security monitoring.

[1373] "Internal company data" refers to various types of information and data generated within a company, including business processes, inquiry history, past email data, and decision-making criteria.

[1374] A "database" is a structured collection of data that systematically stores collected company data, making it easy to search and manipulate.

[1375] A "generative AI model" is an artificial intelligence model that learns from collected data and provides natural responses and inferences to new data.

[1376] "Email" refers to digital messages sent and received using computer networks such as the internet.

[1377] "Means of categorization" refers to algorithms or devices that analyze the content of received emails and sort them into predefined categories (e.g., internal consultation, report / communication).

[1378] An "initial response email" is the first reply email that is automatically generated in response to an email that has been received.

[1379] An "administrator" is a person responsible for the operation and supervision of a system, and is also responsible for analyzing system inquiries and providing feedback as needed.

[1380] A "final response email" is the final reply email that is automatically generated based on the administrator's feedback and sent to the recipient.

[1381] "Means for detecting faces" refers to technologies or devices that use cameras or other imaging devices to identify and recognize human faces within an image.

[1382] An "emotion engine" is an algorithm or system that analyzes emotions from detected facial expressions and generates emotion scores such as positive, negative, or neutral.

[1383] An "emotion score" is a numerical representation of the degree of emotion analyzed by the emotion engine.

[1384] "Abnormal emotions" refer to negative emotions that exceed certain standards, such as anger, fear, or disgust.

[1385] "Means of suggesting countermeasures" refers to a mechanism or system that, when abnormal emotions are detected, indicates appropriate actions or responses in accordance with the situation.

[1386] "Means of notification" refers to a system for promptly informing security personnel and administrators of any detected abnormal emotions or behaviors.

[1387] To implement this invention, the system is constructed according to the following steps.

[1388] Database construction and AI model training

[1389] The server collects internal company data and builds a database. This internal data includes past email data, inquiry history, business processes, and decision-making criteria. Based on this data, a generative AI model is trained. The training dataset includes email content and corresponding replies, and the AI ​​model is trained to generate natural-sounding responses based on this data.

[1390] Emotional engine integration and emotion recognition

[1391] When a device receives a new email, it analyzes its content. During this process, the emotion engine analyzes the text to determine positive, negative, or neutral emotions and generates an emotion score. For example, in the case of a "customer complaint email," negative emotions are identified, and this result is reflected in the subsequent response generation process.

[1392] Face detection and real-time emotion score generation

[1393] The device uses a camera to detect faces in its surroundings in real time. Face detection uses face recognition libraries such as OpenCV. The detected face images are analyzed by an emotion engine, and an emotion score is generated. This allows for suggested actions if abnormal emotions are detected.

[1394] Generation of initial response email and notification to administrator

[1395] The device categorizes emails based on their sentiment score and content, and generates an initial response email. This response email is automatically generated using a template. If an abnormal sentiment is detected, the device notifies the administrator, and if the situation is complex or negative sentiment is identified, the administrator is instructed to perform a detailed analysis.

[1396] Generating and sending the final response email

[1397] Based on administrator feedback, the server automatically generates a final response email. This final email is reviewed and corrected by the administrator, and the system sends it to the recipient.

[1398] Hardware and software to be used

[1399] Hardware: Camera devices, servers, terminals

[1400] Software: OpenCV (face detection), emotion engine (emotion analysis), database management system (management of internal company data)

[1401] Specific example:

[1402] Smart glasses are used as a security system in supermarkets and shopping malls to detect individuals exhibiting unusual emotions in real time. Alerts are sent to security guards, enabling a rapid response.

[1403] Example of a prompt:

[1404] Premise: In a supermarket, security guards are using smart glasses. A real-time emotion recognition and facial recognition system is needed to detect unusual emotions (anger, fear, disgust) and immediately notify the security team.

[1405] prompt:

[1406] 1. The facial recognition system detects customers' faces in the supermarket in real time and performs sentiment analysis.

[1407] 2. The emotion engine analyzes each customer's facial expressions and identifies customers who are experiencing emotions such as anger, fear, or disgust.

[1408] 3. If abnormal emotions are detected, the notification system will send an alert to the security team along with the log and facial image.

[1409] 4. The security team will use this information to respond quickly and ensure security.

[1410] Please create Python code to implement the process described above.

[1411] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1412] Step 1:

[1413] The server collects internal company data. It retrieves data such as past email data, inquiry history, business processes, and decision criteria, and stores it in a database. This database is used to build a training dataset for generative AI models. The input is internal company data, and the output is the constructed database.

[1414] Step 2:

[1415] The server uses a database to train a generative AI model. Email content and corresponding replies are used as the training dataset to train the AI ​​model to generate natural-sounding responses. The input is the training dataset in the database, and the output is the trained generative AI model.

[1416] Step 3:

[1417] The device receives a new email. It analyzes the email's content, uses an emotion engine to identify positive, negative, and neutral emotions, and generates an emotion score. The input is the received email, and the output is the emotion score. Specifically, it analyzes the email text and calculates the emotion score using the emotion engine.

[1418] Step 4:

[1419] The device categorizes emails based on their sentiment score and content. For example, it might categorize them as "Internal Consultation," "Report / Communication," "Confirmation of Sales Activity Requirements," or "Customer Inquiry." The input is the sentiment score and email content, and the output is the categorized email.

[1420] Step 5:

[1421] The terminal generates an initial response email based on the category. Using a template, it generates an initial response that includes appropriate wording according to the sentiment score. The input is the category and sentiment score, and the output is the initial response email.

[1422] Step 6:

[1423] If deemed necessary by the administrator, the terminal will prompt the administrator to perform a detailed analysis of the inquiry. The administrator will then provide feedback and optimize the content of the initial response email. The input is the initial response email, and the output is the administrator's feedback.

[1424] Step 7:

[1425] The server automatically generates a final response email based on the administrator's feedback. This final email is then reviewed and revised by the administrator before being sent to the recipient. The input is the administrator's feedback, and the output is the final response email.

[1426] Step 8:

[1427] The device uses a camera to detect faces in its surroundings in real time. The detected face images are analyzed by an emotion engine, and an emotion score is generated. The input is an image from the camera device, and the output is the face image and the emotion score.

[1428] Step 9:

[1429] Based on the emotion score, if abnormal emotions are detected, the device will suggest actions and send a notification. The notification will be sent to a designated security officer or administrator, requiring a prompt response. The input is the emotion score, and the output is a notification message. Specifically, it evaluates the emotion score and sends a notification according to the level of risk.

[1430] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1431] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1432] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[1433] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1434] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[1435] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[1436] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[1437] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[1438] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[1439] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[1440] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[1441] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[1442] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[1443] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1444] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[1445] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[1446] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[1447] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[1448] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[1449] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[1450] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[1451] The following is further disclosed regarding the embodiments described above.

[1452] (Claim 1)

[1453] Methods for collecting internal company data and building a database,

[1454] A means for training a generative AI model using the aforementioned database,

[1455] A means of receiving emails, analyzing their content, and classifying them into categories,

[1456] Means for generating an initial response email based on the aforementioned categories,

[1457] A means to have the administrator perform a detailed analysis of the inquiry content as needed,

[1458] A means of automatically generating and sending a final response email based on the administrator's feedback,

[1459] A system that includes this.

[1460] (Claim 2)

[1461] The system according to claim 1, characterized in that the category of the aforementioned email is "internal consultation," "report / communication," "confirmation of requirements for sales activities," or "customer inquiry."

[1462] (Claim 3)

[1463] The system according to claim 1, characterized in that if it is not necessary to contact the aforementioned administrator, the initial response email is automatically sent as is.

[1464] "Example 1"

[1465] (Claim 1)

[1466] Methods for collecting internal company data and building a database,

[1467] A means for training a generative AI model using the aforementioned database,

[1468] A means of receiving emails, analyzing their content, and classifying them into categories,

[1469] Means for generating an initial response email based on the aforementioned categories,

[1470] A means for generating a response email by referring to similar cases in the aforementioned database,

[1471] A means to have the administrator perform a detailed analysis of the inquiry content as needed,

[1472] A means of automatically generating and sending a final response email based on the administrator's feedback,

[1473] A system that includes this.

[1474] (Claim 2)

[1475] The system according to claim 1, characterized in that it analyzes the content of the aforementioned email using natural language processing technology.

[1476] (Claim 3)

[1477] The system according to claim 1, characterized in that if it is not necessary to contact the aforementioned administrator, the initial response email is automatically sent as is.

[1478] "Application Example 1"

[1479] (Claim 1)

[1480] Methods for collecting internal company data and building a database,

[1481] A means for training a generated AI model using the aforementioned database,

[1482] A means of receiving emails, analyzing their content, and classifying them into categories,

[1483] Means for generating an initial response email based on the aforementioned categories,

[1484] A method for a robot terminal to receive new emails, analyze their contents, and classify them into work instructions or report categories,

[1485] A means to have the administrator perform a detailed analysis of the inquiry content as needed,

[1486] A means of automatically generating and sending a final response email based on the administrator's feedback,

[1487] A system that includes this.

[1488] (Claim 2)

[1489] The system according to claim 1, characterized in that the categories of the aforementioned emails are "internal consultation," "report / communication," "confirmation of requirements for sales activities," "customer inquiry," or "work instructions," "progress report," "coordination request," or "anomaly report."

[1490] (Claim 3)

[1491] The system according to claim 1, characterized in that if it is not necessary to contact the aforementioned administrator, the initial response email is automatically sent as is.

[1492] "Example 2 of combining an emotion engine"

[1493] (Claim 1)

[1494] Methods for collecting internal company data and building a database,

[1495] A means for training a generative AI model using the aforementioned database,

[1496] A means of receiving emails, analyzing their content, and generating sentiment scores,

[1497] A means for classifying email categories based on the aforementioned sentiment score,

[1498] Means for generating an initial response email based on the aforementioned categories,

[1499] If the initial response email contains negative sentiment, a means of notifying the administrator is provided.

[1500] A means of automatically generating and sending a final response email based on the administrator's feedback,

[1501] A system that includes this.

[1502] (Claim 2)

[1503] The system according to claim 1, characterized in that the category of the aforementioned email is "internal consultation," "report / communication," "confirmation of requirements for sales activities," or "customer inquiry."

[1504] (Claim 3)

[1505] The system according to claim 1, characterized in that if it is not necessary to contact the aforementioned administrator, the initial response email is automatically sent as is.

[1506] "Application example 2 when combining with an emotional engine"

[1507] (Claim 1)

[1508] Methods for collecting internal company data and building a database,

[1509] A means for training a generative AI model using the aforementioned database,

[1510] A means of receiving emails, analyzing their content, and classifying them into categories,

[1511] Means for generating an initial response email based on the aforementioned categories,

[1512] A means to have the administrator perform a detailed analysis of the inquiry content as needed,

[1513] A means of automatically generating and sending a final response email based on the administrator's feedback,

[1514] A means for detecting faces in real time and generating an emotion score using an emotion engine,

[1515] A means of suggesting how to deal with abnormal emotions when they are detected,

[1516] A means of detecting and notifying about abnormal behavior based on an emotional score,

[1517] A system that includes this.

[1518] (Claim 2)

[1519] The system according to claim 1, characterized in that the category of the aforementioned email is "internal consultation," "report / communication," "confirmation of requirements for sales activities," or "customer inquiry."

[1520] (Claim 3)

[1521] The system according to claim 1, characterized in that if it is not necessary to contact the aforementioned administrator, the initial response email is automatically sent as is. [Explanation of symbols]

[1522] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. Methods for collecting internal company data and building a database, A means for training a generative AI model using the aforementioned database, A means of receiving emails, analyzing their content, and classifying them into categories, Means for generating an initial response email based on the aforementioned categories, A means to have the administrator perform a detailed analysis of the inquiry content as needed, A means of automatically generating and sending a final response email based on the administrator's feedback, A system that includes this.

2. The system according to claim 1, characterized in that the category of the aforementioned email is "internal consultation," "report / communication," "confirmation of requirements for sales activities," or "customer inquiry."

3. The system according to claim 1, characterized in that if it is not necessary to contact the aforementioned administrator, the initial response email is automatically sent as is.

Images