system

Abstract

Provide a system. A means for receiving input memos such as scribbles; means for tokenizing the received notes and removing stop words and stemming; means for performing syntactic parsing, entity recognition, and sentiment analysis on the preprocessed data; A means of recognizing situations and objectives based on the analyzed data, A means of producing sentences using appropriate grammar, syntax, and tone; and means for displaying the generated sentence to the user.

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In modern society, many people spend a great deal of time and energy on document creation. Accurate and effective communication is especially important in business situations, but the writing required requires a certain level of skill and experience. This often leads to problems with the psychological burden and time costs associated with document creation, hindering efficient business execution. Unfamiliar document creation also poses the risk of miscommunication and misunderstandings. There is a need for a system that can solve these problems and enable everyone to easily create appropriate documents. [Means for solving the problem]

[0005] To address this issue, we provide a system with the following configuration. First, we provide a means for receiving input, scribbled notes, and incorporate means for tokenizing the received notes, removing stop words, and stemming. We then provide means for syntactic analysis, entity recognition, and sentiment analysis based on the preprocessed data, and provide means for recognizing situations and purposes based on the analyzed data. We then provide a system that includes means for generating sentences using appropriate grammar, syntax, and tone, and means for displaying the generated sentences to the user. Furthermore, we provide a system that includes means for inferring situations based on the user's past notes and behavioral patterns, and means for learning and reflecting user corrections to the generated sentences, allowing users to easily create appropriate sentences. This system significantly reduces the psychological burden and time cost of writing sentences and improves the efficiency and accuracy of communication.

[0006] A "note" is an electronic record containing simple keywords or short phrases entered by the user.

[0007] "Tokenization" is the process of breaking down input notes into words and phrases.

[0008] "Stop words" refer to words with little meaning, such as "wa" and "no," and the process of removing them.

[0009] "Stemming" is the process of converting word forms into root forms and is used to reduce grammatical variation.

[0010] "Syntax analysis" is the process of analyzing the structure of a sentence and identifying the role of each word and phrase.

[0011] "Entity recognition" is the technique of identifying and classifying specific items or concepts.

[0012] "Sentiment analysis" is a technique for analyzing the emotions and tone in notes and texts and identifying their characteristics.

[0013] A "situation" is a specific situation or scene that a sentence is intended to address.

[0014] "Purpose" refers to the specific goal or intention that you are trying to achieve when writing or communicating information.

[0015] A "user profile" is a collection of information that analyzes a user's past actions, preferences, and behavioral patterns.

[0016] "Proper grammar" refers to grammar that follows standard language structures and is free of errors.

[0017] "Syntax" refers to the arrangement and relationship of words and phrases in a sentence.

[0018] "Tone" refers to linguistic features that express the overall mood or feel of a piece of writing.

[0019] "Learning" refers to the technology that allows the system to reflect the user's corrections and input in the subsequent sentence generation process. [Brief explanation of the drawings]

[0020] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram illustrating a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0021] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0022] First, the terms used in the following description will be explained.

[0023] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0024] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0025] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0026] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0027] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0028] [First embodiment]

[0029] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0030] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0031] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0032] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0033] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0034] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0035] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0036] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0037] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0038] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0039] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0040] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0041] The present invention relates to a system that receives inputted scribbled notes and automatically generates sentences that are deemed appropriate for the situation and purpose from them. Specific embodiments of the system will be described below.

[0042] Note entry and preprocessing

[0043] The user inputs a simple memo into the terminal. For example, it might be something like, "Project progress is delayed. Cause: Materials have not arrived. Solution: Consider using alternative materials." The terminal receives this memo and sends it to the server.

[0044] The server performs pre-processing on the received memos, which includes the following steps:

[0045] 1. Tokenization: Splitting notes into words and phrases.

[0046] 2. Stop word removal: remove words that have little meaning.

[0047] 3. Stemming: Converting words into their base forms.

[0048] For example, tokens such as "Project progress," "Delayed," "Cause," "Materials," "Not arrived," "Countermeasures," "Alternative materials," and "Considering use" are generated.

[0049] Natural Language Processing (NLP)

[0050] The server applies NLP algorithms based on the pre-processed data, a process that includes:

[0051] 1. Syntactic analysis: Analyzing the structure of a sentence and identifying the roles of words and phrases.

[0052] 2. Entity Recognition: Identifying specific items.

[0053] 3. Sentiment analysis: Analyze the tone and sentiment of the note.

[0054] For example, "project progress" is recognized as an entity, and "being late" is analyzed as a negative emotion.

[0055] Situational Awareness

[0056] The server uses the analyzed data to understand the current situation and objectives. This process takes into account the following factors:

[0057] Understand context: See notes users have previously entered and the current status of projects.

[0058] User profile: Based on the user's past behavioral patterns and general work methods.

[0059] For example, if the "project progress" is "delayed" and the cause is "materials not arriving," it is determined that this is a situation that requires countermeasures.

[0060] Sentence generation

[0061] The server generates appropriate sentences based on the information from the situational awareness module. The following factors are taken into account in this process:

[0062] Grammar and Syntax: Use correct grammar and appropriate syntax.

[0063] Tone of writing: Generate text in a tone appropriate for the reader (e.g., a formal tone appropriate for business correspondence).

[0064] For example, the following sentence is generated:

[0065] To your boss

[0066] Thank you for your hard work. I'll keep you updated on the progress of the project.

[0067] The project is currently behind schedule because the necessary materials have not yet arrived. To address this issue, we are considering using alternative materials.

[0068] I would appreciate any instructions.

[0069] Thank you very much.

[0070] Output and check the text

[0071] The server sends the generated text to the terminal. The terminal displays this text to the user. The user can check the displayed text and make corrections as necessary. The system learns from these corrections and reflects them in the next generation.

[0072] In this way, the system converts simple notes entered by users into natural-sounding sentences, significantly reducing the burden of writing and improving the efficiency and accuracy of communication.

[0073] The processing flow will be explained below.

[0074] Step 1:

[0075] The user inputs a simple memo into the terminal, which may include keywords or short phrases such as "Project progress is delayed. Cause: Materials have not arrived. Solution: Consider using alternative materials."

[0076] Step 2:

[0077] The terminal receives the entered memo and transmits the memo to the server.

[0078] Step 3:

[0079] The server tokenizes the received memo. During this process, the memo is divided into words and phrases. For example, tokens such as "Project progress," "Delayed," "Cause," "Materials," "Not arrived," "Countermeasures," "Alternative materials," and "Consideration of use" are generated.

[0080] Step 4:

[0081] The server removes stop words from the tokenized data, a process that removes words with little meaning (e.g., "wa" and "no").

[0082] Step 5:

[0083] The server performs stemming on the tokenized and stop-word-removed data. Stemming is the process of converting words into their root forms, for example, converting "use consideration" into "use consideration suru."

[0084] Step 6:

[0085] The server then performs syntactic analysis on the preprocessed data, a process that analyzes the structure of the sentence and identifies the role of each word and phrase.

[0086] Step 7:

[0087] The server performs entity recognition based on the results of the parsing, specifically identifying and classifying specific items (e.g., "project progress" or "materials").

[0088] Step 8:

[0089] The server performs sentiment analysis to determine the tone and sentiment of the note, during which the phrase "late" is analyzed as having a negative sentiment.

[0090] Step 9:

[0091] The server recognizes the situation and purpose based on the analyzed data, referring to the user's past notes, behavioral patterns, and current project status.

[0092] Step 10:

[0093] The server generates appropriate sentences based on the information recognized by the context recognition module, using correct grammar and syntax and in a formal business tone.

[0094] Step 11:

[0095] The server transmits the generated text to the terminal.

[0096] Step 12:

[0097] The terminal displays the generated text to the user, who can then check it and make corrections as necessary.

[0098] Step 13:

[0099] If the user makes any corrections, they are sent to the server, which learns the corrections and applies them to the next sentence generation process.

[0100] In this way, the user only needs to input a simple note, and the server will generate an appropriate sentence, which the device will then provide to the user. This completes the entire process, significantly reducing the burden of writing sentences.

[0101] Example 1

[0102] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0103] Conventionally, even if a user inputs a simple note, converting it into a proper sentence takes a lot of time and effort. Furthermore, manual sentence creation is prone to errors, which can reduce the efficiency and accuracy of communication. An effective system is needed to solve these problems.

[0104] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0105] In this invention, the server includes means for receiving input simple memos, means for dividing the received memos into words and phrases, removing words with little meaning, and converting words into base forms, means for performing syntactic analysis, entity recognition, and sentiment analysis based on the preprocessed data, means for recognizing situations and purposes based on the analyzed data, means for generating sentences using correct grammar, appropriate syntax, and a tone tailored to the reader, means for displaying the generated sentences to the user, and means for the user to check and correct the displayed sentences. This enables the user to quickly generate high-quality and appropriate sentences simply by inputting simple memos.

[0106] "Entered brief notes" are short, condensed pieces of information or items entered by a user using a terminal.

[0107] The "receiving means" is a means for receiving information sent from a terminal and inputting it into the server.

[0108] "Split into words and phrases" means splitting the inputted notes into individual words and phrases to make them easier to process.

[0109] "Removing words with little meaning" means removing words that are less important as information in the preprocessing stage.

[0110] "Converting words to base forms" refers to converting words to their base forms in preparation for further analysis.

[0111] "Syntax analysis" is the process of analyzing the structure of a sentence and identifying the roles and relationships of each word and phrase.

[0112] "Entity recognition" refers to identifying and tagging specific items or concepts.

[0113] "Sentiment analysis" is the analysis of tone or emotion (positive, negative, neutral, etc.) within text.

[0114] "Recognizing the situation and purpose" means understanding and analyzing the current situation and goals based on analyzed data.

[0115] "Correct grammar, appropriate syntax, and appropriate tone for the reader" means producing documents that are grammatically correct, well-structured, and have an appropriate tone for the reader.

[0116] "Means for generating" refers to means for automatically creating appropriate sentences based on information from preprocessing and analysis.

[0117] The "display means" is a means for showing the text generated by the server to the user.

[0118] The "means for making corrections" is a means by which the user can add or correct the generated text.

[0119] The present invention relates to a system that automatically generates appropriate sentences according to the situation and purpose based on simple notes entered by a user. A specific embodiment of this system will be described below.

[0120] Note entry and preprocessing

[0121] The user inputs a simple memo into the terminal. For example, a memo such as "Project progress is delayed. Cause: Materials have not arrived. Solution: Consider using alternative materials." The terminal receives this memo and sends it to the server. The server performs preprocessing on the received memo. This preprocessing includes the following steps:

[0122] 1. Tokenization: Divide notes into words and phrases. For example, tokens such as "Project progress," "Delayed," "Cause," "Materials," "Not arrived," "Countermeasures," "Alternative materials," and "Considering use" are generated.

[0123] 2. Stop word removal: Remove words with little meaning. For example, words such as "no", "ga", and "wa" are removed.

[0124] 3. Stemming: Converting words into their root forms, for example, converting "materials not yet arrived" into "materials not yet arrived."

[0125] The server performs these preprocessing steps using an NLP library written in Python (e.g., NLTK or spaCy).

[0126] Natural Language Processing (NLP)

[0127] The server applies NLP algorithms based on the pre-processed data. This process includes:

[0128] 1. Syntactic analysis: Analyzes the structure of a sentence and identifies the roles of words and phrases. For example, "project progress" is recognized as the subject and "delayed" as the verb.

[0129] 2. Entity Recognition: Identifying specific items, for example, "Project Progress" is recognized as an entity.

[0130] 3. Sentiment analysis: Analyze the tone and sentiment of the note. For example, "I'm late" is analyzed as a negative sentiment.

[0131] The server performs these processes using generative AI models such as BERT or GPT-3 (registered trademark).

[0132] Situational Awareness

[0133] The server uses the analyzed data to understand the current situation and objectives. This process takes into account the following factors:

[0134] Understand context: See notes users have previously entered and the current status of projects.

[0135] User profile: Based on the user's past behavioral patterns and general work methods.

[0136] For example, if the "project progress" is "delayed" and the cause is "materials not yet arrived," it is determined that a countermeasure is required. The server accesses the database and refers to past notes and project history to understand the current situation.

[0137] Sentence generation

[0138] The server generates an appropriate sentence based on the information from the situational awareness module. The following factors are taken into account in this process:

[0139] Grammar and Syntax: Use correct grammar and appropriate syntax.

[0140] Tone of writing: Generate text in a tone appropriate for the reader, for example, using a formal tone appropriate for business correspondence.

[0141] The server uses a generative AI model (e.g., GPT-3) to generate sentences based on the analysis results and contextual awareness information. An example of a generated sentence is as follows:

[0142] To your boss

[0143] Thank you for your hard work. I'll keep you updated on the progress of the project.

[0144] The project is currently behind schedule because the necessary materials have not yet arrived. To address this issue, we are considering using alternative materials.

[0145] I would appreciate any instructions.

[0146] Thank you very much.

[0147] Output and check the text

[0148] The server sends the generated text to the terminal. The terminal displays this text to the user. The user checks the displayed text and makes corrections as necessary. The system learns from these corrections and reflects them in the next generation.

[0149] In this way, the system converts simple notes entered by users into natural-sounding sentences, significantly reducing the burden of writing and improving the efficiency and accuracy of communication.

[0150] Prompt Sentence Examples

[0151] Examples of prompts to input to a generative AI model include:

[0152] Based on the memo "Project progress is delayed. Cause: Materials have not arrived. Solution: Consider using alternative materials," generate a report to send to your boss.

[0153] Based on this prompt, the AI ​​model generates appropriate sentences.

[0154] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0155] Processing steps of this system's program

[0156] Below, the processing flow of the program of this system will be explained step by step, and the specific operations, inputs, and outputs performed at each step will be shown.

[0157] Step 1: Enter and send your note

[0158] Description: The user inputs a simple memo into the terminal. For example, "Project progress is delayed. Cause: Materials have not arrived. Solution: Consider using alternative materials." The terminal receives this memo and sends it to the server.

[0159] Input: Notes entered by the user into the device.

[0160] Output: The note received by the server.

[0161] Specific operation: The user enters a note into the input field on their smartphone or computer and presses the "Send" button, which sends the note to the server.

[0162] Step 2: Preprocessing notes

[0163] Description: The server performs the following data preprocessing on the received memo.

[0164] Tokenization: Splitting notes into words and phrases.

[0165] Stopword removal: Remove words that have little meaning.

[0166] Stemming: Converting words into their root forms.

[0167] Input: Received note.

[0168] Output: Preprocessed data (tokenized, stopword-removed, stemmed words and phrases).

[0169] Specific operation: The server uses Python NLP libraries (NLTK and spaCy) to divide the notes into tokens such as "project progress," "delayed," "cause," "materials," "not arrived," "countermeasures," "alternative materials," and "considering use," and then removes stop words and performs stemming.

[0170] Step 3: NLP analysis

[0171] Description: The server performs parsing, entity recognition, and sentiment analysis on the preprocessed data.

[0172] Syntactic analysis: Analyzing the structure of a sentence.

[0173] Entity Recognition: Identifying specific items.

[0174] Sentiment analysis: Analyze the tone and sentiment of your notes.

[0175] Input: Preprocessed data.

[0176] Output: Results of parsing, entity recognition, and sentiment analysis.

[0177] Specific operation: The server uses a generative AI model such as BERT or GPT-3 to obtain the analysis result, where "project progress" is the subject and "delayed" is a verb that expresses negative sentiment.

[0178] Step 4: Situational Awareness

[0179] Description: The server recognizes the current situation and objectives based on the analyzed data.

[0180] Understand context: See previously entered notes and project status.

[0181] User profile: Based on past behavioral patterns and general work methods.

[0182] Input: Results of parsing, entity recognition, and sentiment analysis.

[0183] Output: Current situation and objectives.

[0184] Specific operation: The server accesses the database, references past notes and history such as "project progress is behind schedule" or "the cause is the non-arrival of materials," and recognizes that action is needed.

[0185] Step 5: Sentence generation

[0186] Description: The server generates appropriate text based on the information from the situational awareness module.

[0187] Grammar and Syntax: Use correct grammar and appropriate syntax.

[0188] Tone of writing: Generate text in a tone appropriate for the reader.

[0189] Input: Current situation and objectives.

[0190] Output: The generated sentence.

[0191] Specific operation: The server uses a generative AI model (such as GPT-3) to generate sentences such as, "To my boss, I'm reporting on the progress of the project. The project is currently delayed because the materials have not yet arrived."

[0192] Step 6: Output and check the text

[0193] Description: The server sends the generated text to the terminal, which displays it to the user. The user can check the displayed text and make corrections if necessary.

[0194] Input: The generated sentence.

[0195] Output: The text reviewed and corrected by the user.

[0196] Specific operation: The generated text is displayed on the terminal, and the user can check and edit it on the terminal. The edits are recorded by the server and reflected in the next text generation.

[0197] Prompt Sentence Examples

[0198] Examples of prompts to input to a generative AI model include:

[0199] Based on the memo "Project progress is delayed. Cause: Materials have not arrived. Solution: Consider using alternative materials," generate a report to send to your boss.

[0200] Based on this prompt, the AI ​​model generates appropriate sentences.

[0201] (Application example 1)

[0202] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0203] In the food delivery industry, it has traditionally been difficult to quickly generate appropriate response messages based on simple notes and feedback from customers. As a result, customer responses have taken time and customer satisfaction has declined. The present invention aims to provide a system that automatically generates appropriate response messages based on notes from customers, thereby achieving fast and effective customer responses.

[0204] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0205] In this invention, the server includes means for receiving input scribbled notes, means for tokenizing the received notes and removing stop words and performing stemming, means for performing syntactic analysis, entity recognition, and sentiment analysis based on the preprocessed data, means for generating a response message for food delivery using appropriate grammar, syntax, and tone, and means for displaying the generated text to the user and accepting feedback. This makes it possible to automatically generate a quick and appropriate response message based on a simple note from a customer, thereby improving the efficiency and satisfaction of customer service.

[0206] "Scribbled notes" refer to short notes or comments that are simply entered by the user.

[0207] "Tokenization" refers to breaking down text or notes into the smallest meaningful units: words and phrases.

[0208] "Stop word removal" refers to the removal of common words (e.g., particles, auxiliary verbs) that have no meaning in a sentence during the preprocessing process.

[0209] "Stemming" refers to the process of converting words into their base forms or stems.

[0210] "Syntax analysis" refers to analyzing the structure of a sentence and identifying the role of each word or phrase in the sentence.

[0211] "Entity recognition" refers to the technology of identifying specific items or proper nouns within a piece of text.

[0212] "Sentiment analysis" refers to analyzing text data to determine the emotional tone (e.g., positive, negative, neutral) of its content.

[0213] "Situation and purpose awareness" refers to understanding the current situation, context, and the objectives to be achieved based on analyzed data.

[0214] "Food delivery response message" refers to text generated for customer responses related to food delivery services.

[0215] "Grammar, syntax, and tone" refers to the grammatical rules used to generate sentences, the structure of sentences, and the tone and mood of sentences.

[0216] "Means for displaying to the user and receiving feedback" refers to a function for displaying the generated text to the user and receiving corrections and comments from the user regarding the text.

[0217] This invention is a system that automatically generates appropriate response messages from simple notes entered by the user in order to improve the efficiency of customer responses in food delivery.

[0218] Hardware and software used

[0219] The hardware used to implement this system is a small smart device (e.g., a smartphone), and the software uses Python, Google® Cloud Natural Language API, and OpenAI®'s GPT-4® API.

[0220] What the program does

[0221] The server receives a user's scribbled note, tokenizes it, removes stop words, and performs stemming. Based on the preprocessed data, it performs syntactic analysis, entity recognition, and sentiment analysis to understand the content of the note and its emotional tone.

[0222] The server then uses the analyzed data to recognize the situation and purpose and generate an appropriate response message for the food delivery, paying attention to grammar, syntax, and tone to ensure a proper response for the customer.

[0223] The generated text is displayed on the user's terminal, and the user can make corrections or provide feedback. The user's feedback and corrections are taken into account by the server and reflected in the next message generation process.

[0224] Specific examples

[0225] For example, a user enters a note on a smart device like this: "My order is over an hour late. Please do something!" This note is sent to the server, where it is tokenized, stopwords are removed, and stemmed. Then, syntactic analysis, entity recognition, and sentiment analysis are performed to analyze the content of the note.

[0226] Based on the analysis results, the system will generate an appropriate response message, such as:

[0227] plain

[0228] Dear Customer,

[0229] We sincerely apologize for the delay in delivering your order. We are currently monitoring the delivery status and will respond promptly. We would appreciate it if you could use a coupon instead for your next order.

[0230] We sincerely apologize for any inconvenience caused.

[0231] The generated message is displayed on the user's smart device, where the user can review and modify it if necessary. Modified messages help the system learn and generate even more accurate response messages in the future.

[0232] Prompt Sentence Examples

[0233] Use the following as your prompt:

[0234] plain

[0235] User Input: "My order is over an hour late. Please do something!"

[0236] Output formats:

[0237] Dear Customer,

[0238] [Sentence generation]

[0239] We sincerely apologize for any inconvenience caused.

[0240] This system will make it possible to improve the efficiency and satisfaction of customer service in food delivery services.

[0241] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0242] Step 1:

[0243] A user enters a scribbled note into a food delivery-related smart device application and presses the send button. An example entry might be, "My order is over an hour late. Please do something!" This note is then sent to the server.

[0244] Input: Notes entered by the user

[0245] Output: Notes sent to the server

[0246] Step 2:

[0247] The server tokenizes the received note, which is the process of breaking the note into words and phrases, for example, "order," "1 hour," "late," "do something," etc.

[0248] Input: The note received by the server

[0249] Output: Tokenized words and phrases

[0250] Step 3:

[0251] The server removes stop words from the tokenized data and performs stemming. Stop word removal means removing words with little meaning (e.g. particles and auxiliary verbs), and stemming means converting words to their base forms. For example, "I'm late" is converted to "I'll be late."

[0252] Input: Tokenized data

[0253] Output: Stopword-removed and stemmed data

[0254] Step 4:

[0255] The server performs syntactic analysis, entity recognition, and sentiment analysis on the preprocessed data. Syntactic analysis analyzes the structure of the sentence, while entity recognition identifies specific items (e.g., "order"). Sentiment analysis analyzes the tone of the note; for example, "We're late" is recognized as a negative sentiment.

[0256] Input: Preprocessed data

[0257] Output: Results of parsing, entity recognition, and sentiment analysis

[0258] Step 5:

[0259] The server recognizes the situation and purpose based on the analyzed data, which is to understand the situation that the customer is unhappy about the delay in their order and generate an appropriate response message.

[0260] Input: Parsed data

[0261] Output: Recognized situation and purpose

[0262] Step 6:

[0263] The server generates a corresponding food delivery message using the appropriate grammar, syntax, and tone using the GPT-4 API. For example, the following message might be generated:

[0264] plain

[0265] Dear Customer,

[0266] We sincerely apologize for the delay in delivering your order. We are currently monitoring the delivery status and will respond promptly. We would appreciate it if you could use a coupon instead for your next order.

[0267] We sincerely apologize for any inconvenience caused.

[0268] Input: Perceived situation and purpose

[0269] Output: The corresponding message generated

[0270] Step 7:

[0271] The generated message is sent to the user's terminal and displayed. The user can check the generated message and make corrections if necessary, and the feedback is sent to the server.

[0272] Input: The corresponding message generated

[0273] Output: Message and feedback displayed to the user

[0274] Step 8:

[0275] The server learns from the user's corrections and feedback and incorporates them into the next message generation process, allowing the system to generate more accurate messages.

[0276] Input: User corrections and feedback

[0277] Output: Updated message generation model

[0278] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0279] The present invention relates to a system that receives input scribbled notes, recognizes the user's emotions, and automatically generates sentences that are deemed appropriate based on the received notes. Specific embodiments of the system will be described below.

[0280] Note entry and preprocessing

[0281] The user inputs a simple memo into the terminal. For example, it might be something like, "Project progress is delayed. Cause: Materials have not arrived. Solution: Consider using alternative materials." The terminal receives this memo and sends it to the server.

[0282] The server performs pre-processing on the received memos, which includes the following steps:

[0283] 1. Tokenization: Splitting notes into words and phrases.

[0284] 2. Stop word removal: remove words that have little meaning.

[0285] 3. Stemming: Converting words into their base forms.

[0286] For example, tokens such as "Project progress," "Delayed," "Cause," "Materials," "Not arrived," "Countermeasures," "Alternative materials," and "Considering use" are generated.

[0287] Natural Language Processing (NLP)

[0288] The server applies NLP algorithms based on the pre-processed data, a process that includes:

[0289] 1. Syntactic analysis: Analyzing the structure of a sentence and identifying the roles of words and phrases.

[0290] 2. Entity Recognition: Identifying specific items.

[0291] 3. Sentiment analysis: Analyze the tone and sentiment of the note.

[0292] For example, "project progress" is recognized as an entity, and "being late" is analyzed as a negative emotion.

[0293] Emotion Engine

[0294] The system also incorporates an emotion engine to analyze the emotion of the user's input and adjust the tone of the sentence based on that emotion. For example, if a user types "I'm late," and the input indicates a negative emotion, the system will make the tone of the entire sentence more formal and careful.

[0295] The server also stores the user's past emotional data and uses that data to learn the user's emotional patterns, allowing the system to understand the emotions expressed by the user when entering notes in the past and generate more appropriate sentences.

[0296] Furthermore, the system receives user feedback on the generated sentences, recognizes emotions from the feedback, and reflects these in the next sentence generation. This continuous learning process allows the system to generate sentences that better match the user's emotions and tone.

[0297] Sentence generation

[0298] The server generates appropriate sentences based on information from the context recognition module and emotion engine, using correct grammar and syntax and with a tone adjusted by the emotion engine.

[0299] For example, the following sentence is generated:

[0300] To your boss

[0301] Thank you for your hard work. I'll keep you updated on the progress of the project.

[0302] The project is currently behind schedule because the necessary materials have not yet arrived. To address this issue, we are considering using alternative materials.

[0303] I would appreciate any instructions.

[0304] Thank you very much.

[0305] Output and check the text

[0306] The server sends the generated text to the terminal. The terminal displays this text to the user. The user can check the displayed text and make corrections as necessary. The system learns from these corrections and reflects them in the next generation.

[0307] In this way, the system can convert simple notes entered by the user into natural-sounding sentences, significantly reducing the burden of writing. It can also recognize the user's emotions and generate sentences based on those emotions, further improving the efficiency and accuracy of communication.

[0308] The processing flow will be explained below.

[0309] Step 1:

[0310] The user inputs a simple memo into the terminal. For example, it might be something like, "Project progress is delayed. Cause: Materials have not arrived. Solution: Consider using alternative materials." The terminal receives this memo and sends it to the server.

[0311] Step 2:

[0312] The server tokenizes the received memo. During this process, the memo is divided into words and phrases. For example, tokens such as "Project progress," "Delayed," "Cause," "Materials," "Not arrived," "Countermeasures," "Alternative materials," and "Consideration of use" are generated.

[0313] Step 3:

[0314] The server removes stop words from the tokenized data, a process that removes words with little meaning (e.g., "wa" and "no").

[0315] Step 4:

[0316] The server will instead perform stemming to convert the tokens to their root form, for example, "use consideration" to "use consideration suru."

[0317] Step 5:

[0318] The server then performs syntactic analysis on the preprocessed data. This process analyzes the structure of the sentence and identifies the role of each word and phrase. For example, "project progress" is recognized as a noun phrase, while "being late" is analyzed as an adjective phrase.

[0319] Step 6:

[0320] The server performs entity recognition based on the results of the parsing, specifically identifying and classifying specific items (e.g., "project progress" or "materials").

[0321] Step 7:

[0322] The server performs sentiment analysis to determine the tone and sentiment of the note, during which the phrase "late" is analyzed as having a negative sentiment.

[0323] Step 8:

[0324] The server uses an emotion engine to analyze the emotion of the user's input and adjusts the tone of the sentence based on that emotion. For example, if "I'm late" is recognized as a negative emotion, the tone of the entire sentence will be made formal and careful.

[0325] Step 9:

[0326] The server accumulates the user's past emotional data and uses that data to learn the user's emotional patterns, allowing the system to understand the emotions expressed by the user when entering notes in the past and generate more appropriate sentences.

[0327] Step 10:

[0328] The server recognizes the situation and purpose based on the analyzed data and information from the emotion engine, referring to the user's past notes, behavioral patterns, and current project status.

[0329] Step 11:

[0330] The server generates appropriate sentences based on information from the context recognition module and emotion engine, using correct grammar and syntax and with a tone adjusted by the emotion engine.

[0331] Step 12:

[0332] The server sends the generated text to the terminal. For example, the following text is generated:

[0333] To your boss

[0334] Thank you for your hard work. I'll keep you updated on the progress of the project.

[0335] The project is currently behind schedule because the necessary materials have not yet arrived. To address this issue, we are considering using alternative materials.

[0336] I would appreciate any instructions.

[0337] Thank you very much.

[0338] Step 13:

[0339] The terminal displays the generated text to the user, who can then check it and make corrections as necessary.

[0340] Step 14:

[0341] If the user makes any corrections, they are sent to the server, which learns the corrections and applies them to the next sentence generation process.

[0342] In this way, the user simply inputs a simple note, and the server generates an appropriate sentence, which the device then provides to the user. Furthermore, by combining it with an emotion engine, the server can generate sentences with an appropriate tone based on the user's emotions, further improving the efficiency and accuracy of communication.

[0343] Example 2

[0344] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0345] In recent years, there has been a demand for technology that can automatically generate meaningful sentences from inputs such as short notes or scribbles. However, conventional systems have difficulty in properly reflecting emotion and tone, and have limitations in generating accurate and appropriate sentences based on the received information. Furthermore, they are unable to incorporate user feedback on the generated sentences, making it difficult to generate sentences that are in line with the user's style and intent. This reduces the efficiency and accuracy of sentence creation, causing problems, especially in business communication.

[0346] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0347] In this invention, the server includes a means for receiving input information, a means for tokenizing the received information, removing unnecessary words, and performing stemming, a means for performing syntactic analysis, item recognition, and sentiment analysis based on the preprocessed data, a means for generating sentences using appropriate grammar, structure, and tone, and a means for displaying the generated sentences. This makes it possible to generate natural sentences that reflect emotions and tone even from simple input notes. Furthermore, by learning from users' past data and feedback, it is possible to generate sentences that reflect the intentions and emotions of individual users.

[0348] The "means for receiving input information" refers to a process or device for transmitting memos or text data input by a user through a terminal to a server.

[0349] "Tokenization" is a text processing technique that divides notes or text data into words and phrases.

[0350] "Removal of unnecessary words" is a process of removing words with little meaning (e.g., "wa," "ga," "no," etc.) from text data.

[0351] "Stemming" is the process of converting words into their root forms (stem), thereby unifying different forms of words.

[0352] "Syntax analysis" is the process of analyzing the grammatical structure of text and identifying the grammatical role of each word and phrase.

[0353] "Item recognition" is the technique of identifying important entities in text (e.g., people's names, organization names, dates, etc.).

[0354] "Sentiment analysis" is the process of analyzing the tone or sentiment of text data, which identifies the sentiment (positive, negative, neutral, etc.) conveyed in the input text.

[0355] A "means for generating sentences with appropriate grammar, structure, and tone" is an algorithm or model (e.g., a generative AI model) for generating natural-sounding sentences based on preprocessed and analyzed data.

[0356] The "means for displaying the generated text" refers to a process or interface for displaying the generated text on the user's terminal.

[0357] "Means of inferring the situation based on preprocessed data and referring to the user's past information and behavioral patterns" is a technology that learns the user's past input data and behavioral history to more accurately infer the situation in response to the current input.

[0358] "When a user makes corrections to a generated sentence, a means for learning those corrections and reflecting them in the generation process" is a feedback function that learns the corrections made by the user to the generated sentence and reflects them in the next sentence generation.

[0359] The present invention relates to a system that receives simple notes entered by a user, recognizes the user's emotions based on the notes, and automatically generates appropriate sentences. This system functions through the interaction between a server, a terminal, and the user.

[0360] Note entry and preprocessing

[0361] First, the user inputs a simple memo into the device. For example, the memo might read, "Project progress is delayed. Cause: Materials have not arrived. Solution: Consider using alternative materials." The device receives this memo and sends it to the server via an Internet connection.

[0362] The server preprocesses the received memos using the following steps: First, the memos are tokenized and divided into words and phrases. Then, unnecessary words are removed, and finally, stemming is performed. This generates tokens such as "project progress," "delayed," "cause," "materials," "not arrived," "countermeasures," "alternative materials," and "considering use."

[0363] Natural Language Processing (NLP)

[0364] The server then applies NLP algorithms to the preprocessed data. Specifically, this process involves syntactic analysis, item recognition, and sentiment analysis. Syntactic analysis analyzes the structure of the sentence and identifies the role of each word or phrase. Item recognition identifies specific items (entities), while sentiment analysis analyzes the tone and sentiment of the note. For example, "project progress" is recognized as an entity, and "being late" is analyzed as a negative sentiment.

[0365] Emotion Engine

[0366] The server then uses an emotion engine to analyze the user's emotions as they type. The system uses NLP models such as BERT and RoBERTa. The emotion engine accumulates the user's past emotional data and uses that data to learn the user's emotional patterns. This allows the system to understand the emotions with which the user has entered notes in the past and adjust the tone of the text based on those emotions.

[0367] Sentence generation

[0368] The server generates appropriate sentences based on information from the situation recognition module and emotion engine. This process uses generative AI models such as GPT-3 and T5. The generated sentences have correct grammar and structure, and are written with a tone adjusted by the emotion engine.

[0369] As a concrete example, the following sentence is generated:

[0370] To your boss

[0371] Thank you for your hard work. I'll keep you updated on the progress of the project.

[0372] The project is currently behind schedule because the necessary materials have not yet arrived. To address this issue, we are considering using alternative materials.

[0373] I would appreciate any instructions.

[0374] Thank you very much.

[0375] Output and check the text

[0376] Finally, the server sends the generated text to the terminal. The terminal displays the generated text to the user. The user can check the displayed text and make corrections as necessary. The corrections made by the user are fed back to the system and reflected in the next text generation.

[0377] This system allows users to simply input simple notes and convert them into natural-sounding sentences, significantly reducing the burden of writing. It also recognizes the user's emotions and generates sentences based on those emotions, improving the efficiency and accuracy of communication.

[0378] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0379] Step 1:

[0380] The user enters a simple note into the input field of the terminal. The entered note is text data that reads, "Project progress is delayed. Cause: Materials have not arrived. Solution: Consider using alternative materials." The terminal receives this note and sends it to the server via the Internet. Specifically, the terminal monitors the user's input in real time and sends the note in text format as an HTTP POST request to the server's API endpoint. The input is the user's note, and the output is the data to be sent to the server.

[0381] Step 2:

[0382] The server preprocesses the received notes. First, it tokenizes the notes and splits them into words and phrases. Next, it removes unnecessary words, and finally it performs stemming. For example, tokens such as "project progress," "delayed," "cause," "materials," "not arrived," "countermeasures," "alternative materials," and "considering use" are generated. Specifically, the server uses a natural language processing library (e.g., SpaCy) to execute scripts for tokenization, removal of unnecessary words, and stemming. The input is the initial note data, and the output is the preprocessed token data.

[0383] Step 3:

[0384] The server applies NLP algorithms to the preprocessed data. First, it performs syntactic analysis to analyze the structure of the sentence. Next, it performs item recognition to identify specific items (entities). Finally, it performs sentiment analysis to analyze the tone and sentiment of the note. For example, "project progress" is recognized as an entity, and "being late" is analyzed as having a negative sentiment. Specifically, the server performs these processes using a deep learning model (e.g., BERT). The input is the preprocessed token data, and the output is the analyzed data and sentiment labels.

[0385] Step 4:

[0386] The server uses an emotion engine to analyze the user's emotions in more detail. It uses models such as BERT and RoBERTa to identify the emotion of the user's input and adjust the tone of the sentence based on that emotion. Specifically, the server inputs previously accumulated user emotion data into a learning model to learn emotion patterns. The input is the analyzed data and emotion labels, and the output is adjusted emotion parameters.

[0387] Step 5:

[0388] The server generates appropriate sentences based on the context recognition module and emotion engine. In this process, it uses a generative AI model (e.g., GPT-3 or T5) to create sentences with correct grammar, structure, and tone. For example, the following sentences are generated:

[0389] To your boss

[0390] Thank you for your hard work. I'll keep you updated on the progress of the project.

[0391] The project is currently behind schedule because the necessary materials have not yet arrived. To address this issue, we are considering using alternative materials.

[0392] I would appreciate any instructions.

[0393] Thank you very much.

[0394] Specifically, the server provides the analyzed data and emotion parameters as input to the generative AI model to generate sentences. The input is the adjusted emotion parameters and analyzed data, and the output is the generated text sentence.

[0395] Step 6:

[0396] The server sends the generated text to the device. Specifically, the server sends the generated text to the device's API endpoint as an HTTP POST request. The input is the generated text data, and the output is the data to be sent to the device.

[0397] Step 7:

[0398] The terminal displays the generated text to the user, who can review the displayed text and make corrections as necessary. Specifically, the terminal displays the received text on a user interface and provides an editable text area. The input is the text data received from the server, and the output is the displayed text. If the user makes corrections, that feedback is sent to the system and reflected in the next text generation.

[0399] (Application example 2)

[0400] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0401] In security services, field personnel are required to quickly and accurately report handwritten notes. However, it is difficult to directly convert handwritten notes into reports, which can result in inconsistent tone and content. It is also challenging to generate appropriate reports that take into account the emotions and tone of field personnel. Furthermore, it is difficult to reflect personnel's corrections to generated reports in the next report. A system that can solve these problems is needed.

[0402] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for receiving input, scribbled notes; means for tokenizing the received notes and removing stop words and performing stemming; means for performing syntactic analysis, entity recognition, and sentiment analysis based on the preprocessed data; means for displaying the generated text to the field staff; means for recognizing the sentiment of the notes entered by the field staff and adjusting the tone of the report; means for inferring the situation based on the preprocessed data and referring to the field staff's past notes and behavioral patterns; and means for learning and reflecting the corrections made by the field staff in the generation process when the field staff makes corrections to the generated text. This makes it possible to quickly generate consistent and appropriate reports from handwritten notes, taking the field staff's sentiment and tone into consideration, and improving the quality of the reports by reflecting the staff's corrections in the next generation process.

[0403] "Means for receiving input scribbled notes" refers to devices or software that have the function of receiving simple handwritten notes entered by field personnel in digital format.

[0404] "Tokenization" is the process of dividing input text into words and phrases.

[0405] "Stop word removal" is the process of removing words with little meaning (such as "no", "wa", and "ni") from text.

[0406] "Stemming" is the process of converting words into their root forms.

[0407] "Syntax analysis" is a method of analyzing the structure of a sentence and clarifying the role of each word and phrase.

[0408] "Entity recognition" is the technique of identifying specific items or information within text (e.g., names of people, places, dates, etc.).

[0409] "Sentiment analysis" is a technique that analyzes a user's emotions and tone from the content of text and identifies positive, negative, or neutral emotions.

[0410] "Means for recognizing the situation and purpose" refers to devices or software that understand the situation and purpose from the analyzed data and then perform the next processing based on that information.

[0411] "Means for generating sentences using appropriate grammar, syntax, and tone" refers to devices or software that automatically create sentences using correct grammar and syntax and with a tone that reflects the results of sentiment analysis.

[0412] The "means for displaying the generated text to the on-site staff" refers to a device or software that displays the generated report on the on-site staff's device.

[0413] The "means for recognizing the emotion of a memo and adjusting the tone of a report" is a technology that analyzes the emotion of an input memo and appropriately adjusts the tone of a report based on that emotion.

[0414] "Means for inferring the situation by referring to past notes and behavioral patterns" refers to devices or software that refer to a database of past notes and behavioral patterns of field personnel and use that information to infer the current situation.

[0415] "Means for learning corrections and reflecting them in the generation process" refers to a technology that learns the corrections made by field personnel to generated text and reflects that knowledge when generating text from the next time onwards.

[0416] The present invention provides a system that enables on-site personnel to quickly and accurately report handwritten notes. Specific embodiments of the system will be described in detail below.

[0417] Note entry and preprocessing

[0418] The user inputs a simple handwritten note into a device such as a smartphone or robot. For example, the note might read, "Door forced open, intruder detected, evidence: fingerprint, alarm not activated." The device receives the note and sends it to the server. The server then performs the following preprocessing on the received note:

[0419] 1. Tokenization:

[0420] The notes are segmented into words and phrases, generating tokens such as "door," "forced," "open," "intruder," "discovered," "evidence," "fingerprint," "alarm," and "not activated."

[0421] 2. Stop word removal:

[0422] Remove words with little meaning. For example, stop words such as "teki" (target), "no" (of), and "ni" (to) are removed.

[0423] 3. Stemming:

[0424] Convert words to their root forms, for example, "open" is converted to "open."

[0425] Natural Language Processing (NLP)

[0426] Based on the preprocessed data, the server applies the following NLP algorithms:

[0427] 1. Parsing:

[0428] It analyzes the structure of a sentence and identifies the role of each word and phrase, for example, "door" is recognized as the subject and "open" as a verb.

[0429] 2. Entity Recognition:

[0430] Identifies a specific item. For example, "Evidence:Fingerprint" is recognized as an entity.

[0431] 3. Sentiment analysis:

[0432] Analyze the tone and sentiment of the memo. For example, "forced," "discovered," and "not working" are analyzed as negative sentiments.

[0433] Emotion Engine

[0434] The server uses an emotion engine to analyze the user's emotion when inputting information and adjusts the tone of the report accordingly. For example, if a user inputs a note containing negative emotions such as "forced," "discovered," or "not working," the tone of the entire document will be made more formal and careful. The server also accumulates the user's past emotional data and behavioral patterns and uses them to generate documents optimized for the current situation.

[0435] Text generation and display

[0436] The server generates sentences using appropriate grammar, syntax, and tone based on the output of the context recognition module and emotion engine. For example, the following sentences are generated:

[0437] "Security, this is a scene surveillance report. The door was forced open and an intruder was found. Fingerprints were left as evidence and the alarm was not activated. Immediate action is required."

[0438] The generated sentences are displayed on the terminals of the on-site personnel. The on-site personnel can check the displayed sentences and make corrections as necessary. These corrections are fed back into the system and reflected in the next sentence generation.

[0439] Examples of specific examples and prompts

[0440] As a specific example, consider the case where a field worker inputs the following memo:

[0441] Note: "Door forced open. Intruder found. Evidence: fingerprint. Alarm not activated."

[0442] An example of a generated prompt statement:

[0443] Prompt input to model:

[0444] On-site report notes: "Door forced open. Intruder found. Evidence: fingerprints. Alarm not activated."

[0445] Generate sentences based on the analysis results.

[0446] The server receives the prompt and uses a predefined algorithm and emotion engine to generate the appropriate response for the field representative, enabling fast and accurate reporting while also taking into account the representative's emotion and tone.

[0447] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0448] Step 1:

[0449] User Input

[0450] The user inputs a simple handwritten note into a device such as a smartphone or robot. For example, a note such as "Door forced open, intruder detected, evidence: fingerprint, alarm not activated" is input. The input note is received by the device.

[0451] Step 2:

[0452] Send a note

[0453] The terminal sends the received memo to the server, and when the memo arrives at the server, the next process begins.

[0454] Step 3:

[0455] Pretreatment

[0456] The server pre-processes the received notes, which includes:

[0457] Tokenization: Splits the input notes into words and phrases, generating tokens such as "door," "forced," "open," "intruder," "discovered," "evidence," "fingerprint," "alarm," and "not activated."

[0458] Stop word removal: Remove words with little meaning. For example, "teki" (target), "no" (of), and "ni" (to) are removed.

[0459] Stemming: Converting words into their root forms, for example, "open" is converted to "open."

[0460] Step 4:

[0461] Natural Language Processing (NLP)

[0462] The server applies the following NLP algorithms based on the preprocessed data:

[0463] Syntactic analysis: Analyzes the structure of a sentence and identifies the role of each word or phrase. For example, "door" is recognized as the subject and "open" as a verb.

[0464] Entity Recognition: Identifying specific items or information within text (e.g., "Evidence: Fingerprint").

[0465] Sentiment Analysis: Analyzes the user's emotions and tone from the text content to identify positive, negative, or neutral sentiment. For example, "forced," "discovered," and "not working" are analyzed as negative sentiments.

[0466] Step 5:

[0467] Emotion Engine

[0468] The server uses an emotion engine to analyze the user's emotions when they type and adjust the tone of the report accordingly. For example, it generates a more formal and careful tone for notes containing negative emotions. It also references the user's past emotional data and behavioral patterns to generate a report optimized for the current situation.

[0469] Step 6:

[0470] Sentence generation

[0471] Based on the output of the context recognition module and emotion engine, the server generates sentences using appropriate grammar, syntax, and tone. For example, the following sentences are generated:

[0472] "Security, this is a scene surveillance report. The door was forced open and an intruder was found. Fingerprints were left as evidence and the alarm was not activated. Immediate action is required."

[0473] Step 7:

[0474] Displaying text

[0475] The generated text is sent to a terminal and displayed to the on-site staff, who can then review the text and make corrections as necessary.

[0476] Step 8:

[0477] Correction Feedback

[0478] If a field worker makes corrections to the text, the server stores the corrections as learning data and reflects them in the next text generation process. This continuous feedback improves the accuracy and adaptability of text generation.

[0479] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0480] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0481] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0482] [Second embodiment]

[0483] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0484] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0485] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0486] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0487] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0488] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0489] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0490] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0491] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0492] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0493] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0494] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0495] The present invention relates to a system that receives inputted scribbled notes and automatically generates sentences that are deemed appropriate for the situation and purpose from them. Specific embodiments of the system will be described below.

[0496] Note entry and preprocessing

[0497] The user inputs a simple memo into the terminal. For example, it might be something like, "Project progress is delayed. Cause: Materials have not arrived. Solution: Consider using alternative materials." The terminal receives this memo and sends it to the server.

[0498] The server performs pre-processing on the received memos, which includes the following steps:

[0499] 1. Tokenization: Splitting notes into words and phrases.

[0500] 2. Stop word removal: remove words that have little meaning.

[0501] 3. Stemming: Converting words into their base forms.

[0502] For example, tokens such as "Project progress," "Delayed," "Cause," "Materials," "Not arrived," "Countermeasures," "Alternative materials," and "Considering use" are generated.

[0503] Natural Language Processing (NLP)

[0504] The server applies NLP algorithms based on the pre-processed data, a process that includes:

[0505] 1. Syntactic analysis: Analyzing the structure of a sentence and identifying the roles of words and phrases.

[0506] 2. Entity Recognition: Identifying specific items.

[0507] 3. Sentiment analysis: Analyze the tone and sentiment of the note.

[0508] For example, "project progress" is recognized as an entity, and "being late" is analyzed as a negative emotion.

[0509] Situational Awareness

[0510] The server uses the analyzed data to understand the current situation and objectives. This process takes into account the following factors:

[0511] Understand context: See notes users have previously entered and the current status of projects.

[0512] User profile: Based on the user's past behavioral patterns and general work methods.

[0513] For example, if the "project progress" is "delayed" and the cause is "materials not arriving," it is determined that this is a situation that requires countermeasures.

[0514] Sentence generation

[0515] The server generates appropriate sentences based on the information from the situational awareness module. The following factors are taken into account in this process:

[0516] Grammar and Syntax: Use correct grammar and appropriate syntax.

[0517] Tone of writing: Generate text in a tone appropriate for the reader (e.g., a formal tone appropriate for business correspondence).

[0518] For example, the following sentence is generated:

[0519] To your boss

[0520] Thank you for your hard work. I'll keep you updated on the progress of the project.

[0521] The project is currently behind schedule because the necessary materials have not yet arrived. To address this issue, we are considering using alternative materials.

[0522] I would appreciate any instructions.

[0523] Thank you very much.

[0524] Output and check the text

[0525] The server sends the generated text to the terminal. The terminal displays this text to the user. The user can check the displayed text and make corrections as necessary. The system learns from these corrections and reflects them in the next generation.

[0526] In this way, the system converts simple notes entered by users into natural-sounding sentences, significantly reducing the burden of writing and improving the efficiency and accuracy of communication.

[0527] The processing flow will be explained below.

[0528] Step 1:

[0529] The user inputs a simple memo into the terminal, which may include keywords or short phrases such as "Project progress is delayed. Cause: Materials have not arrived. Solution: Consider using alternative materials."

[0530] Step 2:

[0531] The terminal receives the entered memo and transmits the memo to the server.

[0532] Step 3:

[0533] The server tokenizes the received memo. During this process, the memo is divided into words and phrases. For example, tokens such as "Project progress," "Delayed," "Cause," "Materials," "Not arrived," "Countermeasures," "Alternative materials," and "Consideration of use" are generated.

[0534] Step 4:

[0535] The server removes stop words from the tokenized data, a process that removes words with little meaning (e.g., "wa" and "no").

[0536] Step 5:

[0537] The server performs stemming on the tokenized and stop-word-removed data. Stemming is the process of converting words into their root forms, for example, converting "use consideration" into "use consideration suru."

[0538] Step 6:

[0539] The server then performs syntactic analysis on the preprocessed data, a process that analyzes the structure of the sentence and identifies the role of each word and phrase.

[0540] Step 7:

[0541] The server performs entity recognition based on the results of the parsing, specifically identifying and classifying specific items (e.g., "project progress" or "materials").

[0542] Step 8:

[0543] The server performs sentiment analysis to determine the tone and sentiment of the note, during which the phrase "late" is analyzed as having a negative sentiment.

[0544] Step 9:

[0545] The server recognizes the situation and purpose based on the analyzed data, referring to the user's past notes, behavioral patterns, and current project status.

[0546] Step 10:

[0547] The server generates appropriate sentences based on the information recognized by the context recognition module, using correct grammar and syntax and in a formal business tone.

[0548] Step 11:

[0549] The server transmits the generated text to the terminal.

[0550] Step 12:

[0551] The terminal displays the generated text to the user, who can then check it and make corrections as necessary.

[0552] Step 13:

[0553] If the user makes any corrections, they are sent to the server, which learns the corrections and applies them to the next sentence generation process.

[0554] In this way, the user only needs to input a simple note, and the server will generate an appropriate sentence, which the device will then provide to the user. This completes the entire process, significantly reducing the burden of writing sentences.

[0555] Example 1

[0556] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0557] Conventionally, even if a user inputs a simple note, converting it into a proper sentence takes a lot of time and effort. Furthermore, manual sentence creation is prone to errors, which can reduce the efficiency and accuracy of communication. An effective system is needed to solve these problems.

[0558] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0559] In this invention, the server includes means for receiving input simple memos, means for dividing the received memos into words and phrases, removing words with little meaning, and converting words into base forms, means for performing syntactic analysis, entity recognition, and sentiment analysis based on the preprocessed data, means for recognizing situations and purposes based on the analyzed data, means for generating sentences using correct grammar, appropriate syntax, and a tone tailored to the reader, means for displaying the generated sentences to the user, and means for the user to check and correct the displayed sentences. This enables the user to quickly generate high-quality and appropriate sentences simply by inputting simple memos.

[0560] "Entered brief notes" are short, condensed pieces of information or items entered by a user using a terminal.

[0561] The "receiving means" is a means for receiving information sent from a terminal and inputting it into the server.

[0562] "Split into words and phrases" means splitting the inputted notes into individual words and phrases to make them easier to process.

[0563] "Removing words with little meaning" means removing words that are less important as information in the preprocessing stage.

[0564] "Converting words to base forms" refers to converting words to their base forms in preparation for further analysis.

[0565] "Syntax analysis" is the process of analyzing the structure of a sentence and identifying the roles and relationships of each word and phrase.

[0566] "Entity recognition" refers to identifying and tagging specific items or concepts.

[0567] "Sentiment analysis" is the analysis of tone or emotion (positive, negative, neutral, etc.) within text.

[0568] "Recognizing the situation and purpose" means understanding and analyzing the current situation and goals based on analyzed data.

[0569] "Correct grammar, appropriate syntax, and appropriate tone for the reader" means producing documents that are grammatically correct, well-structured, and have an appropriate tone for the reader.

[0570] "Means for generating" refers to means for automatically creating appropriate sentences based on information from preprocessing and analysis.

[0571] The "display means" is a means for showing the text generated by the server to the user.

[0572] The "means for making corrections" is a means by which the user can add or correct the generated text.

[0573] The present invention relates to a system that automatically generates appropriate sentences according to the situation and purpose based on simple notes entered by a user. A specific embodiment of this system will be described below.

[0574] Note entry and preprocessing

[0575] The user inputs a simple memo into the terminal. For example, a memo such as "Project progress is delayed. Cause: Materials have not arrived. Solution: Consider using alternative materials." The terminal receives this memo and sends it to the server. The server performs preprocessing on the received memo. This preprocessing includes the following steps:

[0576] 1. Tokenization: Divide notes into words and phrases. For example, tokens such as "Project progress," "Delayed," "Cause," "Materials," "Not arrived," "Countermeasures," "Alternative materials," and "Considering use" are generated.

[0577] 2. Stop word removal: Remove words with little meaning. For example, words such as "no", "ga", and "wa" are removed.

[0578] 3. Stemming: Converting words into their root forms, for example, converting "materials not yet arrived" into "materials not yet arrived."

[0579] The server performs these preprocessing steps using an NLP library written in Python (e.g., NLTK or spaCy).

[0580] Natural Language Processing (NLP)

[0581] The server applies NLP algorithms based on the pre-processed data. This process includes:

[0582] 1. Syntactic analysis: Analyzes the structure of a sentence and identifies the roles of words and phrases. For example, "project progress" is recognized as the subject and "delayed" as the verb.

[0583] 2. Entity Recognition: Identifying specific items, for example, "Project Progress" is recognized as an entity.

[0584] 3. Sentiment analysis: Analyze the tone and sentiment of the note. For example, "I'm late" is analyzed as a negative sentiment.

[0585] The server performs these processes using generative AI models such as BERT and GPT-3.

[0586] Situational Awareness

[0587] The server uses the analyzed data to understand the current situation and objectives. This process takes into account the following factors:

[0588] Understand context: See notes users have previously entered and the current status of projects.

[0589] User profile: Based on the user's past behavioral patterns and general work methods.

[0590] For example, if the "project progress" is "delayed" and the cause is "materials not yet arrived," it is determined that a countermeasure is required. The server accesses the database and refers to past notes and project history to understand the current situation.

[0591] Sentence generation

[0592] The server generates an appropriate sentence based on the information from the situational awareness module. The following factors are taken into account in this process:

[0593] Grammar and Syntax: Use correct grammar and appropriate syntax.

[0594] Tone of writing: Generate text in a tone appropriate for the reader, for example, using a formal tone appropriate for business correspondence.

[0595] The server uses a generative AI model (e.g., GPT-3) to generate sentences based on the analysis results and contextual awareness information. An example of a generated sentence is as follows:

[0596] To your boss

[0597] Thank you for your hard work. I'll keep you updated on the progress of the project.

[0598] The project is currently behind schedule because the necessary materials have not yet arrived. To address this issue, we are considering using alternative materials.

[0599] I would appreciate any instructions.

[0600] Thank you very much.

[0601] Output and check the text

[0602] The server sends the generated text to the terminal. The terminal displays this text to the user. The user checks the displayed text and makes corrections as necessary. The system learns from these corrections and reflects them in the next generation.

[0603] In this way, the system converts simple notes entered by users into natural-sounding sentences, significantly reducing the burden of writing and improving the efficiency and accuracy of communication.

[0604] Prompt Sentence Examples

[0605] Examples of prompts to input to a generative AI model include:

[0606] Based on the memo "Project progress is delayed. Cause: Materials have not arrived. Solution: Consider using alternative materials," generate a report to send to your boss.

[0607] Based on this prompt, the AI ​​model generates appropriate sentences.

[0608] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0609] Processing steps of this system's program

[0610] Below, the processing flow of the program of this system will be explained step by step, and the specific operations, inputs, and outputs performed at each step will be shown.

[0611] Step 1: Enter and send your note

[0612] Description: The user inputs a simple memo into the terminal. For example, "Project progress is delayed. Cause: Materials have not arrived. Solution: Consider using alternative materials." The terminal receives this memo and sends it to the server.

[0613] Input: Notes entered by the user into the device.

[0614] Output: The note received by the server.

[0615] Specific operation: The user enters a note into the input field on their smartphone or computer and presses the "Send" button, which sends the note to the server.

[0616] Step 2: Preprocessing notes

[0617] Description: The server performs the following data preprocessing on the received memo.

[0618] Tokenization: Splitting notes into words and phrases.

[0619] Stopword removal: Remove words that have little meaning.

[0620] Stemming: Converting words into their root forms.

[0621] Input: Received note.

[0622] Output: Preprocessed data (tokenized, stopword-removed, stemmed words and phrases).

[0623] Specific operation: The server uses Python NLP libraries (NLTK and spaCy) to divide the notes into tokens such as "project progress," "delayed," "cause," "materials," "not arrived," "countermeasures," "alternative materials," and "considering use," and then removes stop words and performs stemming.

[0624] Step 3: NLP analysis

[0625] Description: The server performs parsing, entity recognition, and sentiment analysis on the preprocessed data.

[0626] Syntactic analysis: Analyzing the structure of a sentence.

[0627] Entity Recognition: Identifying specific items.

[0628] Sentiment analysis: Analyze the tone and sentiment of your notes.

[0629] Input: Preprocessed data.

[0630] Output: Results of parsing, entity recognition, and sentiment analysis.

[0631] Specific operation: The server uses a generative AI model such as BERT or GPT-3 to obtain the analysis result, where "project progress" is the subject and "delayed" is a verb that expresses negative sentiment.

[0632] Step 4: Situational Awareness

[0633] Description: The server recognizes the current situation and objectives based on the analyzed data.

[0634] Understand context: See previously entered notes and project status.

[0635] User profile: Based on past behavioral patterns and general work methods.

[0636] Input: Results of parsing, entity recognition, and sentiment analysis.

[0637] Output: Current situation and objectives.

[0638] Specific operation: The server accesses the database, references past notes and history such as "project progress is behind schedule" or "the cause is the non-arrival of materials," and recognizes that action is needed.

[0639] Step 5: Sentence generation

[0640] Description: The server generates appropriate text based on the information from the situational awareness module.

[0641] Grammar and Syntax: Use correct grammar and appropriate syntax.

[0642] Tone of writing: Generate text in a tone appropriate for the reader.

[0643] Input: Current situation and objectives.

[0644] Output: The generated sentence.

[0645] Specific operation: The server uses a generative AI model (such as GPT-3) to generate sentences such as, "To my boss, I'm reporting on the progress of the project. The project is currently delayed because the materials have not yet arrived."

[0646] Step 6: Output and check the text

[0647] Description: The server sends the generated text to the terminal, which displays it to the user. The user can check the displayed text and make corrections if necessary.

[0648] Input: The generated sentence.

[0649] Output: The text reviewed and corrected by the user.

[0650] Specific operation: The generated text is displayed on the terminal, and the user can check and edit it on the terminal. The edits are recorded by the server and reflected in the next text generation.

[0651] Prompt Sentence Examples

[0652] Examples of prompts to input to a generative AI model include:

[0653] Based on the memo "Project progress is delayed. Cause: Materials have not arrived. Solution: Consider using alternative materials," generate a report to send to your boss.

[0654] Based on this prompt, the AI ​​model generates appropriate sentences.

[0655] (Application example 1)

[0656] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0657] In the food delivery industry, it has traditionally been difficult to quickly generate appropriate response messages based on simple notes and feedback from customers. As a result, customer responses have taken time and customer satisfaction has declined. The present invention aims to provide a system that automatically generates appropriate response messages based on notes from customers, thereby achieving fast and effective customer responses.

[0658] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0659] In this invention, the server includes means for receiving input scribbled notes, means for tokenizing the received notes and removing stop words and performing stemming, means for performing syntactic analysis, entity recognition, and sentiment analysis based on the preprocessed data, means for generating a response message for food delivery using appropriate grammar, syntax, and tone, and means for displaying the generated text to the user and accepting feedback. This makes it possible to automatically generate a quick and appropriate response message based on a simple note from a customer, thereby improving the efficiency and satisfaction of customer service.

[0660] "Scribbled notes" refer to short notes or comments that are simply entered by the user.

[0661] "Tokenization" refers to breaking down text or notes into the smallest meaningful units: words and phrases.

[0662] "Stop word removal" refers to the removal of common words (e.g., particles, auxiliary verbs) that have no meaning in a sentence during the preprocessing process.

[0663] "Stemming" refers to the process of converting words into their base forms or stems.

[0664] "Syntax analysis" refers to analyzing the structure of a sentence and identifying the role of each word or phrase in the sentence.

[0665] "Entity recognition" refers to the technology of identifying specific items or proper nouns within a piece of text.

[0666] "Sentiment analysis" refers to analyzing text data to determine the emotional tone (e.g., positive, negative, neutral) of its content.

[0667] "Situation and purpose awareness" refers to understanding the current situation, context, and the objectives to be achieved based on analyzed data.

[0668] "Food delivery response message" refers to text generated for customer responses related to food delivery services.

[0669] "Grammar, syntax, and tone" refers to the grammatical rules used to generate sentences, the structure of sentences, and the tone and mood of sentences.

[0670] "Means for displaying to the user and receiving feedback" refers to a function for displaying the generated text to the user and receiving corrections and comments from the user regarding the text.

[0671] This invention is a system that automatically generates appropriate response messages from simple notes entered by the user in order to improve the efficiency of customer responses in food delivery.

[0672] Hardware and software used

[0673] The hardware used to implement this system is a small smart device (e.g., a smartphone), and the software uses Python, Google Cloud Natural Language API, and OpenAI's GPT-4 API.

[0674] What the program does

[0675] The server receives a user's scribbled note, tokenizes it, removes stop words, and performs stemming. Based on the preprocessed data, it performs syntactic analysis, entity recognition, and sentiment analysis to understand the content of the note and its emotional tone.

[0676] The server then uses the analyzed data to recognize the situation and purpose and generate an appropriate response message for the food delivery, paying attention to grammar, syntax, and tone to ensure a proper response for the customer.

[0677] The generated text is displayed on the user's terminal, and the user can make corrections or provide feedback. The user's feedback and corrections are taken into account by the server and reflected in the next message generation process.

[0678] Specific examples

[0679] For example, a user enters a note on a smart device like this: "My order is over an hour late. Please do something!" This note is sent to the server, where it is tokenized, stopwords are removed, and stemmed. Then, syntactic analysis, entity recognition, and sentiment analysis are performed to analyze the content of the note.

[0680] Based on the analysis results, the system will generate an appropriate response message, such as:

[0681] plain

[0682] Dear Customer,

[0683] We sincerely apologize for the delay in delivering your order. We are currently monitoring the delivery status and will respond promptly. We would appreciate it if you could use a coupon instead for your next order.

[0684] We sincerely apologize for any inconvenience caused.

[0685] The generated message is displayed on the user's smart device, where the user can review and modify it if necessary. Modified messages help the system learn and generate even more accurate response messages in the future.

[0686] Prompt Sentence Examples

[0687] Use the following as your prompt:

[0688] plain

[0689] User Input: "My order is over an hour late. Please do something!"

[0690] Output formats:

[0691] Dear Customer,

[0692] [Sentence generation]

[0693] We sincerely apologize for any inconvenience caused.

[0694] This system will make it possible to improve the efficiency and satisfaction of customer service in food delivery services.

[0695] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0696] Step 1:

[0697] A user enters a scribbled note into a food delivery-related smart device application and presses the send button. An example entry might be, "My order is over an hour late. Please do something!" This note is then sent to the server.

[0698] Input: Notes entered by the user

[0699] Output: Notes sent to the server

[0700] Step 2:

[0701] The server tokenizes the received note, which is the process of breaking the note into words and phrases, for example, "order," "1 hour," "late," "do something," etc.

[0702] Input: The note received by the server

[0703] Output: Tokenized words and phrases

[0704] Step 3:

[0705] The server removes stop words from the tokenized data and performs stemming. Stop word removal means removing words with little meaning (e.g. particles and auxiliary verbs), and stemming means converting words to their base forms. For example, "I'm late" is converted to "I'll be late."

[0706] Input: Tokenized data

[0707] Output: Stopword-removed and stemmed data

[0708] Step 4:

[0709] The server performs syntactic analysis, entity recognition, and sentiment analysis on the preprocessed data. Syntactic analysis analyzes the structure of the sentence, while entity recognition identifies specific items (e.g., "order"). Sentiment analysis analyzes the tone of the note; for example, "We're late" is recognized as a negative sentiment.

[0710] Input: Preprocessed data

[0711] Output: Results of parsing, entity recognition, and sentiment analysis

[0712] Step 5:

[0713] The server recognizes the situation and purpose based on the analyzed data, which is to understand the situation that the customer is unhappy about the delay in their order and generate an appropriate response message.

[0714] Input: Parsed data

[0715] Output: Recognized situation and purpose

[0716] Step 6:

[0717] The server generates a corresponding food delivery message using the appropriate grammar, syntax, and tone using the GPT-4 API. For example, the following message might be generated:

[0718] plain

[0719] Dear Customer,

[0720] We sincerely apologize for the delay in delivering your order. We are currently monitoring the delivery status and will respond promptly. We would appreciate it if you could use a coupon instead for your next order.

[0721] We sincerely apologize for any inconvenience caused.

[0722] Input: Perceived situation and purpose

[0723] Output: The corresponding message generated

[0724] Step 7:

[0725] The generated message is sent to the user's terminal and displayed. The user can check the generated message and make corrections if necessary, and the feedback is sent to the server.

[0726] Input: The corresponding message generated

[0727] Output: Message and feedback displayed to the user

[0728] Step 8:

[0729] The server learns from the user's corrections and feedback and incorporates them into the next message generation process, allowing the system to generate more accurate messages.

[0730] Input: User corrections and feedback

[0731] Output: Updated message generation model

[0732] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0733] The present invention relates to a system that receives input scribbled notes, recognizes the user's emotions, and automatically generates sentences that are deemed appropriate based on the received notes. Specific embodiments of the system will be described below.

[0734] Note entry and preprocessing

[0735] The user inputs a simple memo into the terminal. For example, it might be something like, "Project progress is delayed. Cause: Materials have not arrived. Solution: Consider using alternative materials." The terminal receives this memo and sends it to the server.

[0736] The server performs pre-processing on the received memos, which includes the following steps:

[0737] 1. Tokenization: Splitting notes into words and phrases.

[0738] 2. Stop word removal: remove words that have little meaning.

[0739] 3. Stemming: Converting words into their base forms.

[0740] For example, tokens such as "Project progress," "Delayed," "Cause," "Materials," "Not arrived," "Countermeasures," "Alternative materials," and "Considering use" are generated.

[0741] Natural Language Processing (NLP)

[0742] The server applies NLP algorithms based on the pre-processed data, a process that includes:

[0743] 1. Syntactic analysis: Analyzing the structure of a sentence and identifying the roles of words and phrases.

[0744] 2. Entity Recognition: Identifying specific items.

[0745] 3. Sentiment analysis: Analyze the tone and sentiment of the note.

[0746] For example, "project progress" is recognized as an entity, and "being late" is analyzed as a negative emotion.

[0747] Emotion Engine

[0748] The system also incorporates an emotion engine to analyze the emotion of the user's input and adjust the tone of the sentence based on that emotion. For example, if a user types "I'm late," and the input indicates a negative emotion, the system will make the tone of the entire sentence more formal and careful.

[0749] The server also stores the user's past emotional data and uses that data to learn the user's emotional patterns, allowing the system to understand the emotions expressed by the user when entering notes in the past and generate more appropriate sentences.

[0750] Furthermore, the system receives user feedback on the generated sentences, recognizes emotions from the feedback, and reflects these in the next sentence generation. This continuous learning process allows the system to generate sentences that better match the user's emotions and tone.

[0751] Sentence generation

[0752] The server generates appropriate sentences based on information from the context recognition module and emotion engine, using correct grammar and syntax and with a tone adjusted by the emotion engine.

[0753] For example, the following sentence is generated:

[0754] To your boss

[0755] Thank you for your hard work. I'll keep you updated on the progress of the project.

[0756] The project is currently behind schedule because the necessary materials have not yet arrived. To address this issue, we are considering using alternative materials.

[0757] I would appreciate any instructions.

[0758] Thank you very much.

[0759] Output and check the text

[0760] The server sends the generated text to the terminal. The terminal displays this text to the user. The user can check the displayed text and make corrections as necessary. The system learns from these corrections and reflects them in the next generation.

[0761] In this way, the system can convert simple notes entered by the user into natural-sounding sentences, significantly reducing the burden of writing. It can also recognize the user's emotions and generate sentences based on those emotions, further improving the efficiency and accuracy of communication.

[0762] The processing flow will be explained below.

[0763] Step 1:

[0764] The user inputs a simple memo into the terminal. For example, it might be something like, "Project progress is delayed. Cause: Materials have not arrived. Solution: Consider using alternative materials." The terminal receives this memo and sends it to the server.

[0765] Step 2:

[0766] The server tokenizes the received memo. During this process, the memo is divided into words and phrases. For example, tokens such as "Project progress," "Delayed," "Cause," "Materials," "Not arrived," "Countermeasures," "Alternative materials," and "Consideration of use" are generated.

[0767] Step 3:

[0768] The server removes stop words from the tokenized data, a process that removes words with little meaning (e.g., "wa" and "no").

[0769] Step 4:

[0770] The server will instead perform stemming to convert the tokens to their root form, for example, "use consideration" to "use consideration suru."

[0771] Step 5:

[0772] The server then performs syntactic analysis on the preprocessed data. This process analyzes the structure of the sentence and identifies the role of each word and phrase. For example, "project progress" is recognized as a noun phrase, while "being late" is analyzed as an adjective phrase.

[0773] Step 6:

[0774] The server performs entity recognition based on the results of the parsing, specifically identifying and classifying specific items (e.g., "project progress" or "materials").

[0775] Step 7:

[0776] The server performs sentiment analysis to determine the tone and sentiment of the note, during which the phrase "late" is analyzed as having a negative sentiment.

[0777] Step 8:

[0778] The server uses an emotion engine to analyze the emotion of the user's input and adjusts the tone of the sentence based on that emotion. For example, if "I'm late" is recognized as a negative emotion, the tone of the entire sentence will be made formal and careful.

[0779] Step 9:

[0780] The server accumulates the user's past emotional data and uses that data to learn the user's emotional patterns, allowing the system to understand the emotions expressed by the user when entering notes in the past and generate more appropriate sentences.

[0781] Step 10:

[0782] The server recognizes the situation and purpose based on the analyzed data and information from the emotion engine, referring to the user's past notes, behavioral patterns, and current project status.

[0783] Step 11:

[0784] The server generates appropriate sentences based on information from the context recognition module and emotion engine, using correct grammar and syntax and with a tone adjusted by the emotion engine.

[0785] Step 12:

[0786] The server sends the generated text to the terminal. For example, the following text is generated:

[0787] To your boss

[0788] Thank you for your hard work. I'll keep you updated on the progress of the project.

[0789] The project is currently behind schedule because the necessary materials have not yet arrived. To address this issue, we are considering using alternative materials.

[0790] I would appreciate any instructions.

[0791] Thank you very much.

[0792] Step 13:

[0793] The terminal displays the generated text to the user, who can then check it and make corrections as necessary.

[0794] Step 14:

[0795] If the user makes any corrections, they are sent to the server, which learns the corrections and applies them to the next sentence generation process.

[0796] In this way, the user simply inputs a simple note, and the server generates an appropriate sentence, which the device then provides to the user. Furthermore, by combining it with an emotion engine, the server can generate sentences with an appropriate tone based on the user's emotions, further improving the efficiency and accuracy of communication.

[0797] Example 2

[0798] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0799] In recent years, there has been a demand for technology that can automatically generate meaningful sentences from inputs such as short notes or scribbles. However, conventional systems have difficulty in properly reflecting emotion and tone, and have limitations in generating accurate and appropriate sentences based on the received information. Furthermore, they are unable to incorporate user feedback on the generated sentences, making it difficult to generate sentences that are in line with the user's style and intent. This reduces the efficiency and accuracy of sentence creation, causing problems, especially in business communication.

[0800] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0801] In this invention, the server includes a means for receiving input information, a means for tokenizing the received information, removing unnecessary words, and performing stemming, a means for performing syntactic analysis, item recognition, and sentiment analysis based on the preprocessed data, a means for generating sentences using appropriate grammar, structure, and tone, and a means for displaying the generated sentences. This makes it possible to generate natural sentences that reflect emotions and tone even from simple input notes. Furthermore, by learning from users' past data and feedback, it is possible to generate sentences that reflect the intentions and emotions of individual users.

[0802] The "means for receiving input information" refers to a process or device for transmitting memos or text data input by a user through a terminal to a server.

[0803] "Tokenization" is a text processing technique that divides notes or text data into words and phrases.

[0804] "Removal of unnecessary words" is a process of removing words with little meaning (e.g., "wa," "ga," "no," etc.) from text data.

[0805] "Stemming" is the process of converting words into their root forms (stem), thereby unifying different forms of words.

[0806] "Syntax analysis" is the process of analyzing the grammatical structure of text and identifying the grammatical role of each word and phrase.

[0807] "Item recognition" is the technique of identifying important entities in text (e.g., people's names, organization names, dates, etc.).

[0808] "Sentiment analysis" is the process of analyzing the tone or sentiment of text data, which identifies the sentiment (positive, negative, neutral, etc.) conveyed in the input text.

[0809] A "means for generating sentences with appropriate grammar, structure, and tone" is an algorithm or model (e.g., a generative AI model) for generating natural-sounding sentences based on preprocessed and analyzed data.

[0810] The "means for displaying the generated text" refers to a process or interface for displaying the generated text on the user's terminal.

[0811] "Means of inferring the situation based on preprocessed data and referring to the user's past information and behavioral patterns" is a technology that learns the user's past input data and behavioral history to more accurately infer the situation in response to the current input.

[0812] "When a user makes corrections to a generated sentence, a means for learning those corrections and reflecting them in the generation process" is a feedback function that learns the corrections made by the user to the generated sentence and reflects them in the next sentence generation.

[0813] The present invention relates to a system that receives simple notes entered by a user, recognizes the user's emotions based on the notes, and automatically generates appropriate sentences. This system functions through the interaction between a server, a terminal, and the user.

[0814] Note entry and preprocessing

[0815] First, the user inputs a simple memo into the device. For example, the memo might read, "Project progress is delayed. Cause: Materials have not arrived. Solution: Consider using alternative materials." The device receives this memo and sends it to the server via an Internet connection.

[0816] The server preprocesses the received memos using the following steps: First, the memos are tokenized and divided into words and phrases. Then, unnecessary words are removed, and finally, stemming is performed. This generates tokens such as "project progress," "delayed," "cause," "materials," "not arrived," "countermeasures," "alternative materials," and "considering use."

[0817] Natural Language Processing (NLP)

[0818] The server then applies NLP algorithms to the preprocessed data. Specifically, this process involves syntactic analysis, item recognition, and sentiment analysis. Syntactic analysis analyzes the structure of the sentence and identifies the role of each word or phrase. Item recognition identifies specific items (entities), while sentiment analysis analyzes the tone and sentiment of the note. For example, "project progress" is recognized as an entity, and "being late" is analyzed as a negative sentiment.

[0819] Emotion Engine

[0820] The server then uses an emotion engine to analyze the user's emotions as they type. The system uses NLP models such as BERT and RoBERTa. The emotion engine accumulates the user's past emotional data and uses that data to learn the user's emotional patterns. This allows the system to understand the emotions with which the user has entered notes in the past and adjust the tone of the text based on those emotions.

[0821] Sentence generation

[0822] The server generates appropriate sentences based on information from the situation recognition module and emotion engine. This process uses generative AI models such as GPT-3 and T5. The generated sentences have correct grammar and structure, and are written with a tone adjusted by the emotion engine.

[0823] As a concrete example, the following sentence is generated:

[0824] To your boss

[0825] Thank you for your hard work. I'll keep you updated on the progress of the project.

[0826] The project is currently behind schedule because the necessary materials have not yet arrived. To address this issue, we are considering using alternative materials.

[0827] I would appreciate any instructions.

[0828] Thank you very much.

[0829] Output and check the text

[0830] Finally, the server sends the generated text to the terminal. The terminal displays the generated text to the user. The user can check the displayed text and make corrections as necessary. The corrections made by the user are fed back to the system and reflected in the next text generation.

[0831] This system allows users to simply input simple notes and convert them into natural-sounding sentences, significantly reducing the burden of writing. It also recognizes the user's emotions and generates sentences based on those emotions, improving the efficiency and accuracy of communication.

[0832] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0833] Step 1:

[0834] The user enters a simple note into the input field of the terminal. The entered note is text data that reads, "Project progress is delayed. Cause: Materials have not arrived. Solution: Consider using alternative materials." The terminal receives this note and sends it to the server via the Internet. Specifically, the terminal monitors the user's input in real time and sends the note in text format as an HTTP POST request to the server's API endpoint. The input is the user's note, and the output is the data to be sent to the server.

[0835] Step 2:

[0836] The server preprocesses the received notes. First, it tokenizes the notes and splits them into words and phrases. Next, it removes unnecessary words, and finally it performs stemming. For example, tokens such as "project progress," "delayed," "cause," "materials," "not arrived," "countermeasures," "alternative materials," and "considering use" are generated. Specifically, the server uses a natural language processing library (e.g., SpaCy) to execute scripts for tokenization, removal of unnecessary words, and stemming. The input is the initial note data, and the output is the preprocessed token data.

[0837] Step 3:

[0838] The server applies NLP algorithms to the preprocessed data. First, it performs syntactic analysis to analyze the structure of the sentence. Next, it performs item recognition to identify specific items (entities). Finally, it performs sentiment analysis to analyze the tone and sentiment of the note. For example, "project progress" is recognized as an entity, and "being late" is analyzed as having a negative sentiment. Specifically, the server performs these processes using a deep learning model (e.g., BERT). The input is the preprocessed token data, and the output is the analyzed data and sentiment labels.

[0839] Step 4:

[0840] The server uses an emotion engine to analyze the user's emotions in more detail. It uses models such as BERT and RoBERTa to identify the emotion of the user's input and adjust the tone of the sentence based on that emotion. Specifically, the server inputs previously accumulated user emotion data into a learning model to learn emotion patterns. The input is the analyzed data and emotion labels, and the output is adjusted emotion parameters.

[0841] Step 5:

[0842] The server generates appropriate sentences based on the context recognition module and emotion engine. In this process, it uses a generative AI model (e.g., GPT-3 or T5) to create sentences with correct grammar, structure, and tone. For example, the following sentences are generated:

[0843] To your boss

[0844] Thank you for your hard work. I'll keep you updated on the progress of the project.

[0845] The project is currently behind schedule because the necessary materials have not yet arrived. To address this issue, we are considering using alternative materials.

[0846] I would appreciate any instructions.

[0847] Thank you very much.

[0848] Specifically, the server provides the analyzed data and emotion parameters as input to the generative AI model to generate sentences. The input is the adjusted emotion parameters and analyzed data, and the output is the generated text sentence.

[0849] Step 6:

[0850] The server sends the generated text to the device. Specifically, the server sends the generated text to the device's API endpoint as an HTTP POST request. The input is the generated text data, and the output is the data to be sent to the device.

[0851] Step 7:

[0852] The terminal displays the generated text to the user, who can review the displayed text and make corrections as necessary. Specifically, the terminal displays the received text on a user interface and provides an editable text area. The input is the text data received from the server, and the output is the displayed text. If the user makes corrections, that feedback is sent to the system and reflected in the next text generation.

[0853] (Application example 2)

[0854] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0855] In security services, field personnel are required to quickly and accurately report handwritten notes. However, it is difficult to directly convert handwritten notes into reports, which can result in inconsistent tone and content. It is also challenging to generate appropriate reports that take into account the emotions and tone of field personnel. Furthermore, it is difficult to reflect personnel's corrections to generated reports in the next report. A system that can solve these problems is needed.

[0856] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for receiving input, scribbled notes; means for tokenizing the received notes and removing stop words and performing stemming; means for performing syntactic analysis, entity recognition, and sentiment analysis based on the preprocessed data; means for displaying the generated text to the field staff; means for recognizing the sentiment of the notes entered by the field staff and adjusting the tone of the report; means for inferring the situation based on the preprocessed data and referring to the field staff's past notes and behavioral patterns; and means for learning and reflecting the corrections made by the field staff in the generation process when the field staff makes corrections to the generated text. This makes it possible to quickly generate consistent and appropriate reports from handwritten notes, taking the field staff's sentiment and tone into consideration, and improving the quality of the reports by reflecting the staff's corrections in the next generation process.

[0857] "Means for receiving input scribbled notes" refers to devices or software that have the function of receiving simple handwritten notes entered by field personnel in digital format.

[0858] "Tokenization" is the process of dividing input text into words and phrases.

[0859] "Stop word removal" is the process of removing words with little meaning (such as "no", "wa", and "ni") from text.

[0860] "Stemming" is the process of converting words into their root forms.

[0861] "Syntax analysis" is a method of analyzing the structure of a sentence and clarifying the role of each word and phrase.

[0862] "Entity recognition" is the technique of identifying specific items or information within text (e.g., names of people, places, dates, etc.).

[0863] "Sentiment analysis" is a technique that analyzes a user's emotions and tone from the content of text and identifies positive, negative, or neutral emotions.

[0864] "Means for recognizing the situation and purpose" refers to devices or software that understand the situation and purpose from the analyzed data and then perform the next processing based on that information.

[0865] "Means for generating sentences using appropriate grammar, syntax, and tone" refers to devices or software that automatically create sentences using correct grammar and syntax and with a tone that reflects the results of sentiment analysis.

[0866] The "means for displaying the generated text to the on-site staff" refers to a device or software that displays the generated report on the on-site staff's device.

[0867] The "means for recognizing the emotion of a memo and adjusting the tone of a report" is a technology that analyzes the emotion of an input memo and appropriately adjusts the tone of a report based on that emotion.

[0868] "Means for inferring the situation by referring to past notes and behavioral patterns" refers to devices or software that refer to a database of past notes and behavioral patterns of field personnel and use that information to infer the current situation.

[0869] "Means for learning corrections and reflecting them in the generation process" refers to a technology that learns the corrections made by field personnel to generated text and reflects that knowledge when generating text from the next time onwards.

[0870] The present invention provides a system that enables on-site personnel to quickly and accurately report handwritten notes. Specific embodiments of the system will be described in detail below.

[0871] Note entry and preprocessing

[0872] The user inputs a simple handwritten note into a device such as a smartphone or robot. For example, the note might read, "Door forced open, intruder detected, evidence: fingerprint, alarm not activated." The device receives the note and sends it to the server. The server then performs the following preprocessing on the received note:

[0873] 1. Tokenization:

[0874] The notes are segmented into words and phrases, generating tokens such as "door," "forced," "open," "intruder," "discovered," "evidence," "fingerprint," "alarm," and "not activated."

[0875] 2. Stop word removal:

[0876] Remove words with little meaning. For example, stop words such as "teki" (target), "no" (of), and "ni" (to) are removed.

[0877] 3. Stemming:

[0878] Convert words to their root forms, for example, "open" is converted to "open."

[0879] Natural Language Processing (NLP)

[0880] Based on the preprocessed data, the server applies the following NLP algorithms:

[0881] 1. Parsing:

[0882] It analyzes the structure of a sentence and identifies the role of each word and phrase, for example, "door" is recognized as the subject and "open" as a verb.

[0883] 2. Entity Recognition:

[0884] Identifies a specific item. For example, "Evidence:Fingerprint" is recognized as an entity.

[0885] 3. Sentiment analysis:

[0886] Analyze the tone and sentiment of the memo. For example, "forced," "discovered," and "not working" are analyzed as negative sentiments.

[0887] Emotion Engine

[0888] The server uses an emotion engine to analyze the user's emotion when inputting information and adjusts the tone of the report accordingly. For example, if a user inputs a note containing negative emotions such as "forced," "discovered," or "not working," the tone of the entire document will be made more formal and careful. The server also accumulates the user's past emotional data and behavioral patterns and uses them to generate documents optimized for the current situation.

[0889] Text generation and display

[0890] The server generates sentences using appropriate grammar, syntax, and tone based on the output of the context recognition module and emotion engine. For example, the following sentences are generated:

[0891] "Security, this is a scene surveillance report. The door was forced open and an intruder was found. Fingerprints were left as evidence and the alarm was not activated. Immediate action is required."

[0892] The generated sentences are displayed on the terminals of the on-site personnel. The on-site personnel can check the displayed sentences and make corrections as necessary. These corrections are fed back into the system and reflected in the next sentence generation.

[0893] Examples of specific examples and prompts

[0894] As a specific example, consider the case where a field worker inputs the following memo:

[0895] Note: "Door forced open. Intruder found. Evidence: fingerprint. Alarm not activated."

[0896] An example of a generated prompt statement:

[0897] Prompt input to model:

[0898] On-site report notes: "Door forced open. Intruder found. Evidence: fingerprints. Alarm not activated."

[0899] Generate sentences based on the analysis results.

[0900] The server receives the prompt and uses a predefined algorithm and emotion engine to generate the appropriate response for the field representative, enabling fast and accurate reporting while also taking into account the representative's emotion and tone.

[0901] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0902] Step 1:

[0903] User Input

[0904] The user inputs a simple handwritten note into a device such as a smartphone or robot. For example, a note such as "Door forced open, intruder detected, evidence: fingerprint, alarm not activated" is input. The input note is received by the device.

[0905] Step 2:

[0906] Send a note

[0907] The terminal sends the received memo to the server, and when the memo arrives at the server, the next process begins.

[0908] Step 3:

[0909] Pretreatment

[0910] The server pre-processes the received notes, which includes:

[0911] Tokenization: Splits the input notes into words and phrases, generating tokens such as "door," "forced," "open," "intruder," "discovered," "evidence," "fingerprint," "alarm," and "not activated."

[0912] Stop word removal: Remove words with little meaning. For example, "teki" (target), "no" (of), and "ni" (to) are removed.

[0913] Stemming: Converting words into their root forms, for example, "open" is converted to "open."

[0914] Step 4:

[0915] Natural Language Processing (NLP)

[0916] The server applies the following NLP algorithms based on the preprocessed data:

[0917] Syntactic analysis: Analyzes the structure of a sentence and identifies the role of each word or phrase. For example, "door" is recognized as the subject and "open" as a verb.

[0918] Entity Recognition: Identifying specific items or information within text (e.g., "Evidence: Fingerprint").

[0919] Sentiment Analysis: Analyzes the user's emotions and tone from the text content to identify positive, negative, or neutral sentiment. For example, "forced," "discovered," and "not working" are analyzed as negative sentiments.

[0920] Step 5:

[0921] Emotion Engine

[0922] The server uses an emotion engine to analyze the user's emotions when they type and adjust the tone of the report accordingly. For example, it generates a more formal and careful tone for notes containing negative emotions. It also references the user's past emotional data and behavioral patterns to generate a report optimized for the current situation.

[0923] Step 6:

[0924] Sentence generation

[0925] Based on the output of the context recognition module and emotion engine, the server generates sentences using appropriate grammar, syntax, and tone. For example, the following sentences are generated:

[0926] "Security, this is a scene surveillance report. The door was forced open and an intruder was found. Fingerprints were left as evidence and the alarm was not activated. Immediate action is required."

[0927] Step 7:

[0928] Displaying text

[0929] The generated text is sent to a terminal and displayed to the on-site staff, who can then review the text and make corrections as necessary.

[0930] Step 8:

[0931] Correction Feedback

[0932] If a field worker makes corrections to the text, the server stores the corrections as learning data and reflects them in the next text generation process. This continuous feedback improves the accuracy and adaptability of text generation.

[0933] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0934] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0935] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0936] [Third embodiment]

[0937] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0938] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0939] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0940] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0941] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0942] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0943] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0944] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0945] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0946] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0947] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0948] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0949] The present invention relates to a system that receives inputted scribbled notes and automatically generates sentences that are deemed appropriate for the situation and purpose from them. Specific embodiments of the system will be described below.

[0950] Note entry and preprocessing

[0951] The user inputs a simple memo into the terminal. For example, it might be something like, "Project progress is delayed. Cause: Materials have not arrived. Solution: Consider using alternative materials." The terminal receives this memo and sends it to the server.

[0952] The server performs pre-processing on the received memos, which includes the following steps:

[0953] 1. Tokenization: Splitting notes into words and phrases.

[0954] 2. Stop word removal: remove words that have little meaning.

[0955] 3. Stemming: Converting words into their base forms.

[0956] For example, tokens such as "Project progress," "Delayed," "Cause," "Materials," "Not arrived," "Countermeasures," "Alternative materials," and "Considering use" are generated.

[0957] Natural Language Processing (NLP)

[0958] The server applies NLP algorithms based on the pre-processed data, a process that includes:

[0959] 1. Syntactic analysis: Analyzing the structure of a sentence and identifying the roles of words and phrases.

[0960] 2. Entity Recognition: Identifying specific items.

[0961] 3. Sentiment analysis: Analyze the tone and sentiment of the note.

[0962] For example, "project progress" is recognized as an entity, and "being late" is analyzed as a negative emotion.

[0963] Situational Awareness

[0964] The server uses the analyzed data to understand the current situation and objectives. This process takes into account the following factors:

[0965] Understand context: See notes users have previously entered and the current status of projects.

[0966] User profile: Based on the user's past behavioral patterns and general work methods.

[0967] For example, if the "project progress" is "delayed" and the cause is "materials not arriving," it is determined that this is a situation that requires countermeasures.

[0968] Sentence generation

[0969] The server generates appropriate sentences based on the information from the situational awareness module. The following factors are taken into account in this process:

[0970] Grammar and Syntax: Use correct grammar and appropriate syntax.

[0971] Tone of writing: Generate text in a tone appropriate for the reader (e.g., a formal tone appropriate for business correspondence).

[0972] For example, the following sentence is generated:

[0973] To your boss

[0974] Thank you for your hard work. I'll keep you updated on the progress of the project.

[0975] The project is currently behind schedule because the necessary materials have not yet arrived. To address this issue, we are considering using alternative materials.

[0976] I would appreciate any instructions.

[0977] Thank you very much.

[0978] Output and check the text

[0979] The server sends the generated text to the terminal. The terminal displays this text to the user. The user can check the displayed text and make corrections as necessary. The system learns from these corrections and reflects them in the next generation.

[0980] In this way, the system converts simple notes entered by users into natural-sounding sentences, significantly reducing the burden of writing and improving the efficiency and accuracy of communication.

[0981] The processing flow will be explained below.

[0982] Step 1:

[0983] The user inputs a simple memo into the terminal, which may include keywords or short phrases such as "Project progress is delayed. Cause: Materials have not arrived. Solution: Consider using alternative materials."

[0984] Step 2:

[0985] The terminal receives the entered memo and transmits the memo to the server.

[0986] Step 3:

[0987] The server tokenizes the received memo. During this process, the memo is divided into words and phrases. For example, tokens such as "Project progress," "Delayed," "Cause," "Materials," "Not arrived," "Countermeasures," "Alternative materials," and "Consideration of use" are generated.

[0988] Step 4:

[0989] The server removes stop words from the tokenized data, a process that removes words with little meaning (e.g., "wa" and "no").

[0990] Step 5:

[0991] The server performs stemming on the tokenized and stop-word-removed data. Stemming is the process of converting words into their root forms, for example, converting "use consideration" into "use consideration suru."

[0992] Step 6:

[0993] The server then performs syntactic analysis on the preprocessed data, a process that analyzes the structure of the sentence and identifies the role of each word and phrase.

[0994] Step 7:

[0995] The server performs entity recognition based on the results of the parsing, specifically identifying and classifying specific items (e.g., "project progress" or "materials").

[0996] Step 8:

[0997] The server performs sentiment analysis to determine the tone and sentiment of the note, during which the phrase "late" is analyzed as having a negative sentiment.

[0998] Step 9:

[0999] The server recognizes the situation and purpose based on the analyzed data, referring to the user's past notes, behavioral patterns, and current project status.

[1000] Step 10:

[1001] The server generates appropriate sentences based on the information recognized by the context recognition module, using correct grammar and syntax and in a formal business tone.

[1002] Step 11:

[1003] The server transmits the generated text to the terminal.

[1004] Step 12:

[1005] The terminal displays the generated text to the user, who can then check it and make corrections as necessary.

[1006] Step 13:

[1007] If the user makes any corrections, they are sent to the server, which learns the corrections and applies them to the next sentence generation process.

[1008] In this way, the user only needs to input a simple note, and the server will generate an appropriate sentence, which the device will then provide to the user. This completes the entire process, significantly reducing the burden of writing sentences.

[1009] Example 1

[1010] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1011] Conventionally, even if a user inputs a simple note, converting it into a proper sentence takes a lot of time and effort. Furthermore, manual sentence creation is prone to errors, which can reduce the efficiency and accuracy of communication. An effective system is needed to solve these problems.

[1012] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1013] In this invention, the server includes means for receiving input simple memos, means for dividing the received memos into words and phrases, removing words with little meaning, and converting words into base forms, means for performing syntactic analysis, entity recognition, and sentiment analysis based on the preprocessed data, means for recognizing situations and purposes based on the analyzed data, means for generating sentences using correct grammar, appropriate syntax, and a tone tailored to the reader, means for displaying the generated sentences to the user, and means for the user to check and correct the displayed sentences. This enables the user to quickly generate high-quality and appropriate sentences simply by inputting simple memos.

[1014] "Entered brief notes" are short, condensed pieces of information or items entered by a user using a terminal.

[1015] The "receiving means" is a means for receiving information sent from a terminal and inputting it into the server.

[1016] "Split into words and phrases" means splitting the inputted notes into individual words and phrases to make them easier to process.

[1017] "Removing words with little meaning" means removing words that are less important as information in the preprocessing stage.

[1018] "Converting words to base forms" refers to converting words to their base forms in preparation for further analysis.

[1019] "Syntax analysis" is the process of analyzing the structure of a sentence and identifying the roles and relationships of each word and phrase.

[1020] "Entity recognition" refers to identifying and tagging specific items or concepts.

[1021] "Sentiment analysis" is the analysis of tone or emotion (positive, negative, neutral, etc.) within text.

[1022] "Recognizing the situation and purpose" means understanding and analyzing the current situation and goals based on analyzed data.

[1023] "Correct grammar, appropriate syntax, and appropriate tone for the reader" means producing documents that are grammatically correct, well-structured, and have an appropriate tone for the reader.

[1024] "Means for generating" refers to means for automatically creating appropriate sentences based on information from preprocessing and analysis.

[1025] The "display means" is a means for showing the text generated by the server to the user.

[1026] The "means for making corrections" is a means by which the user can add or correct the generated text.

[1027] The present invention relates to a system that automatically generates appropriate sentences according to the situation and purpose based on simple notes entered by a user. A specific embodiment of this system will be described below.

[1028] Note entry and preprocessing

[1029] The user inputs a simple memo into the terminal. For example, a memo such as "Project progress is delayed. Cause: Materials have not arrived. Solution: Consider using alternative materials." The terminal receives this memo and sends it to the server. The server performs preprocessing on the received memo. This preprocessing includes the following steps:

[1030] 1. Tokenization: Divide notes into words and phrases. For example, tokens such as "Project progress," "Delayed," "Cause," "Materials," "Not arrived," "Countermeasures," "Alternative materials," and "Considering use" are generated.

[1031] 2. Stop word removal: Remove words with little meaning. For example, words such as "no", "ga", and "wa" are removed.

[1032] 3. Stemming: Converting words into their root forms, for example, converting "materials not yet arrived" into "materials not yet arrived."

[1033] The server performs these preprocessing steps using an NLP library written in Python (e.g., NLTK or spaCy).

[1034] Natural Language Processing (NLP)

[1035] The server applies NLP algorithms based on the pre-processed data. This process includes:

[1036] 1. Syntactic analysis: Analyzes the structure of a sentence and identifies the roles of words and phrases. For example, "project progress" is recognized as the subject and "delayed" as the verb.

[1037] 2. Entity Recognition: Identifying specific items, for example, "Project Progress" is recognized as an entity.

[1038] 3. Sentiment analysis: Analyze the tone and sentiment of the note. For example, "I'm late" is analyzed as a negative sentiment.

[1039] The server performs these processes using generative AI models such as BERT and GPT-3.

[1040] Situational Awareness

[1041] The server uses the analyzed data to understand the current situation and objectives. This process takes into account the following factors:

[1042] Understand context: See notes users have previously entered and the current status of projects.

[1043] User profile: Based on the user's past behavioral patterns and general work methods.

[1044] For example, if the "project progress" is "delayed" and the cause is "materials not yet arrived," it is determined that a countermeasure is required. The server accesses the database and refers to past notes and project history to understand the current situation.

[1045] Sentence generation

[1046] The server generates an appropriate sentence based on the information from the situational awareness module. The following factors are taken into account in this process:

[1047] Grammar and Syntax: Use correct grammar and appropriate syntax.

[1048] Tone of writing: Generate text in a tone appropriate for the reader, for example, using a formal tone appropriate for business correspondence.

[1049] The server uses a generative AI model (e.g., GPT-3) to generate sentences based on the analysis results and contextual awareness information. An example of a generated sentence is as follows:

[1050] To your boss

[1051] Thank you for your hard work. I'll keep you updated on the progress of the project.

[1052] The project is currently behind schedule because the necessary materials have not yet arrived. To address this issue, we are considering using alternative materials.

[1053] I would appreciate any instructions.

[1054] Thank you very much.

[1055] Output and check the text

[1056] The server sends the generated text to the terminal. The terminal displays this text to the user. The user checks the displayed text and makes corrections as necessary. The system learns from these corrections and reflects them in the next generation.

[1057] In this way, the system converts simple notes entered by users into natural-sounding sentences, significantly reducing the burden of writing and improving the efficiency and accuracy of communication.

[1058] Prompt Sentence Examples

[1059] Examples of prompts to input to a generative AI model include:

[1060] Based on the memo "Project progress is delayed. Cause: Materials have not arrived. Solution: Consider using alternative materials," generate a report to send to your boss.

[1061] Based on this prompt, the AI ​​model generates appropriate sentences.

[1062] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1063] Processing steps of this system's program

[1064] Below, the processing flow of the program of this system will be explained step by step, and the specific operations, inputs, and outputs performed at each step will be shown.

[1065] Step 1: Enter and send your note

[1066] Description: The user inputs a simple memo into the terminal. For example, "Project progress is delayed. Cause: Materials have not arrived. Solution: Consider using alternative materials." The terminal receives this memo and sends it to the server.

[1067] Input: Notes entered by the user into the device.

[1068] Output: The note received by the server.

[1069] Specific operation: The user enters a note into the input field on their smartphone or computer and presses the "Send" button, which sends the note to the server.

[1070] Step 2: Preprocessing notes

[1071] Description: The server performs the following data preprocessing on the received memo.

[1072] Tokenization: Splitting notes into words and phrases.

[1073] Stopword removal: Remove words that have little meaning.

[1074] Stemming: Converting words into their root forms.

[1075] Input: Received note.

[1076] Output: Preprocessed data (tokenized, stopword-removed, stemmed words and phrases).

[1077] Specific operation: The server uses Python NLP libraries (NLTK and spaCy) to divide the notes into tokens such as "project progress," "delayed," "cause," "materials," "not arrived," "countermeasures," "alternative materials," and "considering use," and then removes stop words and performs stemming.

[1078] Step 3: NLP analysis

[1079] Description: The server performs parsing, entity recognition, and sentiment analysis on the preprocessed data.

[1080] Syntactic analysis: Analyzing the structure of a sentence.

[1081] Entity Recognition: Identifying specific items.

[1082] Sentiment analysis: Analyze the tone and sentiment of your notes.

[1083] Input: Preprocessed data.

[1084] Output: Results of parsing, entity recognition, and sentiment analysis.

[1085] Specific operation: The server uses a generative AI model such as BERT or GPT-3 to obtain the analysis result, where "project progress" is the subject and "delayed" is a verb that expresses negative sentiment.

[1086] Step 4: Situational Awareness

[1087] Description: The server recognizes the current situation and objectives based on the analyzed data.

[1088] Understand context: See previously entered notes and project status.

[1089] User profile: Based on past behavioral patterns and general work methods.

[1090] Input: Results of parsing, entity recognition, and sentiment analysis.

[1091] Output: Current situation and objectives.

[1092] Specific operation: The server accesses the database, references past notes and history such as "project progress is behind schedule" or "the cause is the non-arrival of materials," and recognizes that action is needed.

[1093] Step 5: Sentence generation

[1094] Description: The server generates appropriate text based on the information from the situational awareness module.

[1095] Grammar and Syntax: Use correct grammar and appropriate syntax.

[1096] Tone of writing: Generate text in a tone appropriate for the reader.

[1097] Input: Current situation and objectives.

[1098] Output: The generated sentence.

[1099] Specific operation: The server uses a generative AI model (such as GPT-3) to generate sentences such as, "To my boss, I'm reporting on the progress of the project. The project is currently delayed because the materials have not yet arrived."

[1100] Step 6: Output and check the text

[1101] Description: The server sends the generated text to the terminal, which displays it to the user. The user can check the displayed text and make corrections if necessary.

[1102] Input: The generated sentence.

[1103] Output: The text reviewed and corrected by the user.

[1104] Specific operation: The generated text is displayed on the terminal, and the user can check and edit it on the terminal. The edits are recorded by the server and reflected in the next text generation.

[1105] Prompt Sentence Examples

[1106] Examples of prompts to input to a generative AI model include:

[1107] Based on the memo "Project progress is delayed. Cause: Materials have not arrived. Solution: Consider using alternative materials," generate a report to send to your boss.

[1108] Based on this prompt, the AI ​​model generates appropriate sentences.

[1109] (Application example 1)

[1110] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1111] In the food delivery industry, it has traditionally been difficult to quickly generate appropriate response messages based on simple notes and feedback from customers. As a result, customer responses have taken time and customer satisfaction has declined. The present invention aims to provide a system that automatically generates appropriate response messages based on notes from customers, thereby achieving fast and effective customer responses.

[1112] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1113] In this invention, the server includes means for receiving input scribbled notes, means for tokenizing the received notes and removing stop words and performing stemming, means for performing syntactic analysis, entity recognition, and sentiment analysis based on the preprocessed data, means for generating a response message for food delivery using appropriate grammar, syntax, and tone, and means for displaying the generated text to the user and accepting feedback. This makes it possible to automatically generate a quick and appropriate response message based on a simple note from a customer, thereby improving the efficiency and satisfaction of customer service.

[1114] "Scribbled notes" refer to short notes or comments that are simply entered by the user.

[1115] "Tokenization" refers to breaking down text or notes into the smallest meaningful units: words and phrases.

[1116] "Stop word removal" refers to the removal of common words (e.g., particles, auxiliary verbs) that have no meaning in a sentence during the preprocessing process.

[1117] "Stemming" refers to the process of converting words into their base forms or stems.

[1118] "Syntax analysis" refers to analyzing the structure of a sentence and identifying the role of each word or phrase in the sentence.

[1119] "Entity recognition" refers to the technology of identifying specific items or proper nouns within a piece of text.

[1120] "Sentiment analysis" refers to analyzing text data to determine the emotional tone (e.g., positive, negative, neutral) of its content.

[1121] "Situation and purpose awareness" refers to understanding the current situation, context, and the objectives to be achieved based on analyzed data.

[1122] "Food delivery response message" refers to text generated for customer responses related to food delivery services.

[1123] "Grammar, syntax, and tone" refers to the grammatical rules used to generate sentences, the structure of sentences, and the tone and mood of sentences.

[1124] "Means for displaying to the user and receiving feedback" refers to a function for displaying the generated text to the user and receiving corrections and comments from the user regarding the text.

[1125] This invention is a system that automatically generates appropriate response messages from simple notes entered by the user in order to improve the efficiency of customer responses in food delivery.

[1126] Hardware and software used

[1127] The hardware used to implement this system is a small smart device (e.g., a smartphone), and the software uses Python, Google Cloud Natural Language API, and OpenAI's GPT-4 API.

[1128] What the program does

[1129] The server receives a user's scribbled note, tokenizes it, removes stop words, and performs stemming. Based on the preprocessed data, it performs syntactic analysis, entity recognition, and sentiment analysis to understand the content of the note and its emotional tone.

[1130] The server then uses the analyzed data to recognize the situation and purpose and generate an appropriate response message for the food delivery, paying attention to grammar, syntax, and tone to ensure a proper response for the customer.

[1131] The generated text is displayed on the user's terminal, and the user can make corrections or provide feedback. The user's feedback and corrections are taken into account by the server and reflected in the next message generation process.

[1132] Specific examples

[1133] For example, a user enters a note on a smart device like this: "My order is over an hour late. Please do something!" This note is sent to the server, where it is tokenized, stopwords are removed, and stemmed. Then, syntactic analysis, entity recognition, and sentiment analysis are performed to analyze the content of the note.

[1134] Based on the analysis results, the system will generate an appropriate response message, such as:

[1135] plain

[1136] Dear Customer,

[1137] We sincerely apologize for the delay in delivering your order. We are currently monitoring the delivery status and will respond promptly. We would appreciate it if you could use a coupon instead for your next order.

[1138] We sincerely apologize for any inconvenience caused.

[1139] The generated message is displayed on the user's smart device, where the user can review and modify it if necessary. Modified messages help the system learn and generate even more accurate response messages in the future.

[1140] Prompt Sentence Examples

[1141] Use the following as your prompt:

[1142] plain

[1143] User Input: "My order is over an hour late. Please do something!"

[1144] Output formats:

[1145] Dear Customer,

[1146] [Sentence generation]

[1147] We sincerely apologize for any inconvenience caused.

[1148] This system will make it possible to improve the efficiency and satisfaction of customer service in food delivery services.

[1149] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1150] Step 1:

[1151] A user enters a scribbled note into a food delivery-related smart device application and presses the send button. An example entry might be, "My order is over an hour late. Please do something!" This note is then sent to the server.

[1152] Input: Notes entered by the user

[1153] Output: Notes sent to the server

[1154] Step 2:

[1155] The server tokenizes the received note, which is the process of breaking the note into words and phrases, for example, "order," "1 hour," "late," "do something," etc.

[1156] Input: The note received by the server

[1157] Output: Tokenized words and phrases

[1158] Step 3:

[1159] The server removes stop words from the tokenized data and performs stemming. Stop word removal means removing words with little meaning (e.g. particles and auxiliary verbs), and stemming means converting words to their base forms. For example, "I'm late" is converted to "I'll be late."

[1160] Input: Tokenized data

[1161] Output: Stopword-removed and stemmed data

[1162] Step 4:

[1163] The server performs syntactic analysis, entity recognition, and sentiment analysis on the preprocessed data. Syntactic analysis analyzes the structure of the sentence, while entity recognition identifies specific items (e.g., "order"). Sentiment analysis analyzes the tone of the note; for example, "We're late" is recognized as a negative sentiment.

[1164] Input: Preprocessed data

[1165] Output: Results of parsing, entity recognition, and sentiment analysis

[1166] Step 5:

[1167] The server recognizes the situation and purpose based on the analyzed data, which is to understand the situation that the customer is unhappy about the delay in their order and generate an appropriate response message.

[1168] Input: Parsed data

[1169] Output: Recognized situation and purpose

[1170] Step 6:

[1171] The server generates a corresponding food delivery message using the appropriate grammar, syntax, and tone using the GPT-4 API. For example, the following message might be generated:

[1172] plain

[1173] Dear Customer,

[1174] We sincerely apologize for the delay in delivering your order. We are currently monitoring the delivery status and will respond promptly. We would appreciate it if you could use a coupon instead for your next order.

[1175] We sincerely apologize for any inconvenience caused.

[1176] Input: Perceived situation and purpose

[1177] Output: The corresponding message generated

[1178] Step 7:

[1179] The generated message is sent to the user's terminal and displayed. The user can check the generated message and make corrections if necessary, and the feedback is sent to the server.

[1180] Input: The corresponding message generated

[1181] Output: Message and feedback displayed to the user

[1182] Step 8:

[1183] The server learns from the user's corrections and feedback and incorporates them into the next message generation process, allowing the system to generate more accurate messages.

[1184] Input: User corrections and feedback

[1185] Output: Updated message generation model

[1186] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1187] The present invention relates to a system that receives input scribbled notes, recognizes the user's emotions, and automatically generates sentences that are deemed appropriate based on the received notes. Specific embodiments of the system will be described below.

[1188] Note entry and preprocessing

[1189] The user inputs a simple memo into the terminal. For example, it might be something like, "Project progress is delayed. Cause: Materials have not arrived. Solution: Consider using alternative materials." The terminal receives this memo and sends it to the server.

[1190] The server performs pre-processing on the received memos, which includes the following steps:

[1191] 1. Tokenization: Splitting notes into words and phrases.

[1192] 2. Stop word removal: remove words that have little meaning.

[1193] 3. Stemming: Converting words into their base forms.

[1194] For example, tokens such as "Project progress," "Delayed," "Cause," "Materials," "Not arrived," "Countermeasures," "Alternative materials," and "Considering use" are generated.

[1195] Natural Language Processing (NLP)

[1196] The server applies NLP algorithms based on the pre-processed data, a process that includes:

[1197] 1. Syntactic analysis: Analyzing the structure of a sentence and identifying the roles of words and phrases.

[1198] 2. Entity Recognition: Identifying specific items.

[1199] 3. Sentiment analysis: Analyze the tone and sentiment of the note.

[1200] For example, "project progress" is recognized as an entity, and "being late" is analyzed as a negative emotion.

[1201] Emotion Engine

[1202] The system also incorporates an emotion engine to analyze the emotion of the user's input and adjust the tone of the sentence based on that emotion. For example, if a user types "I'm late," and the input indicates a negative emotion, the system will make the tone of the entire sentence more formal and careful.

[1203] The server also stores the user's past emotional data and uses that data to learn the user's emotional patterns, allowing the system to understand the emotions expressed by the user when entering notes in the past and generate more appropriate sentences.

[1204] Furthermore, the system receives user feedback on the generated sentences, recognizes emotions from the feedback, and reflects these in the next sentence generation. This continuous learning process allows the system to generate sentences that better match the user's emotions and tone.

[1205] Sentence generation

[1206] The server generates appropriate sentences based on information from the context recognition module and emotion engine, using correct grammar and syntax and with a tone adjusted by the emotion engine.

[1207] For example, the following sentence is generated:

[1208] To your boss

[1209] Thank you for your hard work. I'll keep you updated on the progress of the project.

[1210] The project is currently behind schedule because the necessary materials have not yet arrived. To address this issue, we are considering using alternative materials.

[1211] I would appreciate any instructions.

[1212] Thank you very much.

[1213] Output and check the text

[1214] The server sends the generated text to the terminal. The terminal displays this text to the user. The user can check the displayed text and make corrections as necessary. The system learns from these corrections and reflects them in the next generation.

[1215] In this way, the system can convert simple notes entered by the user into natural-sounding sentences, significantly reducing the burden of writing. It can also recognize the user's emotions and generate sentences based on those emotions, further improving the efficiency and accuracy of communication.

[1216] The processing flow will be explained below.

[1217] Step 1:

[1218] The user inputs a simple memo into the terminal. For example, it might be something like, "Project progress is delayed. Cause: Materials have not arrived. Solution: Consider using alternative materials." The terminal receives this memo and sends it to the server.

[1219] Step 2:

[1220] The server tokenizes the received memo. During this process, the memo is divided into words and phrases. For example, tokens such as "Project progress," "Delayed," "Cause," "Materials," "Not arrived," "Countermeasures," "Alternative materials," and "Consideration of use" are generated.

[1221] Step 3:

[1222] The server removes stop words from the tokenized data, a process that removes words with little meaning (e.g., "wa" and "no").

[1223] Step 4:

[1224] The server will instead perform stemming to convert the tokens to their root form, for example, "use consideration" to "use consideration suru."

[1225] Step 5:

[1226] The server then performs syntactic analysis on the preprocessed data. This process analyzes the structure of the sentence and identifies the role of each word and phrase. For example, "project progress" is recognized as a noun phrase, while "being late" is analyzed as an adjective phrase.

[1227] Step 6:

[1228] The server performs entity recognition based on the results of the parsing, specifically identifying and classifying specific items (e.g., "project progress" or "materials").

[1229] Step 7:

[1230] The server performs sentiment analysis to determine the tone and sentiment of the note, during which the phrase "late" is analyzed as having a negative sentiment.

[1231] Step 8:

[1232] The server uses an emotion engine to analyze the emotion of the user's input and adjusts the tone of the sentence based on that emotion. For example, if "I'm late" is recognized as a negative emotion, the tone of the entire sentence will be made formal and careful.

[1233] Step 9:

[1234] The server accumulates the user's past emotional data and uses that data to learn the user's emotional patterns, allowing the system to understand the emotions expressed by the user when entering notes in the past and generate more appropriate sentences.

[1235] Step 10:

[1236] The server recognizes the situation and purpose based on the analyzed data and information from the emotion engine, referring to the user's past notes, behavioral patterns, and current project status.

[1237] Step 11:

[1238] The server generates appropriate sentences based on information from the context recognition module and emotion engine, using correct grammar and syntax and with a tone adjusted by the emotion engine.

[1239] Step 12:

[1240] The server sends the generated text to the terminal. For example, the following text is generated:

[1241] To your boss

[1242] Thank you for your hard work. I'll keep you updated on the progress of the project.

[1243] The project is currently behind schedule because the necessary materials have not yet arrived. To address this issue, we are considering using alternative materials.

[1244] I would appreciate any instructions.

[1245] Thank you very much.

[1246] Step 13:

[1247] The terminal displays the generated text to the user, who can then check it and make corrections as necessary.

[1248] Step 14:

[1249] If the user makes any corrections, they are sent to the server, which learns the corrections and applies them to the next sentence generation process.

[1250] In this way, the user simply inputs a simple note, and the server generates an appropriate sentence, which the device then provides to the user. Furthermore, by combining it with an emotion engine, the server can generate sentences with an appropriate tone based on the user's emotions, further improving the efficiency and accuracy of communication.

[1251] Example 2

[1252] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1253] In recent years, there has been a demand for technology that can automatically generate meaningful sentences from inputs such as short notes or scribbles. However, conventional systems have difficulty in properly reflecting emotion and tone, and have limitations in generating accurate and appropriate sentences based on the received information. Furthermore, they are unable to incorporate user feedback on the generated sentences, making it difficult to generate sentences that are in line with the user's style and intent. This reduces the efficiency and accuracy of sentence creation, causing problems, especially in business communication.

[1254] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1255] In this invention, the server includes a means for receiving input information, a means for tokenizing the received information, removing unnecessary words, and performing stemming, a means for performing syntactic analysis, item recognition, and sentiment analysis based on the preprocessed data, a means for generating sentences using appropriate grammar, structure, and tone, and a means for displaying the generated sentences. This makes it possible to generate natural sentences that reflect emotions and tone even from simple input notes. Furthermore, by learning from users' past data and feedback, it is possible to generate sentences that reflect the intentions and emotions of individual users.

[1256] The "means for receiving input information" refers to a process or device for transmitting memos or text data input by a user through a terminal to a server.

[1257] "Tokenization" is a text processing technique that divides notes or text data into words and phrases.

[1258] "Removal of unnecessary words" is a process of removing words with little meaning (e.g., "wa," "ga," "no," etc.) from text data.

[1259] "Stemming" is the process of converting words into their root forms (stem), thereby unifying different forms of words.

[1260] "Syntax analysis" is the process of analyzing the grammatical structure of text and identifying the grammatical role of each word and phrase.

[1261] "Item recognition" is the technique of identifying important entities in text (e.g., people's names, organization names, dates, etc.).

[1262] "Sentiment analysis" is the process of analyzing the tone or sentiment of text data, which identifies the sentiment (positive, negative, neutral, etc.) conveyed in the input text.

[1263] A "means for generating sentences with appropriate grammar, structure, and tone" is an algorithm or model (e.g., a generative AI model) for generating natural-sounding sentences based on preprocessed and analyzed data.

[1264] The "means for displaying the generated text" refers to a process or interface for displaying the generated text on the user's terminal.

[1265] "Means of inferring the situation based on preprocessed data and referring to the user's past information and behavioral patterns" is a technology that learns the user's past input data and behavioral history to more accurately infer the situation in response to the current input.

[1266] "When a user makes corrections to a generated sentence, a means for learning those corrections and reflecting them in the generation process" is a feedback function that learns the corrections made by the user to the generated sentence and reflects them in the next sentence generation.

[1267] The present invention relates to a system that receives simple notes entered by a user, recognizes the user's emotions based on the notes, and automatically generates appropriate sentences. This system functions through the interaction between a server, a terminal, and the user.

[1268] Note entry and preprocessing

[1269] First, the user inputs a simple memo into the device. For example, the memo might read, "Project progress is delayed. Cause: Materials have not arrived. Solution: Consider using alternative materials." The device receives this memo and sends it to the server via an Internet connection.

[1270] The server preprocesses the received memos using the following steps: First, the memos are tokenized and divided into words and phrases. Then, unnecessary words are removed, and finally, stemming is performed. This generates tokens such as "project progress," "delayed," "cause," "materials," "not arrived," "countermeasures," "alternative materials," and "considering use."

[1271] Natural Language Processing (NLP)

[1272] The server then applies NLP algorithms to the preprocessed data. Specifically, this process involves syntactic analysis, item recognition, and sentiment analysis. Syntactic analysis analyzes the structure of the sentence and identifies the role of each word or phrase. Item recognition identifies specific items (entities), while sentiment analysis analyzes the tone and sentiment of the note. For example, "project progress" is recognized as an entity, and "being late" is analyzed as a negative sentiment.

[1273] Emotion Engine

[1274] The server then uses an emotion engine to analyze the user's emotions as they type. The system uses NLP models such as BERT and RoBERTa. The emotion engine accumulates the user's past emotional data and uses that data to learn the user's emotional patterns. This allows the system to understand the emotions with which the user has entered notes in the past and adjust the tone of the text based on those emotions.

[1275] Sentence generation

[1276] The server generates appropriate sentences based on information from the situation recognition module and emotion engine. This process uses generative AI models such as GPT-3 and T5. The generated sentences have correct grammar and structure, and are written with a tone adjusted by the emotion engine.

[1277] As a concrete example, the following sentence is generated:

[1278] To your boss

[1279] Thank you for your hard work. I'll keep you updated on the progress of the project.

[1280] The project is currently behind schedule because the necessary materials have not yet arrived. To address this issue, we are considering using alternative materials.

[1281] I would appreciate any instructions.

[1282] Thank you very much.

[1283] Output and check the text

[1284] Finally, the server sends the generated text to the terminal. The terminal displays the generated text to the user. The user can check the displayed text and make corrections as necessary. The corrections made by the user are fed back to the system and reflected in the next text generation.

[1285] This system allows users to simply input simple notes and convert them into natural-sounding sentences, significantly reducing the burden of writing. It also recognizes the user's emotions and generates sentences based on those emotions, improving the efficiency and accuracy of communication.

[1286] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1287] Step 1:

[1288] The user enters a simple note into the input field of the terminal. The entered note is text data that reads, "Project progress is delayed. Cause: Materials have not arrived. Solution: Consider using alternative materials." The terminal receives this note and sends it to the server via the Internet. Specifically, the terminal monitors the user's input in real time and sends the note in text format as an HTTP POST request to the server's API endpoint. The input is the user's note, and the output is the data to be sent to the server.

[1289] Step 2:

[1290] The server preprocesses the received notes. First, it tokenizes the notes and splits them into words and phrases. Next, it removes unnecessary words, and finally it performs stemming. For example, tokens such as "project progress," "delayed," "cause," "materials," "not arrived," "countermeasures," "alternative materials," and "considering use" are generated. Specifically, the server uses a natural language processing library (e.g., SpaCy) to execute scripts for tokenization, removal of unnecessary words, and stemming. The input is the initial note data, and the output is the preprocessed token data.

[1291] Step 3:

[1292] The server applies NLP algorithms to the preprocessed data. First, it performs syntactic analysis to analyze the structure of the sentence. Next, it performs item recognition to identify specific items (entities). Finally, it performs sentiment analysis to analyze the tone and sentiment of the note. For example, "project progress" is recognized as an entity, and "being late" is analyzed as having a negative sentiment. Specifically, the server performs these processes using a deep learning model (e.g., BERT). The input is the preprocessed token data, and the output is the analyzed data and sentiment labels.

[1293] Step 4:

[1294] The server uses an emotion engine to analyze the user's emotions in more detail. It uses models such as BERT and RoBERTa to identify the emotion of the user's input and adjust the tone of the sentence based on that emotion. Specifically, the server inputs previously accumulated user emotion data into a learning model to learn emotion patterns. The input is the analyzed data and emotion labels, and the output is adjusted emotion parameters.

[1295] Step 5:

[1296] The server generates appropriate sentences based on the context recognition module and emotion engine. In this process, it uses a generative AI model (e.g., GPT-3 or T5) to create sentences with correct grammar, structure, and tone. For example, the following sentences are generated:

[1297] To your boss

[1298] Thank you for your hard work. I'll keep you updated on the progress of the project.

[1299] The project is currently behind schedule because the necessary materials have not yet arrived. To address this issue, we are considering using alternative materials.

[1300] I would appreciate any instructions.

[1301] Thank you very much.

[1302] Specifically, the server provides the analyzed data and emotion parameters as input to the generative AI model to generate sentences. The input is the adjusted emotion parameters and analyzed data, and the output is the generated text sentence.

[1303] Step 6:

[1304] The server sends the generated text to the device. Specifically, the server sends the generated text to the device's API endpoint as an HTTP POST request. The input is the generated text data, and the output is the data to be sent to the device.

[1305] Step 7:

[1306] The terminal displays the generated text to the user, who can review the displayed text and make corrections as necessary. Specifically, the terminal displays the received text on a user interface and provides an editable text area. The input is the text data received from the server, and the output is the displayed text. If the user makes corrections, that feedback is sent to the system and reflected in the next text generation.

[1307] (Application example 2)

[1308] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1309] In security services, field personnel are required to quickly and accurately report handwritten notes. However, it is difficult to directly convert handwritten notes into reports, which can result in inconsistent tone and content. It is also challenging to generate appropriate reports that take into account the emotions and tone of field personnel. Furthermore, it is difficult to reflect personnel's corrections to generated reports in the next report. A system that can solve these problems is needed.

[1310] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for receiving input, scribbled notes; means for tokenizing the received notes and removing stop words and performing stemming; means for performing syntactic analysis, entity recognition, and sentiment analysis based on the preprocessed data; means for displaying the generated text to the field staff; means for recognizing the sentiment of the notes entered by the field staff and adjusting the tone of the report; means for inferring the situation based on the preprocessed data and referring to the field staff's past notes and behavioral patterns; and means for learning and reflecting the corrections made by the field staff in the generation process when the field staff makes corrections to the generated text. This makes it possible to quickly generate consistent and appropriate reports from handwritten notes, taking the field staff's sentiment and tone into consideration, and improving the quality of the reports by reflecting the staff's corrections in the next generation process.

[1311] "Means for receiving input scribbled notes" refers to devices or software that have the function of receiving simple handwritten notes entered by field personnel in digital format.

[1312] "Tokenization" is the process of dividing input text into words and phrases.

[1313] "Stop word removal" is the process of removing words with little meaning (such as "no", "wa", and "ni") from text.

[1314] "Stemming" is the process of converting words into their root forms.

[1315] "Syntax analysis" is a method of analyzing the structure of a sentence and clarifying the role of each word and phrase.

[1316] "Entity recognition" is the technique of identifying specific items or information within text (e.g., names of people, places, dates, etc.).

[1317] "Sentiment analysis" is a technique that analyzes a user's emotions and tone from the content of text and identifies positive, negative, or neutral emotions.

[1318] "Means for recognizing the situation and purpose" refers to devices or software that understand the situation and purpose from the analyzed data and then perform the next processing based on that information.

[1319] "Means for generating sentences using appropriate grammar, syntax, and tone" refers to devices or software that automatically create sentences using correct grammar and syntax and with a tone that reflects the results of sentiment analysis.

[1320] The "means for displaying the generated text to the on-site staff" refers to a device or software that displays the generated report on the on-site staff's device.

[1321] The "means for recognizing the emotion of a memo and adjusting the tone of a report" is a technology that analyzes the emotion of an input memo and appropriately adjusts the tone of a report based on that emotion.

[1322] "Means for inferring the situation by referring to past notes and behavioral patterns" refers to devices or software that refer to a database of past notes and behavioral patterns of field personnel and use that information to infer the current situation.

[1323] "Means for learning corrections and reflecting them in the generation process" refers to a technology that learns the corrections made by field personnel to generated text and reflects that knowledge when generating text from the next time onwards.

[1324] The present invention provides a system that enables on-site personnel to quickly and accurately report handwritten notes. Specific embodiments of the system will be described in detail below.

[1325] Note entry and preprocessing

[1326] The user inputs a simple handwritten note into a device such as a smartphone or robot. For example, the note might read, "Door forced open, intruder detected, evidence: fingerprint, alarm not activated." The device receives the note and sends it to the server. The server then performs the following preprocessing on the received note:

[1327] 1. Tokenization:

[1328] The notes are segmented into words and phrases, generating tokens such as "door," "forced," "open," "intruder," "discovered," "evidence," "fingerprint," "alarm," and "not activated."

[1329] 2. Stop word removal:

[1330] Remove words with little meaning. For example, stop words such as "teki" (target), "no" (of), and "ni" (to) are removed.

[1331] 3. Stemming:

[1332] Convert words to their root forms, for example, "open" is converted to "open."

[1333] Natural Language Processing (NLP)

[1334] Based on the preprocessed data, the server applies the following NLP algorithms:

[1335] 1. Parsing:

[1336] It analyzes the structure of a sentence and identifies the role of each word and phrase, for example, "door" is recognized as the subject and "open" as a verb.

[1337] 2. Entity Recognition:

[1338] Identifies a specific item. For example, "Evidence:Fingerprint" is recognized as an entity.

[1339] 3. Sentiment analysis:

[1340] Analyze the tone and sentiment of the memo. For example, "forced," "discovered," and "not working" are analyzed as negative sentiments.

[1341] Emotion Engine

[1342] The server uses an emotion engine to analyze the user's emotion when inputting information and adjusts the tone of the report accordingly. For example, if a user inputs a note containing negative emotions such as "forced," "discovered," or "not working," the tone of the entire document will be made more formal and careful. The server also accumulates the user's past emotional data and behavioral patterns and uses them to generate documents optimized for the current situation.

[1343] Text generation and display

[1344] The server generates sentences using appropriate grammar, syntax, and tone based on the output of the context recognition module and emotion engine. For example, the following sentences are generated:

[1345] "Security, this is a scene surveillance report. The door was forced open and an intruder was found. Fingerprints were left as evidence and the alarm was not activated. Immediate action is required."

[1346] The generated sentences are displayed on the terminals of the on-site personnel. The on-site personnel can check the displayed sentences and make corrections as necessary. These corrections are fed back into the system and reflected in the next sentence generation.

[1347] Examples of specific examples and prompts

[1348] As a specific example, consider the case where a field worker inputs the following memo:

[1349] Note: "Door forced open. Intruder found. Evidence: fingerprint. Alarm not activated."

[1350] An example of a generated prompt statement:

[1351] Prompt input to model:

[1352] On-site report notes: "Door forced open. Intruder found. Evidence: fingerprints. Alarm not activated."

[1353] Generate sentences based on the analysis results.

[1354] The server receives the prompt and uses a predefined algorithm and emotion engine to generate the appropriate response for the field representative, enabling fast and accurate reporting while also taking into account the representative's emotion and tone.

[1355] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1356] Step 1:

[1357] User Input

[1358] The user inputs a simple handwritten note into a device such as a smartphone or robot. For example, a note such as "Door forced open, intruder detected, evidence: fingerprint, alarm not activated" is input. The input note is received by the device.

[1359] Step 2:

[1360] Send a note

[1361] The terminal sends the received memo to the server, and when the memo arrives at the server, the next process begins.

[1362] Step 3:

[1363] Pretreatment

[1364] The server pre-processes the received notes, which includes:

[1365] Tokenization: Splits the input notes into words and phrases, generating tokens such as "door," "forced," "open," "intruder," "discovered," "evidence," "fingerprint," "alarm," and "not activated."

[1366] Stop word removal: Remove words with little meaning. For example, "teki" (target), "no" (of), and "ni" (to) are removed.

[1367] Stemming: Converting words into their root forms, for example, "open" is converted to "open."

[1368] Step 4:

[1369] Natural Language Processing (NLP)

[1370] The server applies the following NLP algorithms based on the preprocessed data:

[1371] Syntactic analysis: Analyzes the structure of a sentence and identifies the role of each word or phrase. For example, "door" is recognized as the subject and "open" as a verb.

[1372] Entity Recognition: Identifying specific items or information within text (e.g., "Evidence: Fingerprint").

[1373] Sentiment Analysis: Analyzes the user's emotions and tone from the text content to identify positive, negative, or neutral sentiment. For example, "forced," "discovered," and "not working" are analyzed as negative sentiments.

[1374] Step 5:

[1375] Emotion Engine

[1376] The server uses an emotion engine to analyze the user's emotions when they type and adjust the tone of the report accordingly. For example, it generates a more formal and careful tone for notes containing negative emotions. It also references the user's past emotional data and behavioral patterns to generate a report optimized for the current situation.

[1377] Step 6:

[1378] Sentence generation

[1379] Based on the output of the context recognition module and emotion engine, the server generates sentences using appropriate grammar, syntax, and tone. For example, the following sentences are generated:

[1380] "Security, this is a scene surveillance report. The door was forced open and an intruder was found. Fingerprints were left as evidence and the alarm was not activated. Immediate action is required."

[1381] Step 7:

[1382] Displaying text

[1383] The generated text is sent to a terminal and displayed to the on-site staff, who can then review the text and make corrections as necessary.

[1384] Step 8:

[1385] Correction Feedback

[1386] If a field worker makes corrections to the text, the server stores the corrections as learning data and reflects them in the next text generation process. This continuous feedback improves the accuracy and adaptability of text generation.

[1387] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1388] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1389] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1390] [Fourth embodiment]

[1391] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1392] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1393] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1394] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1395] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1396] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1397] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1398] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1399] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1400] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1401] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1402] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1403] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1404] The present invention relates to a system that receives inputted scribbled notes and automatically generates sentences that are deemed appropriate for the situation and purpose from them. Specific embodiments of the system will be described below.

[1405] Note entry and preprocessing

[1406] The user inputs a simple memo into the terminal. For example, it might be something like, "Project progress is delayed. Cause: Materials have not arrived. Solution: Consider using alternative materials." The terminal receives this memo and sends it to the server.

[1407] The server performs pre-processing on the received memos, which includes the following steps:

[1408] 1. Tokenization: Splitting notes into words and phrases.

[1409] 2. Stop word removal: remove words that have little meaning.

[1410] 3. Stemming: Converting words into their base forms.

[1411] For example, tokens such as "Project progress," "Delayed," "Cause," "Materials," "Not arrived," "Countermeasures," "Alternative materials," and "Considering use" are generated.

[1412] Natural Language Processing (NLP)

[1413] The server applies NLP algorithms based on the pre-processed data, a process that includes:

[1414] 1. Syntactic analysis: Analyzing the structure of a sentence and identifying the roles of words and phrases.

[1415] 2. Entity Recognition: Identifying specific items.

[1416] 3. Sentiment analysis: Analyze the tone and sentiment of the note.

[1417] For example, "project progress" is recognized as an entity, and "being late" is analyzed as a negative emotion.

[1418] Situational Awareness

[1419] The server uses the analyzed data to understand the current situation and objectives. This process takes into account the following factors:

[1420] Understand context: See notes users have previously entered and the current status of projects.

[1421] User profile: Based on the user's past behavioral patterns and general work methods.

[1422] For example, if the "project progress" is "delayed" and the cause is "materials not arriving," it is determined that this is a situation that requires countermeasures.

[1423] Sentence generation

[1424] The server generates appropriate sentences based on the information from the situational awareness module. The following factors are taken into account in this process:

[1425] Grammar and Syntax: Use correct grammar and appropriate syntax.

[1426] Tone of writing: Generate text in a tone appropriate for the reader (e.g., a formal tone appropriate for business correspondence).

[1427] For example, the following sentence is generated:

[1428] To your boss

[1429] Thank you for your hard work. I'll keep you updated on the progress of the project.

[1430] The project is currently behind schedule because the necessary materials have not yet arrived. To address this issue, we are considering using alternative materials.

[1431] I would appreciate any instructions.

[1432] Thank you very much.

[1433] Output and check the text

[1434] The server sends the generated text to the terminal. The terminal displays this text to the user. The user can check the displayed text and make corrections as necessary. The system learns from these corrections and reflects them in the next generation.

[1435] In this way, the system converts simple notes entered by users into natural-sounding sentences, significantly reducing the burden of writing and improving the efficiency and accuracy of communication.

[1436] The processing flow will be explained below.

[1437] Step 1:

[1438] The user inputs a simple memo into the terminal, which may include keywords or short phrases such as "Project progress is delayed. Cause: Materials have not arrived. Solution: Consider using alternative materials."

[1439] Step 2:

[1440] The terminal receives the entered memo and transmits the memo to the server.

[1441] Step 3:

[1442] The server tokenizes the received memo. During this process, the memo is divided into words and phrases. For example, tokens such as "Project progress," "Delayed," "Cause," "Materials," "Not arrived," "Countermeasures," "Alternative materials," and "Consideration of use" are generated.

[1443] Step 4:

[1444] The server removes stop words from the tokenized data, a process that removes words with little meaning (e.g., "wa" and "no").

[1445] Step 5:

[1446] The server performs stemming on the tokenized and stop-word-removed data. Stemming is the process of converting words into their root forms, for example, converting "use consideration" into "use consideration suru."

[1447] Step 6:

[1448] The server then performs syntactic analysis on the preprocessed data, a process that analyzes the structure of the sentence and identifies the role of each word and phrase.

[1449] Step 7:

[1450] The server performs entity recognition based on the results of the parsing, specifically identifying and classifying specific items (e.g., "project progress" or "materials").

[1451] Step 8:

[1452] The server performs sentiment analysis to determine the tone and sentiment of the note, during which the phrase "late" is analyzed as having a negative sentiment.

[1453] Step 9:

[1454] The server recognizes the situation and purpose based on the analyzed data, referring to the user's past notes, behavioral patterns, and current project status.

[1455] Step 10:

[1456] The server generates appropriate sentences based on the information recognized by the context recognition module, using correct grammar and syntax and in a formal business tone.

[1457] Step 11:

[1458] The server transmits the generated text to the terminal.

[1459] Step 12:

[1460] The terminal displays the generated text to the user, who can then check it and make corrections as necessary.

[1461] Step 13:

[1462] If the user makes any corrections, they are sent to the server, which learns the corrections and applies them to the next sentence generation process.

[1463] In this way, the user only needs to input a simple note, and the server will generate an appropriate sentence, which the device will then provide to the user. This completes the entire process, significantly reducing the burden of writing sentences.

[1464] Example 1

[1465] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1466] Conventionally, even if a user inputs a simple note, converting it into a proper sentence takes a lot of time and effort. Furthermore, manual sentence creation is prone to errors, which can reduce the efficiency and accuracy of communication. An effective system is needed to solve these problems.

[1467] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1468] In this invention, the server includes means for receiving input simple memos, means for dividing the received memos into words and phrases, removing words with little meaning, and converting words into base forms, means for performing syntactic analysis, entity recognition, and sentiment analysis based on the preprocessed data, means for recognizing situations and purposes based on the analyzed data, means for generating sentences using correct grammar, appropriate syntax, and a tone tailored to the reader, means for displaying the generated sentences to the user, and means for the user to check and correct the displayed sentences. This enables the user to quickly generate high-quality and appropriate sentences simply by inputting simple memos.

[1469] "Entered brief notes" are short, condensed pieces of information or items entered by a user using a terminal.

[1470] The "receiving means" is a means for receiving information sent from a terminal and inputting it into the server.

[1471] "Split into words and phrases" means splitting the inputted notes into individual words and phrases to make them easier to process.

[1472] "Removing words with little meaning" means removing words that are less important as information in the preprocessing stage.

[1473] "Converting words to base forms" refers to converting words to their base forms in preparation for further analysis.

[1474] "Syntax analysis" is the process of analyzing the structure of a sentence and identifying the roles and relationships of each word and phrase.

[1475] "Entity recognition" refers to identifying and tagging specific items or concepts.

[1476] "Sentiment analysis" is the analysis of tone or emotion (positive, negative, neutral, etc.) within text.

[1477] "Recognizing the situation and purpose" means understanding and analyzing the current situation and goals based on analyzed data.

[1478] "Correct grammar, appropriate syntax, and appropriate tone for the reader" means producing documents that are grammatically correct, well-structured, and have an appropriate tone for the reader.

[1479] "Means for generating" refers to means for automatically creating appropriate sentences based on information from preprocessing and analysis.

[1480] The "display means" is a means for showing the text generated by the server to the user.

[1481] The "means for making corrections" is a means by which the user can add or correct the generated text.

[1482] The present invention relates to a system that automatically generates appropriate sentences according to the situation and purpose based on simple notes entered by a user. A specific embodiment of this system will be described below.

[1483] Note entry and preprocessing

[1484] The user inputs a simple memo into the terminal. For example, a memo such as "Project progress is delayed. Cause: Materials have not arrived. Solution: Consider using alternative materials." The terminal receives this memo and sends it to the server. The server performs preprocessing on the received memo. This preprocessing includes the following steps:

[1485] 1. Tokenization: Divide notes into words and phrases. For example, tokens such as "Project progress," "Delayed," "Cause," "Materials," "Not arrived," "Countermeasures," "Alternative materials," and "Considering use" are generated.

[1486] 2. Stop word removal: Remove words with little meaning. For example, words such as "no", "ga", and "wa" are removed.

[1487] 3. Stemming: Converting words into their root forms, for example, converting "materials not yet arrived" into "materials not yet arrived."

[1488] The server performs these preprocessing steps using an NLP library written in Python (e.g., NLTK or spaCy).

[1489] Natural Language Processing (NLP)

[1490] The server applies NLP algorithms based on the pre-processed data. This process includes:

[1491] 1. Syntactic analysis: Analyzes the structure of a sentence and identifies the roles of words and phrases. For example, "project progress" is recognized as the subject and "delayed" as the verb.

[1492] 2. Entity Recognition: Identifying specific items, for example, "Project Progress" is recognized as an entity.

[1493] 3. Sentiment analysis: Analyze the tone and sentiment of the note. For example, "I'm late" is analyzed as a negative sentiment.

[1494] The server performs these processes using generative AI models such as BERT and GPT-3.

[1495] Situational Awareness

[1496] The server uses the analyzed data to understand the current situation and objectives. This process takes into account the following factors:

[1497] Understand context: See notes users have previously entered and the current status of projects.

[1498] User profile: Based on the user's past behavioral patterns and general work methods.

[1499] For example, if the "project progress" is "delayed" and the cause is "materials not yet arrived," it is determined that a countermeasure is required. The server accesses the database and refers to past notes and project history to understand the current situation.

[1500] Sentence generation

[1501] The server generates an appropriate sentence based on the information from the situational awareness module. The following factors are taken into account in this process:

[1502] Grammar and Syntax: Use correct grammar and appropriate syntax.

[1503] Tone of writing: Generate text in a tone appropriate for the reader, for example, using a formal tone appropriate for business correspondence.

[1504] The server uses a generative AI model (e.g., GPT-3) to generate sentences based on the analysis results and contextual awareness information. An example of a generated sentence is as follows:

[1505] To your boss

[1506] Thank you for your hard work. I'll keep you updated on the progress of the project.

[1507] The project is currently behind schedule because the necessary materials have not yet arrived. To address this issue, we are considering using alternative materials.

[1508] I would appreciate any instructions.

[1509] Thank you very much.

[1510] Output and check the text

[1511] The server sends the generated text to the terminal. The terminal displays this text to the user. The user checks the displayed text and makes corrections as necessary. The system learns from these corrections and reflects them in the next generation.

[1512] In this way, the system converts simple notes entered by users into natural-sounding sentences, significantly reducing the burden of writing and improving the efficiency and accuracy of communication.

[1513] Prompt Sentence Examples

[1514] Examples of prompts to input to a generative AI model include:

[1515] Based on the memo "Project progress is delayed. Cause: Materials have not arrived. Solution: Consider using alternative materials," generate a report to send to your boss.

[1516] Based on this prompt, the AI ​​model generates appropriate sentences.

[1517] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1518] Processing steps of this system's program

[1519] Below, the processing flow of the program of this system will be explained step by step, and the specific operations, inputs, and outputs performed at each step will be shown.

[1520] Step 1: Enter and send your note

[1521] Description: The user inputs a simple memo into the terminal. For example, "Project progress is delayed. Cause: Materials have not arrived. Solution: Consider using alternative materials." The terminal receives this memo and sends it to the server.

[1522] Input: Notes entered by the user into the device.

[1523] Output: The note received by the server.

[1524] Specific operation: The user enters a note into the input field on their smartphone or computer and presses the "Send" button, which sends the note to the server.

[1525] Step 2: Preprocessing notes

[1526] Description: The server performs the following data preprocessing on the received memo.

[1527] Tokenization: Splitting notes into words and phrases.

[1528] Stopword removal: Remove words that have little meaning.

[1529] Stemming: Converting words into their root forms.

[1530] Input: Received note.

[1531] Output: Preprocessed data (tokenized, stopword-removed, stemmed words and phrases).

[1532] Specific operation: The server uses Python NLP libraries (NLTK and spaCy) to divide the notes into tokens such as "project progress," "delayed," "cause," "materials," "not arrived," "countermeasures," "alternative materials," and "considering use," and then removes stop words and performs stemming.

[1533] Step 3: NLP analysis

[1534] Description: The server performs parsing, entity recognition, and sentiment analysis on the preprocessed data.

[1535] Syntactic analysis: Analyzing the structure of a sentence.

[1536] Entity Recognition: Identifying specific items.

[1537] Sentiment analysis: Analyze the tone and sentiment of your notes.

[1538] Input: Preprocessed data.

[1539] Output: Results of parsing, entity recognition, and sentiment analysis.

[1540] Specific operation: The server uses a generative AI model such as BERT or GPT-3 to obtain the analysis result, where "project progress" is the subject and "delayed" is a verb that expresses negative sentiment.

[1541] Step 4: Situational Awareness

[1542] Description: The server recognizes the current situation and objectives based on the analyzed data.

[1543] Understand context: See previously entered notes and project status.

[1544] User profile: Based on past behavioral patterns and general work methods.

[1545] Input: Results of parsing, entity recognition, and sentiment analysis.

[1546] Output: Current situation and objectives.

[1547] Specific operation: The server accesses the database, references past notes and history such as "project progress is behind schedule" or "the cause is the non-arrival of materials," and recognizes that action is needed.

[1548] Step 5: Sentence generation

[1549] Description: The server generates appropriate text based on the information from the situational awareness module.

[1550] Grammar and Syntax: Use correct grammar and appropriate syntax.

[1551] Tone of writing: Generate text in a tone appropriate for the reader.

[1552] Input: Current situation and objectives.

[1553] Output: The generated sentence.

[1554] Specific operation: The server uses a generative AI model (such as GPT-3) to generate sentences such as, "To my boss, I'm reporting on the progress of the project. The project is currently delayed because the materials have not yet arrived."

[1555] Step 6: Output and check the text

[1556] Description: The server sends the generated text to the terminal, which displays it to the user. The user can check the displayed text and make corrections if necessary.

[1557] Input: The generated sentence.

[1558] Output: The text reviewed and corrected by the user.

[1559] Specific operation: The generated text is displayed on the terminal, and the user can check and edit it on the terminal. The edits are recorded by the server and reflected in the next text generation.

[1560] Prompt Sentence Examples

[1561] Examples of prompts to input to a generative AI model include:

[1562] Based on the memo "Project progress is delayed. Cause: Materials have not arrived. Solution: Consider using alternative materials," generate a report to send to your boss.

[1563] Based on this prompt, the AI ​​model generates appropriate sentences.

[1564] (Application example 1)

[1565] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1566] In the food delivery industry, it has traditionally been difficult to quickly generate appropriate response messages based on simple notes and feedback from customers. As a result, customer responses have taken time and customer satisfaction has declined. The present invention aims to provide a system that automatically generates appropriate response messages based on notes from customers, thereby achieving fast and effective customer responses.

[1567] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1568] In this invention, the server includes means for receiving input scribbled notes, means for tokenizing the received notes and removing stop words and performing stemming, means for performing syntactic analysis, entity recognition, and sentiment analysis based on the preprocessed data, means for generating a response message for food delivery using appropriate grammar, syntax, and tone, and means for displaying the generated text to the user and accepting feedback. This makes it possible to automatically generate a quick and appropriate response message based on a simple note from a customer, thereby improving the efficiency and satisfaction of customer service.

[1569] "Scribbled notes" refer to short notes or comments that are simply entered by the user.

[1570] "Tokenization" refers to breaking down text or notes into the smallest meaningful units: words and phrases.

[1571] "Stop word removal" refers to the removal of common words (e.g., particles, auxiliary verbs) that have no meaning in a sentence during the preprocessing process.

[1572] "Stemming" refers to the process of converting words into their base forms or stems.

[1573] "Syntax analysis" refers to analyzing the structure of a sentence and identifying the role of each word or phrase in the sentence.

[1574] "Entity recognition" refers to the technology of identifying specific items or proper nouns within a piece of text.

[1575] "Sentiment analysis" refers to analyzing text data to determine the emotional tone (e.g., positive, negative, neutral) of its content.

[1576] "Situation and purpose awareness" refers to understanding the current situation, context, and the objectives to be achieved based on analyzed data.

[1577] "Food delivery response message" refers to text generated for customer responses related to food delivery services.

[1578] "Grammar, syntax, and tone" refers to the grammatical rules used to generate sentences, the structure of sentences, and the tone and mood of sentences.

[1579] "Means for displaying to the user and receiving feedback" refers to a function for displaying the generated text to the user and receiving corrections and comments from the user regarding the text.

[1580] This invention is a system that automatically generates appropriate response messages from simple notes entered by the user in order to improve the efficiency of customer responses in food delivery.

[1581] Hardware and software used

[1582] The hardware used to implement this system is a small smart device (e.g., a smartphone), and the software uses Python, Google Cloud Natural Language API, and OpenAI's GPT-4 API.

[1583] What the program does

[1584] The server receives a user's scribbled note, tokenizes it, removes stop words, and performs stemming. Based on the preprocessed data, it performs syntactic analysis, entity recognition, and sentiment analysis to understand the content of the note and its emotional tone.

[1585] The server then uses the analyzed data to recognize the situation and purpose and generate an appropriate response message for the food delivery, paying attention to grammar, syntax, and tone to ensure a proper response for the customer.

[1586] The generated text is displayed on the user's terminal, and the user can make corrections or provide feedback. The user's feedback and corrections are taken into account by the server and reflected in the next message generation process.

[1587] Specific examples

[1588] For example, a user enters a note on a smart device like this: "My order is over an hour late. Please do something!" This note is sent to the server, where it is tokenized, stopwords are removed, and stemmed. Then, syntactic analysis, entity recognition, and sentiment analysis are performed to analyze the content of the note.

[1589] Based on the analysis results, the system will generate an appropriate response message, such as:

[1590] plain

[1591] Dear Customer,

[1592] We sincerely apologize for the delay in delivering your order. We are currently monitoring the delivery status and will respond promptly. We would appreciate it if you could use a coupon instead for your next order.

[1593] We sincerely apologize for any inconvenience caused.

[1594] The generated message is displayed on the user's smart device, where the user can review and modify it if necessary. Modified messages help the system learn and generate even more accurate response messages in the future.

[1595] Prompt Sentence Examples

[1596] Use the following as your prompt:

[1597] plain

[1598] User Input: "My order is over an hour late. Please do something!"

[1599] Output formats:

[1600] Dear Customer,

[1601] [Sentence generation]

[1602] We sincerely apologize for any inconvenience caused.

[1603] This system will make it possible to improve the efficiency and satisfaction of customer service in food delivery services.

[1604] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1605] Step 1:

[1606] A user enters a scribbled note into a food delivery-related smart device application and presses the send button. An example entry might be, "My order is over an hour late. Please do something!" This note is then sent to the server.

[1607] Input: Notes entered by the user

[1608] Output: Notes sent to the server

[1609] Step 2:

[1610] The server tokenizes the received note, which is the process of breaking the note into words and phrases, for example, "order," "1 hour," "late," "do something," etc.

[1611] Input: The note received by the server

[1612] Output: Tokenized words and phrases

[1613] Step 3:

[1614] The server removes stop words from the tokenized data and performs stemming. Stop word removal means removing words with little meaning (e.g. particles and auxiliary verbs), and stemming means converting words to their base forms. For example, "I'm late" is converted to "I'll be late."

[1615] Input: Tokenized data

[1616] Output: Stopword-removed and stemmed data

[1617] Step 4:

[1618] The server performs syntactic analysis, entity recognition, and sentiment analysis on the preprocessed data. Syntactic analysis analyzes the structure of the sentence, while entity recognition identifies specific items (e.g., "order"). Sentiment analysis analyzes the tone of the note; for example, "We're late" is recognized as a negative sentiment.

[1619] Input: Preprocessed data

[1620] Output: Results of parsing, entity recognition, and sentiment analysis

[1621] Step 5:

[1622] The server recognizes the situation and purpose based on the analyzed data, which is to understand the situation that the customer is unhappy about the delay in their order and generate an appropriate response message.

[1623] Input: Parsed data

[1624] Output: Recognized situation and purpose

[1625] Step 6:

[1626] The server generates a corresponding food delivery message using the appropriate grammar, syntax, and tone using the GPT-4 API. For example, the following message might be generated:

[1627] plain

[1628] Dear Customer,

[1629] We sincerely apologize for the delay in delivering your order. We are currently monitoring the delivery status and will respond promptly. We would appreciate it if you could use a coupon instead for your next order.

[1630] We sincerely apologize for any inconvenience caused.

[1631] Input: Perceived situation and purpose

[1632] Output: The corresponding message generated

[1633] Step 7:

[1634] The generated message is sent to the user's terminal and displayed. The user can check the generated message and make corrections if necessary, and the feedback is sent to the server.

[1635] Input: The corresponding message generated

[1636] Output: Message and feedback displayed to the user

[1637] Step 8:

[1638] The server learns from the user's corrections and feedback and incorporates them into the next message generation process, allowing the system to generate more accurate messages.

[1639] Input: User corrections and feedback

[1640] Output: Updated message generation model

[1641] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1642] The present invention relates to a system that receives input scribbled notes, recognizes the user's emotions, and automatically generates sentences that are deemed appropriate based on the received notes. Specific embodiments of the system will be described below.

[1643] Note entry and preprocessing

[1644] The user inputs a simple memo into the terminal. For example, it might be something like, "Project progress is delayed. Cause: Materials have not arrived. Solution: Consider using alternative materials." The terminal receives this memo and sends it to the server.

[1645] The server performs pre-processing on the received memos, which includes the following steps:

[1646] 1. Tokenization: Splitting notes into words and phrases.

[1647] 2. Stop word removal: remove words that have little meaning.

[1648] 3. Stemming: Converting words into their base forms.

[1649] For example, tokens such as "Project progress," "Delayed," "Cause," "Materials," "Not arrived," "Countermeasures," "Alternative materials," and "Considering use" are generated.

[1650] Natural Language Processing (NLP)

[1651] The server applies NLP algorithms based on the pre-processed data, a process that includes:

[1652] 1. Syntactic analysis: Analyzing the structure of a sentence and identifying the roles of words and phrases.

[1653] 2. Entity Recognition: Identifying specific items.

[1654] 3. Sentiment analysis: Analyze the tone and sentiment of the note.

[1655] For example, "project progress" is recognized as an entity, and "being late" is analyzed as a negative emotion.

[1656] Emotion Engine

[1657] The system also incorporates an emotion engine to analyze the emotion of the user's input and adjust the tone of the sentence based on that emotion. For example, if a user types "I'm late," and the input indicates a negative emotion, the system will make the tone of the entire sentence more formal and careful.

[1658] The server also stores the user's past emotional data and uses that data to learn the user's emotional patterns, allowing the system to understand the emotions expressed by the user when entering notes in the past and generate more appropriate sentences.

[1659] Furthermore, the system receives user feedback on the generated sentences, recognizes emotions from the feedback, and reflects these in the next sentence generation. This continuous learning process allows the system to generate sentences that better match the user's emotions and tone.

[1660] Sentence generation

[1661] The server generates appropriate sentences based on information from the context recognition module and emotion engine, using correct grammar and syntax and with a tone adjusted by the emotion engine.

[1662] For example, the following sentence is generated:

[1663] To your boss

[1664] Thank you for your hard work. I'll keep you updated on the progress of the project.

[1665] The project is currently behind schedule because the necessary materials have not yet arrived. To address this issue, we are considering using alternative materials.

[1666] I would appreciate any instructions.

[1667] Thank you very much.

[1668] Output and check the text

[1669] The server sends the generated text to the terminal. The terminal displays this text to the user. The user can check the displayed text and make corrections as necessary. The system learns from these corrections and reflects them in the next generation.

[1670] In this way, the system can convert simple notes entered by the user into natural-sounding sentences, significantly reducing the burden of writing. It can also recognize the user's emotions and generate sentences based on those emotions, further improving the efficiency and accuracy of communication.

[1671] The processing flow will be explained below.

[1672] Step 1:

[1673] The user inputs a simple memo into the terminal. For example, it might be something like, "Project progress is delayed. Cause: Materials have not arrived. Solution: Consider using alternative materials." The terminal receives this memo and sends it to the server.

[1674] Step 2:

[1675] The server tokenizes the received memo. During this process, the memo is divided into words and phrases. For example, tokens such as "Project progress," "Delayed," "Cause," "Materials," "Not arrived," "Countermeasures," "Alternative materials," and "Consideration of use" are generated.

[1676] Step 3:

[1677] The server removes stop words from the tokenized data, a process that removes words with little meaning (e.g., "wa" and "no").

[1678] Step 4:

[1679] The server will instead perform stemming to convert the tokens to their root form, for example, "use consideration" to "use consideration suru."

[1680] Step 5:

[1681] The server then performs syntactic analysis on the preprocessed data. This process analyzes the structure of the sentence and identifies the role of each word and phrase. For example, "project progress" is recognized as a noun phrase, while "being late" is analyzed as an adjective phrase.

[1682] Step 6:

[1683] The server performs entity recognition based on the results of the parsing, specifically identifying and classifying specific items (e.g., "project progress" or "materials").

[1684] Step 7:

[1685] The server performs sentiment analysis to determine the tone and sentiment of the note, during which the phrase "late" is analyzed as having a negative sentiment.

[1686] Step 8:

[1687] The server uses an emotion engine to analyze the emotion of the user's input and adjusts the tone of the sentence based on that emotion. For example, if "I'm late" is recognized as a negative emotion, the tone of the entire sentence will be made formal and careful.

[1688] Step 9:

[1689] The server accumulates the user's past emotional data and uses that data to learn the user's emotional patterns, allowing the system to understand the emotions expressed by the user when entering notes in the past and generate more appropriate sentences.

[1690] Step 10:

[1691] The server recognizes the situation and purpose based on the analyzed data and information from the emotion engine, referring to the user's past notes, behavioral patterns, and current project status.

[1692] Step 11:

[1693] The server generates appropriate sentences based on information from the context recognition module and emotion engine, using correct grammar and syntax and with a tone adjusted by the emotion engine.

[1694] Step 12:

[1695] The server sends the generated text to the terminal. For example, the following text is generated:

[1696] To your boss

[1697] Thank you for your hard work. I'll keep you updated on the progress of the project.

[1698] The project is currently behind schedule because the necessary materials have not yet arrived. To address this issue, we are considering using alternative materials.

[1699] I would appreciate any instructions.

[1700] Thank you very much.

[1701] Step 13:

[1702] The terminal displays the generated text to the user, who can then check it and make corrections as necessary.

[1703] Step 14:

[1704] If the user makes any corrections, they are sent to the server, which learns the corrections and applies them to the next sentence generation process.

[1705] In this way, the user simply inputs a simple note, and the server generates an appropriate sentence, which the device then provides to the user. Furthermore, by combining it with an emotion engine, the server can generate sentences with an appropriate tone based on the user's emotions, further improving the efficiency and accuracy of communication.

[1706] Example 2

[1707] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1708] In recent years, there has been a demand for technology that can automatically generate meaningful sentences from inputs such as short notes or scribbles. However, conventional systems have difficulty in properly reflecting emotion and tone, and have limitations in generating accurate and appropriate sentences based on the received information. Furthermore, they are unable to incorporate user feedback on the generated sentences, making it difficult to generate sentences that are in line with the user's style and intent. This reduces the efficiency and accuracy of sentence creation, causing problems, especially in business communication.

[1709] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1710] In this invention, the server includes a means for receiving input information, a means for tokenizing the received information, removing unnecessary words, and performing stemming, a means for performing syntactic analysis, item recognition, and sentiment analysis based on the preprocessed data, a means for generating sentences using appropriate grammar, structure, and tone, and a means for displaying the generated sentences. This makes it possible to generate natural sentences that reflect emotions and tone even from simple input notes. Furthermore, by learning from users' past data and feedback, it is possible to generate sentences that reflect the intentions and emotions of individual users.

[1711] The "means for receiving input information" refers to a process or device for transmitting memos or text data input by a user through a terminal to a server.

[1712] "Tokenization" is a text processing technique that divides notes or text data into words and phrases.

[1713] "Removal of unnecessary words" is a process of removing words with little meaning (e.g., "wa," "ga," "no," etc.) from text data.

[1714] "Stemming" is the process of converting words into their root forms (stem), thereby unifying different forms of words.

[1715] "Syntax analysis" is the process of analyzing the grammatical structure of text and identifying the grammatical role of each word and phrase.

[1716] "Item recognition" is the technique of identifying important entities in text (e.g., people's names, organization names, dates, etc.).

[1717] "Sentiment analysis" is the process of analyzing the tone or sentiment of text data, which identifies the sentiment (positive, negative, neutral, etc.) conveyed in the input text.

[1718] A "means for generating sentences with appropriate grammar, structure, and tone" is an algorithm or model (e.g., a generative AI model) for generating natural-sounding sentences based on preprocessed and analyzed data.

[1719] The "means for displaying the generated text" refers to a process or interface for displaying the generated text on the user's terminal.

[1720] "Means of inferring the situation based on preprocessed data and referring to the user's past information and behavioral patterns" is a technology that learns the user's past input data and behavioral history to more accurately infer the situation in response to the current input.

[1721] "When a user makes corrections to a generated sentence, a means for learning those corrections and reflecting them in the generation process" is a feedback function that learns the corrections made by the user to the generated sentence and reflects them in the next sentence generation.

[1722] The present invention relates to a system that receives simple notes entered by a user, recognizes the user's emotions based on the notes, and automatically generates appropriate sentences. This system functions through the interaction between a server, a terminal, and the user.

[1723] Note entry and preprocessing

[1724] First, the user inputs a simple memo into the device. For example, the memo might read, "Project progress is delayed. Cause: Materials have not arrived. Solution: Consider using alternative materials." The device receives this memo and sends it to the server via an Internet connection.

[1725] The server preprocesses the received memos using the following steps: First, the memos are tokenized and divided into words and phrases. Then, unnecessary words are removed, and finally, stemming is performed. This generates tokens such as "project progress," "delayed," "cause," "materials," "not arrived," "countermeasures," "alternative materials," and "considering use."

[1726] Natural Language Processing (NLP)

[1727] The server then applies NLP algorithms to the preprocessed data. Specifically, this process involves syntactic analysis, item recognition, and sentiment analysis. Syntactic analysis analyzes the structure of the sentence and identifies the role of each word or phrase. Item recognition identifies specific items (entities), while sentiment analysis analyzes the tone and sentiment of the note. For example, "project progress" is recognized as an entity, and "being late" is analyzed as a negative sentiment.

[1728] Emotion Engine

[1729] The server then uses an emotion engine to analyze the user's emotions as they type. The system uses NLP models such as BERT and RoBERTa. The emotion engine accumulates the user's past emotional data and uses that data to learn the user's emotional patterns. This allows the system to understand the emotions with which the user has entered notes in the past and adjust the tone of the text based on those emotions.

[1730] Sentence generation

[1731] The server generates appropriate sentences based on information from the situation recognition module and emotion engine. This process uses generative AI models such as GPT-3 and T5. The generated sentences have correct grammar and structure, and are written with a tone adjusted by the emotion engine.

[1732] As a concrete example, the following sentence is generated:

[1733] To your boss

[1734] Thank you for your hard work. I'll keep you updated on the progress of the project.

[1735] The project is currently behind schedule because the necessary materials have not yet arrived. To address this issue, we are considering using alternative materials.

[1736] I would appreciate any instructions.

[1737] Thank you very much.

[1738] Output and check the text

[1739] Finally, the server sends the generated text to the terminal. The terminal displays the generated text to the user. The user can check the displayed text and make corrections as necessary. The corrections made by the user are fed back to the system and reflected in the next text generation.

[1740] This system allows users to simply input simple notes and convert them into natural-sounding sentences, significantly reducing the burden of writing. It also recognizes the user's emotions and generates sentences based on those emotions, improving the efficiency and accuracy of communication.

[1741] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1742] Step 1:

[1743] The user enters a simple note into the input field of the terminal. The entered note is text data that reads, "Project progress is delayed. Cause: Materials have not arrived. Solution: Consider using alternative materials." The terminal receives this note and sends it to the server via the Internet. Specifically, the terminal monitors the user's input in real time and sends the note in text format as an HTTP POST request to the server's API endpoint. The input is the user's note, and the output is the data to be sent to the server.

[1744] Step 2:

[1745] The server preprocesses the received notes. First, it tokenizes the notes and splits them into words and phrases. Next, it removes unnecessary words, and finally it performs stemming. For example, tokens such as "project progress," "delayed," "cause," "materials," "not arrived," "countermeasures," "alternative materials," and "considering use" are generated. Specifically, the server uses a natural language processing library (e.g., SpaCy) to execute scripts for tokenization, removal of unnecessary words, and stemming. The input is the initial note data, and the output is the preprocessed token data.

[1746] Step 3:

[1747] The server applies NLP algorithms to the preprocessed data. First, it performs syntactic analysis to analyze the structure of the sentence. Next, it performs item recognition to identify specific items (entities). Finally, it performs sentiment analysis to analyze the tone and sentiment of the note. For example, "project progress" is recognized as an entity, and "being late" is analyzed as having a negative sentiment. Specifically, the server performs these processes using a deep learning model (e.g., BERT). The input is the preprocessed token data, and the output is the analyzed data and sentiment labels.

[1748] Step 4:

[1749] The server uses an emotion engine to analyze the user's emotions in more detail. It uses models such as BERT and RoBERTa to identify the emotion of the user's input and adjust the tone of the sentence based on that emotion. Specifically, the server inputs previously accumulated user emotion data into a learning model to learn emotion patterns. The input is the analyzed data and emotion labels, and the output is adjusted emotion parameters.

[1750] Step 5:

[1751] The server generates appropriate sentences based on the context recognition module and emotion engine. In this process, it uses a generative AI model (e.g., GPT-3 or T5) to create sentences with correct grammar, structure, and tone. For example, the following sentences are generated:

[1752] To your boss

[1753] Thank you for your hard work. I'll keep you updated on the progress of the project.

[1754] The project is currently behind schedule because the necessary materials have not yet arrived. To address this issue, we are considering using alternative materials.

[1755] I would appreciate any instructions.

[1756] Thank you very much.

[1757] Specifically, the server provides the analyzed data and emotion parameters as input to the generative AI model to generate sentences. The input is the adjusted emotion parameters and analyzed data, and the output is the generated text sentence.

[1758] Step 6:

[1759] The server sends the generated text to the device. Specifically, the server sends the generated text to the device's API endpoint as an HTTP POST request. The input is the generated text data, and the output is the data to be sent to the device.

[1760] Step 7:

[1761] The terminal displays the generated text to the user, who can review the displayed text and make corrections as necessary. Specifically, the terminal displays the received text on a user interface and provides an editable text area. The input is the text data received from the server, and the output is the displayed text. If the user makes corrections, that feedback is sent to the system and reflected in the next text generation.

[1762] (Application example 2)

[1763] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1764] In security services, field personnel are required to quickly and accurately report handwritten notes. However, it is difficult to directly convert handwritten notes into reports, which can result in inconsistent tone and content. It is also challenging to generate appropriate reports that take into account the emotions and tone of field personnel. Furthermore, it is difficult to reflect personnel's corrections to generated reports in the next report. A system that can solve these problems is needed.

[1765] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for receiving input, scribbled notes; means for tokenizing the received notes and removing stop words and performing stemming; means for performing syntactic analysis, entity recognition, and sentiment analysis based on the preprocessed data; means for displaying the generated text to the field staff; means for recognizing the sentiment of the notes entered by the field staff and adjusting the tone of the report; means for inferring the situation based on the preprocessed data and referring to the field staff's past notes and behavioral patterns; and means for learning and reflecting the corrections made by the field staff in the generation process when the field staff makes corrections to the generated text. This makes it possible to quickly generate consistent and appropriate reports from handwritten notes, taking the field staff's sentiment and tone into consideration, and improving the quality of the reports by reflecting the staff's corrections in the next generation process.

[1766] "Means for receiving input scribbled notes" refers to devices or software that have the function of receiving simple handwritten notes entered by field personnel in digital format.

[1767] "Tokenization" is the process of dividing input text into words and phrases.

[1768] "Stop word removal" is the process of removing words with little meaning (such as "no", "wa", and "ni") from text.

[1769] "Stemming" is the process of converting words into their root forms.

[1770] "Syntax analysis" is a method of analyzing the structure of a sentence and clarifying the role of each word and phrase.

[1771] "Entity recognition" is the technique of identifying specific items or information within text (e.g., names of people, places, dates, etc.).

[1772] "Sentiment analysis" is a technique that analyzes a user's emotions and tone from the content of text and identifies positive, negative, or neutral emotions.

[1773] "Means for recognizing the situation and purpose" refers to devices or software that understand the situation and purpose from the analyzed data and then perform the next processing based on that information.

[1774] "Means for generating sentences using appropriate grammar, syntax, and tone" refers to devices or software that automatically create sentences using correct grammar and syntax and with a tone that reflects the results of sentiment analysis.

[1775] The "means for displaying the generated text to the on-site staff" refers to a device or software that displays the generated report on the on-site staff's device.

[1776] The "means for recognizing the emotion of a memo and adjusting the tone of a report" is a technology that analyzes the emotion of an input memo and appropriately adjusts the tone of a report based on that emotion.

[1777] "Means for inferring the situation by referring to past notes and behavioral patterns" refers to devices or software that refer to a database of past notes and behavioral patterns of field personnel and use that information to infer the current situation.

[1778] "Means for learning corrections and reflecting them in the generation process" refers to a technology that learns the corrections made by field personnel to generated text and reflects that knowledge when generating text from the next time onwards.

[1779] The present invention provides a system that enables on-site personnel to quickly and accurately report handwritten notes. Specific embodiments of the system will be described in detail below.

[1780] Note entry and preprocessing

[1781] The user inputs a simple handwritten note into a device such as a smartphone or robot. For example, the note might read, "Door forced open, intruder detected, evidence: fingerprint, alarm not activated." The device receives the note and sends it to the server. The server then performs the following preprocessing on the received note:

[1782] 1. Tokenization:

[1783] The notes are segmented into words and phrases, generating tokens such as "door," "forced," "open," "intruder," "discovered," "evidence," "fingerprint," "alarm," and "not activated."

[1784] 2. Stop word removal:

[1785] Remove words with little meaning. For example, stop words such as "teki" (target), "no" (of), and "ni" (to) are removed.

[1786] 3. Stemming:

[1787] Convert words to their root forms, for example, "open" is converted to "open."

[1788] Natural Language Processing (NLP)

[1789] Based on the preprocessed data, the server applies the following NLP algorithms:

[1790] 1. Parsing:

[1791] It analyzes the structure of a sentence and identifies the role of each word and phrase, for example, "door" is recognized as the subject and "open" as a verb.

[1792] 2. Entity Recognition:

[1793] Identifies a specific item. For example, "Evidence:Fingerprint" is recognized as an entity.

[1794] 3. Sentiment analysis:

[1795] Analyze the tone and sentiment of the memo. For example, "forced," "discovered," and "not working" are analyzed as negative sentiments.

[1796] Emotion Engine

[1797] The server uses an emotion engine to analyze the user's emotion when inputting information and adjusts the tone of the report accordingly. For example, if a user inputs a note containing negative emotions such as "forced," "discovered," or "not working," the tone of the entire document will be made more formal and careful. The server also accumulates the user's past emotional data and behavioral patterns and uses them to generate documents optimized for the current situation.

[1798] Text generation and display

[1799] The server generates sentences using appropriate grammar, syntax, and tone based on the output of the context recognition module and emotion engine. For example, the following sentences are generated:

[1800] "Security, this is a scene surveillance report. The door was forced open and an intruder was found. Fingerprints were left as evidence and the alarm was not activated. Immediate action is required."

[1801] The generated sentences are displayed on the terminals of the on-site personnel. The on-site personnel can check the displayed sentences and make corrections as necessary. These corrections are fed back into the system and reflected in the next sentence generation.

[1802] Examples of specific examples and prompts

[1803] As a specific example, consider the case where a field worker inputs the following memo:

[1804] Note: "Door forced open. Intruder found. Evidence: fingerprint. Alarm not activated."

[1805] An example of a generated prompt statement:

[1806] Prompt input to model:

[1807] On-site report notes: "Door forced open. Intruder found. Evidence: fingerprints. Alarm not activated."

[1808] Generate sentences based on the analysis results.

[1809] The server receives the prompt and uses a predefined algorithm and emotion engine to generate the appropriate response for the field representative, enabling fast and accurate reporting while also taking into account the representative's emotion and tone.

[1810] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1811] Step 1:

[1812] User Input

[1813] The user inputs a simple handwritten note into a device such as a smartphone or robot. For example, a note such as "Door forced open, intruder detected, evidence: fingerprint, alarm not activated" is input. The input note is received by the device.

[1814] Step 2:

[1815] Send a note

[1816] The terminal sends the received memo to the server, and when the memo arrives at the server, the next process begins.

[1817] Step 3:

[1818] Pretreatment

[1819] The server pre-processes the received notes, which includes:

[1820] Tokenization: Splits the input notes into words and phrases, generating tokens such as "door," "forced," "open," "intruder," "discovered," "evidence," "fingerprint," "alarm," and "not activated."

[1821] Stop word removal: Remove words with little meaning. For example, "teki" (target), "no" (of), and "ni" (to) are removed.

[1822] Stemming: Converting words into their root forms, for example, "open" is converted to "open."

[1823] Step 4:

[1824] Natural Language Processing (NLP)

[1825] The server applies the following NLP algorithms based on the preprocessed data:

[1826] Syntactic analysis: Analyzes the structure of a sentence and identifies the role of each word or phrase. For example, "door" is recognized as the subject and "open" as a verb.

[1827] Entity Recognition: Identifying specific items or information within text (e.g., "Evidence: Fingerprint").

[1828] Sentiment Analysis: Analyzes the user's emotions and tone from the text content to identify positive, negative, or neutral sentiment. For example, "forced," "discovered," and "not working" are analyzed as negative sentiments.

[1829] Step 5:

[1830] Emotion Engine

[1831] The server uses an emotion engine to analyze the user's emotions when they type and adjust the tone of the report accordingly. For example, it generates a more formal and careful tone for notes containing negative emotions. It also references the user's past emotional data and behavioral patterns to generate a report optimized for the current situation.

[1832] Step 6:

[1833] Sentence generation

[1834] Based on the output of the context recognition module and emotion engine, the server generates sentences using appropriate grammar, syntax, and tone. For example, the following sentences are generated:

[1835] "Security, this is a scene surveillance report. The door was forced open and an intruder was found. Fingerprints were left as evidence and the alarm was not activated. Immediate action is required."

[1836] Step 7:

[1837] Displaying text

[1838] The generated text is sent to a terminal and displayed to the on-site staff, who can then review the text and make corrections as necessary.

[1839] Step 8:

[1840] Correction Feedback

[1841] If a field worker makes corrections to the text, the server stores the corrections as learning data and reflects them in the next text generation process. This continuous feedback improves the accuracy and adaptability of text generation.

[1842] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1843] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1844] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1845] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1846] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1847] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1848] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1849] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1850] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1851] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1852] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1853] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1854] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1855] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1856] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1857] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1858] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1859] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1860] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1861] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1862] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1863] The following is further disclosed regarding the above embodiment.

[1864] (Claim 1)

[1865] A means for receiving inputted scribbled notes;

[1866] means for tokenizing the received notes and removing stop words and stemming;

[1867] means for performing syntactic parsing, entity recognition, and sentiment analysis on the preprocessed data;

[1868] A means of recognizing situations and objectives based on the analyzed data,

[1869] A means of producing sentences using appropriate grammar, syntax, and tone;

[1870] and means for displaying the generated sentence to the user.

[1871] (Claim 2)

[1872] The system according to claim 1, further comprising means for inferring a situation based on the preprocessed data and taking into account the user's past notes and behavioral patterns.

[1873] (Claim 3)

[1874] 2. The system according to claim 1, further comprising means for learning and reflecting in the generation process, when a user makes corrections to the generated sentence.

[1875] "Example 1"

[1876] (Claim 1)

[1877] means for receiving the entered brief memo;

[1878] means for dividing received notes into words and phrases, removing words with little meaning, and converting words to their root forms;

[1879] means for performing syntactic parsing, entity recognition, and sentiment analysis on the preprocessed data;

[1880] A means of recognizing situations and objectives based on the analyzed data,

[1881] A means of producing text using correct grammar, appropriate syntax, and a tone appropriate to the reader;

[1882] means for displaying the generated text to a user;

[1883] and means for the user to review and correct the displayed text.

[1884] (Claim 2)

[1885] The system according to claim 1, further comprising means for inferring a situation based on the preprocessed data and taking into account the user's past input history and behavioral patterns.

[1886] (Claim 3)

[1887] 2. The system according to claim 1, further comprising means for learning, when a user makes corrections to the generated sentence, the corrections and reflecting them in subsequent sentence generation processes.

[1888] "Application Example 1"

[1889] Claiming a new invention

[1890] (Claim 1)

[1891] A means for receiving inputted scribbled notes;

[1892] means for tokenizing the received notes and removing stop words and stemming;

[1893] means for performing syntactic parsing, entity recognition, and sentiment analysis on the preprocessed data;

[1894] A means of recognizing situations and objectives based on the analyzed data,

[1895] A means for generating food delivery correspondence messages using appropriate grammar, syntax, and tone;

[1896] and means for displaying the generated text to a user and accepting feedback.

[1897] (Claim 2)

[1898] The system of claim 1 further comprises means for inferring a situation based on the preprocessed data and referring to the user's past notes and behavioral patterns, and customizing the generated food delivery message.

[1899] (Claim 3)

[1900] The system of claim 1, further comprising means for learning a modification made by a user to a generated food delivery message and reflecting the modification in a next generation process.

[1901] "Example 2: Combining Emotion Engines"

[1902] (Claim 1)

[1903] means for receiving input information;

[1904] means for tokenizing the received information and for removing unnecessary words and stemming;

[1905] means for performing syntactic parsing, item recognition, and sentiment analysis on the preprocessed data;

[1906] A means of recognizing situations and objectives based on analyzed data;

[1907] A means of producing sentences using appropriate grammar, structure, and tone;

[1908] and means for displaying the generated text.

[1909] (Claim 2)

[1910] The system according to claim 1, further comprising means for inferring a situation based on the preprocessed data and taking into account the user's past information and behavioral patterns.

[1911] (Claim 3)

[1912] 2. The system according to claim 1, further comprising means for learning and reflecting in the generation process, when a user makes corrections to the generated sentence.

[1913] "Application example 2 when combining emotion engines"

[1914] (Claim 1)

[1915] A means for receiving inputted scribbled notes;

[1916] means for tokenizing the received notes and removing stop words and stemming;

[1917] means for performing syntactic parsing, entity recognition, and sentiment analysis on the preprocessed data;

[1918] A means of recognizing situations and objectives based on the analyzed data,

[1919] A means of producing sentences using appropriate grammar, syntax, and tone;

[1920] A means for displaying the generated text to the field personnel; and

[1921] The system includes a means for recognizing the sentiment of notes entered by field personnel and adjusting the tone of the report.

[1922] (Claim 2)

[1923] The system according to claim 1, further comprising means for inferring a situation based on the preprocessed data and referring to past notes and behavioral patterns of field personnel.

[1924] (Claim 3)

[1925] 2. The system according to claim 1, further comprising means for learning and reflecting in the generation process, when a field worker makes corrections to the generated sentences. [Explanation of symbols]

[1926] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for receiving inputted scribbled notes; means for tokenizing the received notes and removing stop words and stemming; means for performing syntactic parsing, entity recognition, and sentiment analysis on the preprocessed data; A means of recognizing situations and objectives based on the analyzed data, A means of producing sentences using appropriate grammar, syntax, and tone; and means for displaying the generated sentence to the user.

2. The system according to claim 1 , further comprising means for inferring a situation based on the preprocessed data and taking into account the user's past notes and behavioral patterns.

3. The system according to claim 1 , further comprising means for learning corrections made by a user to the generated text and reflecting the corrections in the generation process.

Images