System

Abstract

A system is provided.SOLUTION: The system includes a means for inputting data created by a material creator, a means for analyzing the data and extracting an important element, a means for performing evaluation on the basis of the extracted element, a means for generating feedback on the basis of an evaluation result, and a means for providing the feedback to the material creator.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] When creating documents, the process of a supervisor reviewing documents created by employees and issuing instructions for revisions takes a lot of time and effort. In particular, when the supervisor's evaluation criteria are unclear or when multiple revisions and review are required, the efficiency of document creation drops significantly. Furthermore, the supervisor's evaluation criteria are often not accurately communicated to employees, resulting in the inability to complete high-quality documents in a short period of time. To solve these issues, a system was needed that uses AI that has learned the supervisor's evaluation criteria to streamline the process from document creation to revision. [Means for solving the problem]

[0005] The present invention provides a system that includes a means for inputting data created by a document creator, analyzing the data, and extracting important elements, a means for evaluating the data based on the extracted elements, a means for generating feedback based on the evaluation results, and a means for providing the feedback to the document creator. Using this system, the document creator can receive feedback based on the supervisor's evaluation criteria, revise the document, and re-enter the feedback into the system to efficiently create high-quality documents. Furthermore, by using natural language processing and image recognition technology to analyze the data, the system can more accurately evaluate the content of the document. This improves the efficiency of document creation and enables the rapid provision of documents that meet the supervisor's evaluation criteria.

[0006] A "document creator" is someone who is responsible for creating materials such as presentations and reports.

[0007] "Data" refers to the contents of presentation files and reports created by document creators.

[0008] "Means" refers to a method or apparatus for performing a specified function or operation.

[0009] "Input" refers to the action of providing data created by the document creator to the system.

[0010] "Analysis" is the process of breaking down the content of a document and extracting its important elements.

[0011] "Important elements" refer to points or content that should be particularly evaluated in the material.

[0012] "Extraction" is the process of extracting specific elements from data through analysis.

[0013] "Evaluation" refers to the act of judging the quality and content of materials based on extracted elements.

[0014] "Feedback" refers to comments and instructions for correction or improvement provided to the document creator based on the evaluation results.

[0015] "Providing" refers to the act of passing the feedback generated by the system to the creator of the material.

[0016] "Natural language processing" is a technology that allows computers to understand and analyze human language.

[0017] "Image recognition technology" is a technology that allows a computer to understand and analyze the content of an image.

[0018] "Revision" refers to the act of changing the content of the material based on feedback.

[0019] "Re-entering" refers to the act of providing the corrected data to the system again.

[0020] "Evaluation criteria" refers to the standards and indicators used to judge the quality and content of materials.

[0021] "Efficiency" refers to making it possible to complete the process from document creation to revision in a short amount of time. [Brief explanation of the drawings]

[0022] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5]FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0023] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0024] First, the terms used in the following description will be explained.

[0025] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0026] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0027] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0028] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0029] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0030] [First embodiment]

[0031] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0032] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0033] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0034] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0035] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0036] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0037] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0038] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0039] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0040] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0041] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0042] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0043] The system of the present invention aims to enable a document creator to quickly and efficiently create documents that meet the supervisor's evaluation criteria. To achieve this aim, the system includes the following means.

[0044] 1. Uploading materials

[0045] The user uploads the materials (presentation files and reports) they have created to the system from their terminal. The user selects the material file using the system interface and provides the data by clicking the upload button.

[0046] 2. Analysis of the data

[0047] The server analyzes the received data using natural language processing (NLP) and image recognition technologies to extract important elements such as text, graphs, and images. This analysis provides a detailed understanding of the data's contents.

[0048] 3. Evaluation of materials

[0049] The server evaluates the document based on the extracted key elements. The evaluation is performed using an AI model that reflects the supervisor's evaluation criteria. The model judges the quality of the document from aspects such as clarity, specificity, and design.

[0050] 4. Generate feedback

[0051] The server generates specific feedback based on the evaluation results. The generated feedback details which parts of the material should be improved and how. The feedback includes specific correction instructions such as improving text, rearranging graphs, adding images, etc.

[0052] 5. Providing Feedback

[0053] The server provides the generated feedback to the terminal, and the user can see which parts of the material need to be revised through the feedback displayed on the terminal screen.

[0054] 6. Modification of Materials

[0055] The user modifies the document based on the provided feedback. The user then refers to the feedback and changes the content of the document to make it meet the manager's evaluation criteria.

[0056] 7. Re-upload and Re-rating

[0057] The user then uploads the revised document back to the system. The server then repeats the process, evaluating the document and providing new feedback. This process continues until the document meets the supervisor's evaluation criteria.

[0058] Specific examples

[0059] Example 1: Improving the title

[0060] A user uploads a document called "Annual Sales Report." Based on the server's evaluation criteria, the server returns feedback that the title is not specific. The user then corrects the title to "Annual Sales Report 2023 – Q1 Overview" and re-uploads the document.

[0061] Example 2: Improving graph visualization

[0062] A user uploads a slide showing sales data in the form of a bar graph. The server returns feedback that "there is too much data and it's difficult to understand." The user highlights some of the data and adds a sub-slide based on that. The revised slide is then uploaded again, and the next feedback is received.

[0063] These procedures enable document creators to create high-quality documents that satisfy their superiors in a short period of time. Each method in the system is designed to streamline the document creation process and reduce repetition of work.

[0064] The processing flow will be explained below.

[0065] Step 1: Upload your data

[0066] Users upload PowerPoint presentations or reports created on their own devices to the system as files. Users select the files from the system interface and click the upload button. The device then sends the selected files to the server.

[0067] Step 2: Receiving and storing materials

[0068] The server receives the document file sent by the user and stores the received file in temporary storage within the server.

[0069] Step 3: Analyze the material

[0070] The server analyzes the stored files and breaks down the content of the documents, using natural language processing (NLP) technology to analyze the text content and image recognition technology to extract visual elements such as images and graphs.

[0071] Step 4: Extracting important elements

[0072] The server extracts important elements from the document (e.g., title, headings, graphs, tables, key points, etc.) from the analyzed content. Each element is recorded in the database as an independent element.

[0073] Step 5: Evaluate the material

[0074] The server evaluates the entire document based on the extracted important elements, using specific indicators (e.g., clarity, specificity, and design consistency) based on the supervisor's evaluation criteria that it has learned in advance.

[0075] Step 6: Generate feedback

[0076] The server generates feedback based on the evaluation results, creating comments that point out areas for improvement or missing elements, and presents them in a format that is easy for the user to understand.

[0077] Step 7: Provide feedback

[0078] The server sends the generated feedback to the user's device, where the user can view the feedback on the device screen.

[0079] Step 8: Modify the material

[0080] Users can then modify their own documents based on the feedback they receive from the server, for example by making the title more specific or improving the layout of the graphs.

[0081] Step 9: Re-upload

[0082] The user then re-uploads the revised document to the system. The re-uploading procedure is the same as in step 1.

[0083] Step 10: Reassess and feedback iteratively

[0084] The server then repeats steps 2 to 7 for the newly uploaded materials. This process is repeated until the materials meet the supervisor's evaluation criteria. The user can continue to receive feedback and make revisions as many times as they like.

[0085] This series of steps enables the document creator to efficiently create documents that will satisfy their superiors.

[0086] Example 1

[0087] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0088] The goal of this project is to solve the problem of time-consuming and labor-intensive manual corrections and repetitive tasks that are required when document creators create documents that meet their superiors' evaluation criteria efficiently and quickly. Specifically, a system is needed that supports document creators in making effective corrections without hesitation to meet evaluation criteria such as whether the content of the document is specific, easy to understand, and well-designed.

[0089] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0090] In this invention, the server includes means for inputting data created by a document creator, means for analyzing the data and extracting important elements, means for making an evaluation based on the extracted elements, means for generating feedback based on the evaluation results, means for providing the feedback to the document creator, means for the document creator to correct the data based on the generated feedback, means for re-inputting the corrected data, and means for re-analyzing and re-evaluating the corrected data. This allows the document creator to make corrections quickly and accurately based on the feedback provided by the system, making it possible to efficiently create high-quality documents that meet the evaluation criteria of their superiors.

[0091] A "material creator" is a user who creates a material and uploads it to the system.

[0092] "Data" refers to document files such as presentation files and reports created by document creators.

[0093] "Means of input" refers to the interface that allows document creators to upload data into the system.

[0094] "Means of analysis" refers to the process by which the server analyzes the data received using natural language processing and image recognition technology to extract important elements.

[0095] "Important elements" refer to items necessary for evaluating materials such as text, graphs, and images in the data.

[0096] "Means of evaluation" refers to an AI model that evaluates the quality of materials based on extracted elements.

[0097] "Means of generating feedback" refers to the process of creating specific corrective instructions based on the evaluation results.

[0098] "Means for providing" refers to an interface for notifying the material creator of the generated feedback and displaying it.

[0099] "Measures to revise" refers to actions taken by the material creator to revise the material based on the feedback provided.

[0100] "Means for re-entering" refers to an interface for uploading corrected materials back into the system.

[0101] "Reanalysis and reevaluation" refers to the process of reanalyzing and evaluating the corrected data.

[0102] "Generative AI model" refers to an artificial intelligence model used to evaluate materials and generate feedback.

[0103] A "prompt" refers to a specific question or instruction that feeds data into a generative AI model.

[0104] The present invention relates to a system that enables document creators to efficiently and quickly create documents that meet their supervisor's evaluation criteria. To achieve this goal, the system includes a series of means, specifically, procedures for uploading documents, analyzing, evaluating, generating feedback, providing feedback, correcting, and reevaluating.

[0105] Hardware and Software

[0106] The hardware that realizes this system includes a server and user terminals. The server is equipped with a high-performance processor and large-capacity memory, and by incorporating a GPU in particular, the processing power of the AI ​​model can be improved. Terminals can be general-purpose personal computers or tablet terminals.

[0107] The following software is used:

[0108] Natural Language Processing (NLP) libraries: NLTK (Natural Language Toolkit), SpaCy

[0109] Image recognition library: OpenCV, TensorFlow

[0110] AI model frameworks: TensorFlow, PyTorch

[0111] Uploading materials

[0112] The user opens the system interface using a terminal and uploads the created document. This operation is completed by clicking the "Select File" button on the simple interface, selecting the document file, and then clicking the "Upload" button. The terminal sends the document file to the server using an HTTP request.

[0113] Analysis of data

[0114] The server uses natural language processing (NLP) and image recognition technologies to analyze the documents received. Specifically, the server uses NLTK and SpaCy to analyze the text, and OpenCV and TensorFlow to analyze graphs and images. This allows the server to extract important elements from the documents.

[0115] Evaluation of materials

[0116] The server evaluates the documents based on the analyzed data. This evaluation uses a generative AI model that reflects the supervisor's evaluation criteria. The AI ​​model used is a generative AI model (e.g., GPT-3), which evaluates the documents' clarity, specificity, design, etc.

[0117] Generate feedback

[0118] The server generates specific feedback based on the evaluation results. The generated feedback includes specific instructions on how to improve the material, such as improving the text, rearranging graphs, or adding images. The server uses a generative AI model to generate the feedback in natural language.

[0119] Providing feedback

[0120] The generated feedback is sent from the server to the device, where the user can check it. The feedback specifically indicates which slides in the document need to be revised and what kind of revisions are required. The user can review this and begin revising the document.

[0121] Corrections to the material

[0122] The user then modifies the document based on the provided feedback, such as by modifying the text, rearranging the graphs, and adding new images as needed, according to the feedback instructions. After modifying the document, the user then saves the modified document.

[0123] Re-upload and re-evaluation

[0124] The user then uploads the revised material back into the system, after which the server repeats the analysis and evaluation process described above and provides new feedback. This process is repeated until the material meets the supervisor's evaluation criteria.

[0125] Examples of concrete examples and prompts

[0126] Example 1: Improving the title

[0127] A user uploads a document called "Annual Sales Report." Based on the server's evaluation criteria, the server returns feedback that the title is not specific. The user corrects the title to "Annual Sales Report 2023 – Q1 Overview" and re-uploads it. Example prompt: "How can you make the title of this presentation more specific?"

[0128] Example 2: Improving graph visualization

[0129] A user uploads a slide showing sales data in the form of a bar graph. The server provides feedback that "there is too much data and it's difficult to understand." The user highlights some of the data and adds a sub-slide based on that. The revised slide is then uploaded again and the following feedback is received: Example prompt: "How can you modify this graph to make it easier to read?"

[0130] As described above, this system provides a series of functions to streamline the document creation process and is designed to enable document creators to quickly create high-quality documents.

[0131] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0132] Step 1: Upload your materials

[0133] The user opens the system interface using a terminal and selects the created document. The user clicks the "Select File" button, selects the document file, and then clicks the "Upload" button. The terminal sends the document file to the server using an HTTP request.

[0134] Input: User-created material file

[0135] Output: The document file sent to the server

[0136] Specific operation: When the user clicks the "Upload" button, the device sends the document file to the server. The file is sent as an attachment to the HTTP request.

[0137] Step 2: Analyze the data

[0138] The server analyzes the received file. To analyze, it first reads the file contents and separates the text from the images. It then uses natural language processing (NLP) technology to analyze the text portion, and image recognition technology to analyze the graphs and images.

[0139] Input: Data file sent to the server

[0140] Output: Extracted important elements (text data, image data)

[0141] Specific operation: The server uses NLTK and SpaCy to analyze the text and identify keywords and sentence structure, and OpenCV and TensorFlow to extract elements from graphs and images and obtain features.

[0142] Step 3: Evaluate the material

[0143] The server evaluates the document based on the analyzed data. This evaluation uses a generative AI model that reflects the supervisor's evaluation criteria. The generative AI model receives the analyzed data as input and evaluates the document based on that.

[0144] Input: Extracted important elements (text data, image data)

[0145] Output: Evaluation results (quality score and evaluation comments)

[0146] Specific operation: The server inputs the analysis results into the generative AI model and generates an evaluation result. The server compares the results with the evaluation criteria to assign a score and generate evaluation comments.

[0147] Step 4: Generate feedback

[0148] The server generates specific feedback based on the evaluation results, including specific instructions on how to improve the material.

[0149] Input: Evaluation results (quality score and evaluation comments)

[0150] Output: Generated feedback (correction instructions)

[0151] Specific behavior: Based on the evaluation results, the server uses a generative AI model to generate feedback, which includes specific correction instructions such as improving text, rearranging graphs, and adding images.

[0152] Step 5: Provide feedback

[0153] The server generates feedback and sends it to the device. The user can view the feedback on the device. The server formats the feedback data into structured data such as JSON format and sends it to the device.

[0154] Input: Generated feedback (correction instructions)

[0155] Output: Feedback provided to the terminal

[0156] Specific operation: The server sends feedback data to the terminal, and the terminal displays the received feedback on the screen and notifies the user.

[0157] Step 6: Modify the material

[0158] The user modifies the material based on the feedback provided, specifically by modifying the text, rearranging charts, or adding new images as directed by the feedback.

[0159] Input: Feedback provided

[0160] Output: Modified documentation file

[0161] Specific operation: The user makes corrections using document editing software (e.g., Microsoft PowerPoint, Google Slides) and saves the corrected document.

[0162] Step 7: Re-upload and re-evaluate

[0163] The user uploads the revised material back into the system, and the server receives it again, performs the analysis and evaluation process described above, and provides new feedback.

[0164] Input: Modified document file

[0165] Output: New evaluation results and new feedback

[0166] How it works: The user re-uploads the revised document, and the server analyzes and evaluates it again, generating and providing new feedback. This process is repeated until the document meets the supervisor's evaluation criteria.

[0167] (Application example 1)

[0168] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0169] Existing document creation support systems are primarily used in office environments, making it difficult to support document creation and reporting in the field. Furthermore, there was a lack of means to quickly analyze data acquired in the field and provide feedback, making it difficult for document creators to efficiently create high-quality documents. To solve these issues, a system that can seamlessly acquire and analyze data in the field and provide feedback is needed.

[0170] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0171] In this invention, the server includes means for inputting data created by a document creator, means for analyzing the data and extracting important elements, means for evaluating the data based on the extracted elements, means for generating feedback based on the evaluation results, means for providing the feedback to the document creator, means for using images and videos captured on-site as input, means for analyzing the data using natural language processing, image recognition technology, and OCR, and means for displaying the generated feedback on a smart device display. This makes it possible to instantly acquire and analyze data and provide feedback when creating documents on-site. As a result, document creators can quickly create high-quality documents that reflect the situation on-site.

[0172] A "document creator" is a person or organization that creates a presentation or report.

[0173] "Data" refers to a collection of information such as presentation files and reports created by document creators.

[0174] "Uploading" is the act of sending materials from a terminal to a server.

[0175] "Analysis" is the process of carefully examining and evaluating data to extract important elements.

[0176] "Key elements" are items of text, graphs, images, etc. that deserve special attention in the content of the document.

[0177] "Evaluation" is the act of judging the quality of material based on extracted important elements.

[0178] "Feedback" refers to specific comments based on the evaluation results, including suggestions for improvements and corrections to the materials.

[0179] The "site" is a place where actual work is carried out, such as a factory or a production line.

[0180] A "smart device" is a terminal that can connect to the Internet, such as smart glasses or a head-mounted display.

[0181] "OCR" stands for optical character recognition, a technology that extracts text from an image as digital data.

[0182] "Natural language processing" is a technology that allows computers to understand and analyze human language.

[0183] "Image recognition technology" is a technology that allows a computer to analyze images and understand their content.

[0184] A "display" is the display screen of a smart device, and is a device for visually checking information.

[0185] The system for realizing this invention includes the following programs. First, the document creator uses smart glasses on-site to capture images and videos. This device can be Google Glass or Microsoft HoloLens. The captured data is uploaded from the smart glasses to a server. The server receives this data and begins analysis.

[0186] This analysis process uses natural language processing (NLP), image recognition technology, and OCR technology. NLP is used to analyze the text information in the document and extract important elements. This includes understanding the content of the text and extracting keywords. Image recognition technology recognizes specific objects and situations from the image and extracts them as part of the document. OCR technology digitizes the text in the image so that it can be processed as data.

[0187] The server uses these technologies to analyze the data and evaluate the materials based on the extracted key elements. This evaluation is performed using a generative AI model that reflects the supervisor's evaluation criteria. This AI model evaluates the materials' clarity, specificity, design, and other criteria. Once the evaluation is complete, the server generates specific feedback.

[0188] The generated feedback is instantly displayed on the smart glasses' display, allowing workers to review the feedback and make on-site corrections to the materials. This feedback includes specific instructions such as suggestions for improving text, rearranging graphs, or adding images.

[0189] For example, if a worker discovers an abnormality on a production line and takes a photo of the abnormal area with smart glasses, the image is uploaded to the server. The server analyzes the image and evaluates the details of the abnormality. Feedback such as "Add a photo of the abnormality to the report" is then generated, allowing the worker to make corrections. As a result, high-quality documents that reflect the situation on-site are quickly created.

[0190] Examples of prompts for generative AI models include:

[0191] "This image shows an anomaly on the production line. Please use the system to analyze the content and generate feedback to create a report on the anomaly."

[0192] This invention allows document creators to instantly obtain documents on-site and analyze, evaluate, and provide feedback, making it possible to quickly create high-quality documents.

[0193] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0194] Step 1:

[0195] Users use smart glasses to capture images and videos needed for document creation on-site. Devices such as Google Glass and Microsoft HoloLens can be used for this purpose. The captured data is stored in the smart glasses' internal storage.

[0196] Input: Images and videos from the scene

[0197] Output: Data stored in the smart glasses storage

[0198] Step 2:

[0199] The user operates the smart glasses through an interface and uploads captured images and videos to a server, which is the process of transmitting data from the device to the server.

[0200] Input: Data stored in the smart glasses storage

[0201] Output: Data uploaded to the server

[0202] Step 3:

[0203] The server receives the uploaded data and begins analyzing it, first using OCR technology to extract text from the image, then using natural language processing (NLP) and image recognition technology to extract key elements from the image or video.

[0204] Input: Data uploaded to the server

[0205] Output: Extracted text data and image recognition results

[0206] Step 4:

[0207] The server evaluates the document based on the extracted key elements. This evaluation uses a generative AI model that reflects the supervisor's evaluation criteria. The model evaluates the document's clarity, specificity, design, and other criteria.

[0208] Input: Extracted text data and image recognition results

[0209] Output: Evaluation results

[0210] Step 5:

[0211] The server generates specific feedback based on the evaluation results, detailing how and where the material should be improved, including suggestions for improving text, rearranging graphs, adding images, and other specific corrections.

[0212] Input: Evaluation result

[0213] Output: Feedback

[0214] Step 6:

[0215] The server displays the generated feedback on the smart glasses display, allowing workers to check the feedback in real time and make corrections to the materials on-site.

[0216] Input: Feedback

[0217] Output: Feedback displayed on the smart glasses display

[0218] Step 7:

[0219] The user modifies the data based on the feedback provided through the smart glasses, and if necessary, re-acquires on-site data using the smart glasses and uploads the newly modified data to the server.

[0220] Input: Corrective instructions based on feedback

[0221] Output: Corrected material

[0222] By repeating this process, the document is adjusted until it meets the supervisor's evaluation criteria, allowing document creators to quickly create high-quality documents that reflect the situation on-site.

[0223] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0224] The system of the present invention aims to improve the process by which document creators can efficiently create documents that meet their superiors' evaluation criteria. In particular, by combining it with an emotion engine, it provides feedback that takes into account the user's emotional state.

[0225] 1. Uploading materials

[0226] The user uploads the documents (e.g., PowerPoint files or reports) they have created from their terminal to the system. The user selects the files using the system interface, clicks the upload button, and the terminal sends the selected files to the server.

[0227] 2. Receipt and storage of materials

[0228] The server receives the data file sent by the user and stores it in its internal temporary storage. The stored file is then subjected to analysis processing.

[0229] 3. Analysis of the data

[0230] The server analyzes the stored data files and breaks down their contents, using natural language processing (NLP) and image recognition technology to extract important elements such as text, graphs, and images.

[0231] 4. Extracting important elements

[0232] The server extracts important elements from the analyzed content (e.g., titles, headings, graphs, tables, key points, etc.), and records each important element in the database.

[0233] 5. Evaluation of materials

[0234] The server then evaluates the entire document based on the extracted important elements and in accordance with the supervisor's pre-trained evaluation criteria, including clarity, specificity, and design consistency.

[0235] 6. Generate feedback

[0236] The server generates specific feedback based on the evaluation results, detailing how and where the material should be improved, for example providing specific advice on improving text or instructions on rearranging graphs.

[0237] 7. Leveraging Emotional Engines

[0238] The server utilizes an emotion engine to recognize the user's emotions. The emotion engine analyzes the user's facial expressions and tone of voice to determine their current emotional state. This allows the server to determine whether the user is feeling stressed or has a high level of understanding.

[0239] 8. Providing Feedback

[0240] The server considers the results of the emotion engine and provides the generated feedback to the user. For example, if the user is stressed, the server will simplify the feedback, and if the user is calm, it will provide detailed instructions.

[0241] 9. Modification of Materials

[0242] The user then modifies the document based on the feedback received from the server. The modification work is carried out on the user's terminal, and the content of the document is changed while referring to the feedback.

[0243] 10. Re-uploading and Re-rating

[0244] The user then uploads the revised document back to the system. The server then repeats the process described above (steps 2 to 9), reassessing the document and providing new feedback. This process continues until the document meets the supervisor's evaluation criteria.

[0245] Specific examples

[0246] Example 1: Improving the title

[0247] A user uploads a document called "Annual Sales Report." Based on the server's evaluation criteria, the server returns feedback that the title is not specific. If the emotion engine analyzes the user's facial expression and recognizes that the user is stressed, it offers a simple suggestion for improvement. For example, it suggests revising the title to "Annual Sales Report 2023 – Q1 Overview."

[0248] Example 2: Improving graph visualization

[0249] A user uploads a slide displaying sales data in the form of a bar graph. When the server responds with feedback that "there's too much data and it's difficult to understand," the emotion engine determines from the user's tone of voice that they are calm and provides detailed instructions for improvement, such as suggesting ways to highlight parts of the data or adding new sub-slides.

[0250] These methods enable document creators to create high-quality documents that satisfy their superiors in a short period of time. Each method and emotion engine of the system is designed to streamline the document creation process and reduce repetition of work.

[0251] The processing flow will be explained below.

[0252] Step 1: Upload your materials

[0253] Users select PowerPoint presentations or report files they have created on their own devices, and then click the upload button on the device interface to send the files to the system.

[0254] Step 2: Receive and save the file

[0255] The server receives the uploaded file and stores it in its internal storage. This process allows the server to proceed to the next step without losing any data.

[0256] Step 3: Analyze the material

[0257] The server analyzes the stored data files, extracting text content using natural language processing (NLP) technology and analyzing visual elements such as graphs and images using image recognition technology. This analysis allows the data content to be structured.

[0258] Step 4: Extracting important elements

[0259] The server extracts important elements from the analyzed data, such as titles, headings, graphs, tables, and key points, and records each extracted element in a database for use in subsequent evaluation processes.

[0260] Step 5: Evaluate the material

[0261] The server evaluates the document based on the extracted important elements. Based on the supervisor's evaluation criteria, which it has learned in advance, the server evaluates the entire document in terms of clarity, specificity, design consistency, etc. The results of this evaluation form the basis for generating feedback.

[0262] Step 6: Generate feedback

[0263] The server generates specific feedback based on the evaluation results. For example, if the title is not specific enough, it will generate an instruction to "use a more specific title." It also includes specific advice on graph visualization, such as "the amount of data is too large, please add highlights."

[0264] Step 7: Emotion Recognition with the Emotion Engine

[0265] The server uses an emotion engine to recognize the user's emotional state. When the user checks the feedback, the server analyzes their facial expressions and tone of voice via a camera and microphone to determine their stress level and emotional state.

[0266] Step 8: Feedback adjustment based on emotional information

[0267] The server adjusts the content and presentation of the generated feedback based on the user's emotional information recognized by the emotion engine. For example, if the user is feeling stressed, the server will simplify the feedback. Conversely, if the user is calm, the server will provide detailed instructions.

[0268] Step 9: Provide feedback

[0269] The server sends the adjusted feedback to the terminal and provides it to the user, who can check the feedback on the terminal and understand the necessary corrections.

[0270] Step 10: Modifying the material

[0271] The user then modifies the material based on the feedback provided, and the modifications are performed on the user's device, updating the material according to the specific improvements.

[0272] Step 11: Re-upload and re-evaluate

[0273] The user then uploads the revised document back to the system, and the server repeats steps 2 to 9 for the newly uploaded document. This process is repeated until the document meets the supervisor's evaluation criteria.

[0274] Specific examples

[0275] Example 1: Improving the title

[0276] A user uploads a document called "Annual Sales Report." The server analyzes and evaluates it, generating feedback that the title is not specific enough. The emotion engine recognizes the user's stress level and offers a simple suggestion to revise it to "Annual Sales Report 2023 – Q1 Overview."

[0277] Example 2: Improving graph visualization

[0278] A user uploads a slide displaying sales data in the form of a bar graph. The server evaluates the slide and returns feedback that the data is too large and difficult to understand. If the emotion engine recognizes that the user is calm, it offers detailed instructions such as highlighting some of the data and adding a new sub-slide.

[0279] This series of processes enables users to efficiently create high-quality documents that meet their superiors' expectations. The system utilizes emotion recognition by an emotion engine and flexibly adjusts feedback according to the user's emotional state, providing effective support for document creation.

[0280] Example 2

[0281] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0282] The traditional document creation process has the problem that it is difficult for document creators to efficiently create documents that meet their superiors' evaluation criteria. Furthermore, the complicated process of making corrections after receiving feedback is often inefficient and time-consuming. Another problem is that the feedback content is uniform, and appropriate feedback is not provided based on the user's emotional state.

[0283] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for inputting data created by a material creator, a means for analyzing the data and extracting important elements, a means for making an evaluation based on the extracted elements, a means for generating feedback based on the evaluation results, a means for providing the feedback to the material creator, and a means for recognizing the user's emotions and adjusting the content of the feedback. This enables the material creator to efficiently create materials that meet the supervisor's evaluation criteria and provide appropriate feedback according to the user's emotional state.

[0284] A "document creator" is a person whose role is to create reports, presentation materials, etc.

[0285] "Data" is a collection of information including text, images, graphs, tables, etc. created by the document creator.

[0286] "Input means" refers to a method or device for providing data created by a document creator to the system.

[0287] "Means of analysis" are the methods and techniques that a system uses to process data, understand its contents, and extract important elements.

[0288] "Key elements" are the main information needed for evaluation and feedback, such as the title, headings, graphs, tables, and key points within the document.

[0289] "Means of evaluation" are methods or algorithms for determining and evaluating the quality of data based on the extracted key elements.

[0290] "Means for generating feedback" refers to methods and techniques for providing improvements and advice to document creators based on the evaluation results.

[0291] The "means for providing" refers to a method or device for conveying the generated feedback to the material creator.

[0292] "Means for recognizing emotions and adjusting the content of feedback" refers to methods or techniques for recognizing the user's current emotional state and adjusting the content and level of detail of the feedback accordingly.

[0293] "Natural language processing" is a technology that enables computers to understand and process human language.

[0294] "Image recognition technology" is a technology that allows a computer to analyze image data and recognize the information contained within it.

[0295] The system of the present invention allows document creators to efficiently create documents that meet their superiors' evaluation criteria. In particular, by combining this system with an emotion engine, it is possible to provide feedback that takes into account the user's emotional state.

[0296] The system consists of the following main elements:

[0297] 1. Input Method

[0298] Users use an interface to upload created document files (such as PowerPoint files or reports) from their terminals to the system. The user selects the target file in the file selection dialog and clicks the upload button, and the terminal sends the selected file to the server.

[0299] 2. Analysis Methods

[0300] The server uses natural language processing (NLP) and image recognition technologies to analyze the received data files. Specifically, it uses Apache Tika to analyze the file contents, NLTK (Natural Language Toolkit) to extract text elements, and OpenCV to recognize images and graphs.

[0301] 3. Means of extracting important elements

[0302] The server extracts important elements from the analyzed content (titles, headings, graphs, tables, key points, etc.) and records them in a database. For example, it connects to an H2 database and stores the extracted titles and graph metadata in the database.

[0303] 4. Evaluation methods

[0304] The server evaluates the documents based on the extracted key elements, using a Scikit-learn model to score them based on pre-trained evaluation criteria, including clarity, specificity, and design consistency.

[0305] 5. Means of generating feedback

[0306] The server generates specific feedback based on the evaluation results. It uses the Jinja template engine to create an HTML feedback page with suggestions for improvement. This feedback details what parts of the material should be improved and how.

[0307] 6. Emotional awareness and regulation tools

[0308] The server uses an emotion engine to recognize the user's emotions and adjusts the feedback accordingly. Specifically, it uses OpenCV and librosa to analyze the user's facial expressions and tone of voice to determine their emotional state. For example, if the user is stressed, the server will simplify the feedback, but if they are calm, it will provide more detailed instructions.

[0309] 7. Means of Providing Feedback

[0310] The server provides the generated feedback to the user, for example, when the user views the feedback in a web browser, detailed or simplified instructions are displayed based on the dynamic response from the server.

[0311] Specific examples

[0312] Example 1: Improving the title

[0313] If a user uploads a document titled "Annual Sales Report," the server will evaluate it as "not specific enough" and suggest a more specific title, "Annual Sales Report 2023 – Q1 Overview." If the emotion engine analyzes the user's facial expressions and recognizes that they are feeling stressed, the server will offer simple suggestions for improvement.

[0314] Example 2: Improving graph visualization

[0315] If a user uploads a slide showing sales data in the form of a bar graph, the server will determine that there is too much data and that it is difficult to understand. The emotion engine will then determine if the user is calm and provide detailed instructions on how to improve the slide, such as how to highlight the data or add new sub-slides.

[0316] Prompt Sentence Examples

[0317] "Please improve the title of this PowerPoint file to make it more specific and descriptive."

[0318] "How can I change the data in this bar chart to make it more visually understandable?"

[0319] As a result, document creators can create high-quality documents that satisfy their superiors in a short period of time. Each method and emotion engine of the system is designed to streamline the document creation process and reduce repetition of work.

[0320] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0321] System program processing flow

[0322] Step 1:

[0323] A document file created by a user is uploaded from the terminal to the system. The user selects a file using the system interface and clicks the upload button. The terminal sends the file selected by the user to the server. The input here is the document file selected by the user, and the output is the file received by the server. Specifically, a file selection dialog is displayed, the user selects a file, and it is sent to the server via an Ajax request using JavaScript.

[0324] Step 2:

[0325] The server receives the file sent from the device and temporarily stores it in its internal storage. The input is the file sent from the device, and the output is the temporarily stored file and its metadata. Specifically, the server receives a secure HTTP request, temporarily stores the file in RAM, and simultaneously records metadata such as the file name and upload date and time in a database.

[0326] Step 3:

[0327] The server analyzes the stored data files and decomposes their contents. It performs the analysis using natural language processing (NLP) and image recognition technologies. The input is the stored data file, and the output is the analyzed text elements and image data. Specifically, it uses Apache Tika to extract text from the file, NLTK to further decompose the text, and OpenCV to analyze images and graphs.

[0328] Step 4:

[0329] The server extracts important elements from the analyzed content. Important elements include titles, headings, graphs, tables, key points, etc. The input is the analyzed text elements and image data, and the output is the extracted important elements. Specifically, the extracted elements are stored in an H2 database, and metadata is also recorded.

[0330] Step 5:

[0331] The server evaluates the materials based on the extracted important elements. The evaluation uses pre-trained evaluation criteria and scores using a Scikit-learn model. The input is the extracted important elements, and the output is the evaluation score and comments. Specifically, the server automatically calculates the score based on the evaluation criteria and records it in a database.

[0332] Step 6:

[0333] The server generates specific feedback based on the evaluation results. The feedback includes areas for improvement and advice. The input is the evaluation score and comments, and the output is the generated feedback. Specifically, it uses the Jinja template engine to create an HTML feedback page.

[0334] Step 7:

[0335] The server uses an emotion engine to recognize the user's emotions and adjust the feedback content. The input is the user's facial expression and voice data, and the output is adjusted feedback. Specifically, OpenCV and librosa are used to analyze the user's facial expressions and tone of voice in real time, and the feedback content is adjusted according to the user's emotional state.

[0336] Step 8:

[0337] The server provides the generated feedback to the user, who then modifies the document based on the feedback. The input is the adjusted feedback, and the output is the document modified by the user. Specifically, the user views the feedback in a web browser, modifies the document, and saves it as a new file.

[0338] Step 9:

[0339] The user re-uploads the revised material to the system. The server then repeats the process described above, providing a new evaluation and feedback. The input is the revised material file, and the output is a re-evaluated score and new feedback. Specifically, the user re-uploads, and the server begins a new evaluation process.

[0340] Step 10:

[0341] This process is repeated until the document meets the supervisor's evaluation criteria. The input is a document file that is repeatedly revised and reevaluated, and the output is a final document that meets the supervisor's evaluation criteria. Specifically, the cycle of feedback and revision is repeated until a satisfactory document is completed.

[0342] (Application example 2)

[0343] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0344] In the past, when document creators tried to efficiently create documents that met their superiors' evaluation criteria, the supervisor's specific evaluation criteria and areas for improvement in the document were often not clearly presented, forcing the creator to make repeated revisions. Furthermore, feedback that took into account the creator's emotional state was not provided, which increased work stress. This resulted in an inefficient document creation process and made it difficult to create high-quality documents in a short period of time.

[0345] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0346] In this invention, the server includes means for inputting data created by a material creator, means for analyzing the data and extracting important elements, means for evaluating based on the extracted elements, means for generating feedback based on the evaluation results, means for analyzing the user's emotional state using an emotion engine, means for adjusting the feedback according to the emotional state, and means for providing the feedback to the material creator. This not only enables the material creator to efficiently create high-quality materials that meet the supervisor's evaluation criteria, but also reduces work stress and provides more effective support by providing feedback that takes into account the material creator's emotional state.

[0347] "Document Creator" refers to the individual or organization that creates the document.

[0348] "Data" refers to the entire content, including information such as text, graphs, and images created by the document creator.

[0349] "Important elements" are particularly important parts of the material extracted through analysis, including titles, headings, graphs, tables, key points, etc.

[0350] "Evaluation" refers to the process of judging the quality of material based on extracted key elements.

[0351] "Feedback" refers to specific improvement suggestions and advice provided to the document creator based on the evaluation results.

[0352] An "emotion engine" refers to software or hardware that analyzes a user's facial expressions and tone of voice to determine their current emotional state.

[0353] "Emotional situation" refers to the user's current emotional state, such as stress level or level of understanding.

[0354] "Adjustment" refers to the process of changing the content and format of feedback depending on the emotional situation analyzed by the emotion engine.

[0355] The "document creation process" refers to the series of steps from creating a document to evaluation, feedback, and revision.

[0356] The system for realizing this invention is for a document creator to efficiently create documents that conform to the supervisor's evaluation criteria, and includes the following processes.

[0357] First, the user uploads the documents (e.g., PowerPoint files or reports) they have created from their terminal to the system. The user selects the files using the system interface and clicks the upload button. The terminal then sends the selected files to the server.

[0358] Next, the server receives the document file sent by the user and stores it in internal temporary storage. The stored file is then subjected to analysis. Natural language processing (NLP) and image recognition technologies are used for the analysis to extract important elements such as text, graphs, and images. NLP technologies such as "SpaCy" and "BERT" are used, while image recognition technologies such as "OpenCV" and "TensorFlow" can be used.

[0359] The server then extracts important elements (such as titles, headings, graphs, tables, and key points) from the analyzed content, and records each important element in a database.The server then evaluates the entire document based on the extracted important elements and in accordance with the supervisor's evaluation criteria, which it has learned in advance.Evaluation criteria include clarity, specificity, and design consistency.

[0360] Based on the evaluation results, the server generates specific feedback that details which parts of the material should be improved and how. For example, it provides specific advice for improving the text or instructions for rearranging graphs. The text generation model used in this process is a generative AI model such as GPT-3.

[0361] Furthermore, the server uses an emotion engine to analyze the user's emotional state. The emotion engine analyzes the user's facial expressions and tone of voice to determine their current emotional state. The emotion engine uses tools such as "Affectiva" and "Microsoft Azure Emotion API." This allows the server to determine whether the user is feeling stressed or has a high level of understanding.

[0362] When providing feedback, the system takes into account the results of the emotion engine and provides the generated feedback to the user. For example, if the user is feeling stressed, the system will simplify the feedback content, and conversely, if the user is calm, it will provide detailed instructions. In this way, feedback appropriate to the emotional state of the document creator is provided.

[0363] As a concrete example, consider the case where a user uploads a document titled "Annual Sales Report." Based on the server's evaluation criteria, the server returns feedback that the title is not specific. If the emotion engine analyzes the user's facial expression and recognizes that the user is stressed, it offers a simple suggestion for improvement. For example, it suggests revising the title to "Annual Sales Report 2023 – Q1 Overview."

[0364] Examples of prompts include:

[0365] "Please suggest ways to make the document titles concise and specific."

[0366] "What are some best practices for rearranging graphs for better visibility?"

[0367] In this way, the system can support the document creator in efficiently creating high-quality documents that meet the supervisor's evaluation criteria.

[0368] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0369] Step 1:

[0370] Upload user-created materials.

[0371] Input: User selection of file (e.g. PowerPoint or report file)

[0372] Specific operation: The user selects the document file through the system interface and clicks the upload button.

[0373] Output: The selected files are sent to the server.

[0374] Step 2:

[0375] The server receives the document file and stores it in temporary storage.

[0376] Input: Data file sent from the user's terminal

[0377] Specific operation: The server receives the sent file and stores it in its internal temporary storage.

[0378] Output: The saved file is the target for analysis processing.

[0379] Step 3:

[0380] The server analyzes the document files and extracts important elements.

[0381] Input: Data file saved in temporary storage

[0382] Specific operation: The server uses natural language processing (NLP) techniques (e.g., SpaCy and BERT) and image recognition techniques (e.g., OpenCV and TensorFlow) to extract important elements such as text, graphs, and images.

[0383] Data processing / calculation: Analysis of text and image data in documents

[0384] Output: The extracted important elements (titles, headings, graphs, tables, key points, etc.) are recorded in a database.

[0385] Step 4:

[0386] The server evaluates the entire document based on the extracted important elements.

[0387] Input: Key elements recorded in the database

[0388] Specific behavior: The server evaluates the clarity, specificity, and design consistency based on the supervisor's evaluation criteria that it has learned in advance.

[0389] Data processing / calculation: Assessing the quality of the material based on key factors

[0390] Output: The evaluation result is generated.

[0391] Step 5:

[0392] The server generates specific feedback based on the evaluation results.

[0393] Input: Evaluation result

[0394] Specific operation: The server uses a generative AI model (e.g., "GPT-3") to generate specific improvement suggestions for the material.

[0395] Data processing / calculation: Text generation based on evaluation results

[0396] Output: Specific feedback is generated.

[0397] Step 6:

[0398] The server uses an emotion engine to analyze the user's emotional state.

[0399] Input: User's facial expression data and tone of voice data

[0400] Specific operation: The server performs analysis using an emotion engine (e.g., "Affectiva" or "Microsoft Azure Emotion API").

[0401] Data processing / calculation: Analysis of user facial expressions and tone of voice

[0402] Output: The user's emotional state is determined.

[0403] Step 7:

[0404] The server adjusts the feedback depending on the emotional situation.

[0405] Input: Generated feedback and the user's emotional state

[0406] Specific behavior: The server simplifies the feedback if the user is stressed, and provides detailed instructions if the user is calm.

[0407] Data processing / calculation: Adjusting the content and format of feedback

[0408] Output: Feedback appropriate to the user's emotions is generated.

[0409] Step 8:

[0410] The server provides feedback to the user.

[0411] Input: Feedback appropriate to the user's emotions

[0412] Specific Actions: The server provides the generated feedback to the user.

[0413] Output: The feedback the user receives is displayed on the screen.

[0414] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0415] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0416] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0417] [Second embodiment]

[0418] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0419] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0420] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0421] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0422] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0423] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0424] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0425] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0426] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0427] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0428] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0429] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0430] The system of the present invention aims to enable a document creator to quickly and efficiently create documents that meet the supervisor's evaluation criteria. To achieve this aim, the system includes the following means.

[0431] 1. Uploading materials

[0432] The user uploads the materials (presentation files and reports) they have created to the system from their terminal. The user selects the material file using the system interface and provides the data by clicking the upload button.

[0433] 2. Analysis of the data

[0434] The server analyzes the received data using natural language processing (NLP) and image recognition technologies to extract important elements such as text, graphs, and images. This analysis provides a detailed understanding of the data's contents.

[0435] 3. Evaluation of materials

[0436] The server evaluates the document based on the extracted key elements. The evaluation is performed using an AI model that reflects the supervisor's evaluation criteria. The model judges the quality of the document from aspects such as clarity, specificity, and design.

[0437] 4. Generate feedback

[0438] The server generates specific feedback based on the evaluation results. The generated feedback details which parts of the material should be improved and how. The feedback includes specific correction instructions such as improving text, rearranging graphs, adding images, etc.

[0439] 5. Providing Feedback

[0440] The server provides the generated feedback to the terminal, and the user can see which parts of the material need to be revised through the feedback displayed on the terminal screen.

[0441] 6. Modification of Materials

[0442] The user modifies the document based on the provided feedback. The user then refers to the feedback and changes the content of the document to make it meet the manager's evaluation criteria.

[0443] 7. Re-upload and Re-rating

[0444] The user then uploads the revised document back to the system. The server then repeats the process, evaluating the document and providing new feedback. This process continues until the document meets the supervisor's evaluation criteria.

[0445] Specific examples

[0446] Example 1: Improving the title

[0447] A user uploads a document called "Annual Sales Report." Based on the server's evaluation criteria, the server returns feedback that the title is not specific. The user then corrects the title to "Annual Sales Report 2023 – Q1 Overview" and re-uploads the document.

[0448] Example 2: Improving graph visualization

[0449] A user uploads a slide showing sales data in the form of a bar graph. The server returns feedback that "there is too much data and it's difficult to understand." The user highlights some of the data and adds a sub-slide based on that. The revised slide is then uploaded again, and the next feedback is received.

[0450] These procedures enable document creators to create high-quality documents that satisfy their superiors in a short period of time. Each method in the system is designed to streamline the document creation process and reduce repetition of work.

[0451] The processing flow will be explained below.

[0452] Step 1: Upload your data

[0453] Users upload PowerPoint presentations or reports created on their own devices to the system as files. Users select the files from the system interface and click the upload button. The device then sends the selected files to the server.

[0454] Step 2: Receiving and storing materials

[0455] The server receives the document file sent by the user and stores the received file in temporary storage within the server.

[0456] Step 3: Analyze the material

[0457] The server analyzes the stored files and breaks down the content of the documents, using natural language processing (NLP) technology to analyze the text content and image recognition technology to extract visual elements such as images and graphs.

[0458] Step 4: Extracting important elements

[0459] The server extracts important elements from the document (e.g., title, headings, graphs, tables, key points, etc.) from the analyzed content. Each element is recorded in the database as an independent element.

[0460] Step 5: Evaluate the material

[0461] The server evaluates the entire document based on the extracted important elements, using specific indicators (e.g., clarity, specificity, and design consistency) based on the supervisor's evaluation criteria that it has learned in advance.

[0462] Step 6: Generate feedback

[0463] The server generates feedback based on the evaluation results, specifically creating comments pointing out areas for improvement or missing elements, and presenting them in a format that is easy for the user to understand.

[0464] Step 7: Provide feedback

[0465] The server sends the generated feedback to the user's device, where the user can view the feedback on the device screen.

[0466] Step 8: Modify the material

[0467] Users can then modify their own documents based on the feedback they receive from the server, for example by making the title more specific or improving the layout of the graphs.

[0468] Step 9: Re-upload

[0469] The user then re-uploads the revised document to the system. The re-uploading procedure is the same as in step 1.

[0470] Step 10: Reassess and feedback iteratively

[0471] The server then repeats steps 2 to 7 for the newly uploaded materials. This process is repeated until the materials meet the supervisor's evaluation criteria. The user can continue to receive feedback and make revisions as many times as they like.

[0472] This series of steps enables the document creator to efficiently create documents that will satisfy their superiors.

[0473] Example 1

[0474] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0475] The goal of this project is to solve the problem of time-consuming and labor-intensive manual corrections and repetitive tasks that are required when document creators create documents that meet their superiors' evaluation criteria efficiently and quickly. Specifically, a system is needed that supports document creators in making effective corrections without hesitation to meet evaluation criteria such as whether the content of the document is specific, easy to understand, and well-designed.

[0476] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0477] In this invention, the server includes means for inputting data created by a document creator, means for analyzing the data and extracting important elements, means for making an evaluation based on the extracted elements, means for generating feedback based on the evaluation results, means for providing the feedback to the document creator, means for the document creator to correct the data based on the generated feedback, means for re-inputting the corrected data, and means for re-analyzing and re-evaluating the corrected data. This allows the document creator to make corrections quickly and accurately based on the feedback provided by the system, making it possible to efficiently create high-quality documents that meet the evaluation criteria of their superiors.

[0478] A "material creator" is a user who creates a material and uploads it to the system.

[0479] "Data" refers to document files such as presentation files and reports created by document creators.

[0480] "Means of input" refers to the interface that allows document creators to upload data into the system.

[0481] "Means of analysis" refers to the process by which the server analyzes the data received using natural language processing and image recognition technology to extract important elements.

[0482] "Important elements" refer to items necessary for evaluating materials such as text, graphs, and images in the data.

[0483] "Means of evaluation" refers to an AI model that evaluates the quality of materials based on extracted elements.

[0484] "Means of generating feedback" refers to the process of creating specific corrective instructions based on the evaluation results.

[0485] "Means for providing" refers to an interface for notifying the material creator of the generated feedback and displaying it.

[0486] "Measures to revise" refers to actions taken by the material creator to revise the material based on the feedback provided.

[0487] "Means for re-entering" refers to an interface for uploading corrected materials back into the system.

[0488] "Reanalysis and reevaluation" refers to the process of reanalyzing and evaluating the corrected data.

[0489] "Generative AI model" refers to an artificial intelligence model used to evaluate materials and generate feedback.

[0490] A "prompt" refers to a specific question or instruction that feeds data into a generative AI model.

[0491] The present invention relates to a system that enables document creators to efficiently and quickly create documents that meet their supervisor's evaluation criteria. To achieve this goal, the system includes a series of means, specifically, procedures for uploading documents, analyzing, evaluating, generating feedback, providing feedback, correcting, and reevaluating.

[0492] Hardware and Software

[0493] The hardware that realizes this system includes a server and user terminals. The server is equipped with a high-performance processor and large-capacity memory, and by incorporating a GPU in particular, the processing power of the AI ​​model can be improved. Terminals can be general-purpose personal computers or tablet terminals.

[0494] The following software is used:

[0495] Natural Language Processing (NLP) libraries: NLTK (Natural Language Toolkit), SpaCy

[0496] Image recognition library: OpenCV, TensorFlow

[0497] AI model frameworks: TensorFlow, PyTorch

[0498] Uploading materials

[0499] The user opens the system interface using a terminal and uploads the created document. This operation is completed by clicking the "Select File" button on the simple interface, selecting the document file, and then clicking the "Upload" button. The terminal sends the document file to the server using an HTTP request.

[0500] Analysis of data

[0501] The server uses natural language processing (NLP) and image recognition technologies to analyze the documents received. Specifically, the server uses NLTK and SpaCy to analyze the text, and OpenCV and TensorFlow to analyze graphs and images. This allows the server to extract important elements from the documents.

[0502] Evaluation of materials

[0503] The server evaluates the documents based on the analyzed data. This evaluation uses a generative AI model that reflects the supervisor's evaluation criteria. The AI ​​model used is a generative AI model (e.g., GPT-3), which evaluates the documents' clarity, specificity, design, etc.

[0504] Generate feedback

[0505] The server generates specific feedback based on the evaluation results. The generated feedback includes specific instructions on how to improve the material, such as improving the text, rearranging graphs, or adding images. The server uses a generative AI model to generate the feedback in natural language.

[0506] Providing feedback

[0507] The generated feedback is sent from the server to the device, where the user can check it. The feedback specifically indicates which slides in the document need to be revised and what kind of revisions are required. The user can review this and begin revising the document.

[0508] Corrections to the material

[0509] The user then modifies the document based on the provided feedback, such as by modifying the text, rearranging the graphs, and adding new images as needed, according to the feedback instructions. After modifying the document, the user then saves the modified document.

[0510] Re-upload and re-evaluation

[0511] The user then uploads the revised material back into the system, after which the server repeats the analysis and evaluation process described above and provides new feedback. This process is repeated until the material meets the supervisor's evaluation criteria.

[0512] Examples of concrete examples and prompts

[0513] Example 1: Improving the title

[0514] A user uploads a document called "Annual Sales Report." Based on the server's evaluation criteria, the server returns feedback that the title is not specific. The user corrects the title to "Annual Sales Report 2023 – Q1 Overview" and re-uploads it. Example prompt: "How can you make the title of this presentation more specific?"

[0515] Example 2: Improving graph visualization

[0516] A user uploads a slide showing sales data in the form of a bar graph. The server provides feedback that "there is too much data and it's difficult to understand." The user highlights some of the data and adds a sub-slide based on that. The revised slide is then uploaded again and the following feedback is received: Example prompt: "How can you modify this graph to make it easier to read?"

[0517] As described above, this system provides a series of functions to streamline the document creation process and is designed to enable document creators to quickly create high-quality documents.

[0518] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0519] Step 1: Upload your materials

[0520] The user opens the system interface using a terminal and selects the created document. The user clicks the "Select File" button, selects the document file, and then clicks the "Upload" button. The terminal sends the document file to the server using an HTTP request.

[0521] Input: User-created material file

[0522] Output: The document file sent to the server

[0523] Specific operation: When the user clicks the "Upload" button, the device sends the document file to the server. The file is sent as an attachment to the HTTP request.

[0524] Step 2: Analyze the data

[0525] The server analyzes the received file. To analyze, it first reads the file contents and separates the text from the images. It then uses natural language processing (NLP) technology to analyze the text portion, and image recognition technology to analyze the graphs and images.

[0526] Input: Data file sent to the server

[0527] Output: Extracted important elements (text data, image data)

[0528] Specific operation: The server uses NLTK and SpaCy to analyze the text and identify keywords and sentence structure, and OpenCV and TensorFlow to extract elements from graphs and images and obtain features.

[0529] Step 3: Evaluate the material

[0530] The server evaluates the document based on the analyzed data. This evaluation uses a generative AI model that reflects the supervisor's evaluation criteria. The generative AI model receives the analyzed data as input and evaluates the document based on that.

[0531] Input: Extracted important elements (text data, image data)

[0532] Output: Evaluation results (quality score and evaluation comments)

[0533] Specific operation: The server inputs the analysis results into the generative AI model and generates an evaluation result. The server compares the results with the evaluation criteria to assign a score and generate evaluation comments.

[0534] Step 4: Generate feedback

[0535] The server generates specific feedback based on the evaluation results, including specific instructions on how to improve the material.

[0536] Input: Evaluation results (quality score and evaluation comments)

[0537] Output: Generated feedback (correction instructions)

[0538] Specific behavior: Based on the evaluation results, the server uses a generative AI model to generate feedback, which includes specific correction instructions such as improving text, rearranging graphs, and adding images.

[0539] Step 5: Provide feedback

[0540] The server generates feedback and sends it to the device. The user can view the feedback on the device. The server formats the feedback data into structured data such as JSON format and sends it to the device.

[0541] Input: Generated feedback (correction instructions)

[0542] Output: Feedback provided to the terminal

[0543] Specific operation: The server sends feedback data to the terminal, and the terminal displays the received feedback on the screen and notifies the user.

[0544] Step 6: Modify the material

[0545] The user modifies the material based on the feedback provided, specifically by modifying the text, rearranging charts, or adding new images as directed by the feedback.

[0546] Input: Feedback provided

[0547] Output: Modified documentation file

[0548] Specific operation: The user makes corrections using document editing software (e.g., Microsoft PowerPoint, Google Slides) and saves the corrected document.

[0549] Step 7: Re-upload and re-evaluate

[0550] The user uploads the revised material back into the system, and the server receives it again, performs the analysis and evaluation process described above, and provides new feedback.

[0551] Input: Modified document file

[0552] Output: New evaluation results and new feedback

[0553] How it works: The user re-uploads the revised document, and the server analyzes and evaluates it again, generating and providing new feedback. This process is repeated until the document meets the supervisor's evaluation criteria.

[0554] (Application example 1)

[0555] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0556] Existing document creation support systems are primarily used in office environments, making it difficult to support document creation and reporting in the field. Furthermore, there was a lack of means to quickly analyze data acquired in the field and provide feedback, making it difficult for document creators to efficiently create high-quality documents. To solve these issues, a system that can seamlessly acquire and analyze data in the field and provide feedback is needed.

[0557] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0558] In this invention, the server includes means for inputting data created by a document creator, means for analyzing the data and extracting important elements, means for evaluating the data based on the extracted elements, means for generating feedback based on the evaluation results, means for providing the feedback to the document creator, means for using images and videos captured on-site as input, means for analyzing the data using natural language processing, image recognition technology, and OCR, and means for displaying the generated feedback on a smart device display. This makes it possible to instantly acquire and analyze data and provide feedback when creating documents on-site. As a result, document creators can quickly create high-quality documents that reflect the situation on-site.

[0559] A "document creator" is a person or organization that creates a presentation or report.

[0560] "Data" refers to a collection of information such as presentation files and reports created by document creators.

[0561] "Uploading" is the act of sending materials from a terminal to a server.

[0562] "Analysis" is the process of carefully examining and evaluating data to extract important elements.

[0563] "Key elements" are items of text, graphs, images, etc. that deserve special attention in the content of the document.

[0564] "Evaluation" is the act of judging the quality of material based on extracted important elements.

[0565] "Feedback" refers to specific comments based on the evaluation results, including suggestions for improvements and corrections to the materials.

[0566] The "site" is a place where actual work is carried out, such as a factory or a production line.

[0567] A "smart device" is a terminal that can connect to the Internet, such as smart glasses or a head-mounted display.

[0568] "OCR" stands for optical character recognition, a technology that extracts text from an image as digital data.

[0569] "Natural language processing" is a technology that allows computers to understand and analyze human language.

[0570] "Image recognition technology" is a technology that allows a computer to analyze images and understand their content.

[0571] A "display" is the display screen of a smart device, and is a device for visually checking information.

[0572] The system for realizing this invention includes the following programs. First, the document creator uses smart glasses on-site to capture images and videos. This device can be Google Glass or Microsoft HoloLens. The captured data is uploaded from the smart glasses to a server. The server receives this data and begins analysis.

[0573] This analysis process uses natural language processing (NLP), image recognition technology, and OCR technology. NLP is used to analyze the text information in the document and extract important elements. This includes understanding the content of the text and extracting keywords. Image recognition technology recognizes specific objects and situations from the image and extracts them as part of the document. OCR technology digitizes the text in the image so that it can be processed as data.

[0574] The server uses these technologies to analyze the data and evaluate the materials based on the extracted key elements. This evaluation is performed using a generative AI model that reflects the supervisor's evaluation criteria. This AI model evaluates the materials' clarity, specificity, design, and other criteria. Once the evaluation is complete, the server generates specific feedback.

[0575] The generated feedback is instantly displayed on the smart glasses' display, allowing workers to review the feedback and make on-site corrections to the materials. This feedback includes specific instructions such as suggestions for improving text, rearranging graphs, or adding images.

[0576] For example, if a worker discovers an abnormality on a production line and takes a photo of the abnormal area with smart glasses, the image is uploaded to the server. The server analyzes the image and evaluates the details of the abnormality. Feedback such as "Add a photo of the abnormality to the report" is then generated, allowing the worker to make corrections. As a result, high-quality documents that reflect the situation on-site are quickly created.

[0577] Examples of prompts for generative AI models include:

[0578] "This image shows an anomaly on the production line. Please use the system to analyze the content and generate feedback to create a report on the anomaly."

[0579] This invention allows document creators to instantly obtain documents on-site and analyze, evaluate, and provide feedback, making it possible to quickly create high-quality documents.

[0580] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0581] Step 1:

[0582] Users use smart glasses to capture images and videos needed for document creation on-site. Devices such as Google Glass and Microsoft HoloLens can be used for this purpose. The captured data is stored in the smart glasses' internal storage.

[0583] Input: Images and videos from the scene

[0584] Output: Data stored in the smart glasses storage

[0585] Step 2:

[0586] The user operates the smart glasses through an interface and uploads captured images and videos to a server, which is the process of transmitting data from the device to the server.

[0587] Input: Data stored in the smart glasses storage

[0588] Output: Data uploaded to the server

[0589] Step 3:

[0590] The server receives the uploaded data and begins analyzing it, first using OCR technology to extract text from the image, then using natural language processing (NLP) and image recognition technology to extract key elements from the image or video.

[0591] Input: Data uploaded to the server

[0592] Output: Extracted text data and image recognition results

[0593] Step 4:

[0594] The server evaluates the document based on the extracted key elements. This evaluation uses a generative AI model that reflects the supervisor's evaluation criteria. The model evaluates the document's clarity, specificity, design, and other criteria.

[0595] Input: Extracted text data and image recognition results

[0596] Output: Evaluation results

[0597] Step 5:

[0598] The server generates specific feedback based on the evaluation results, detailing how and where the material should be improved, including suggestions for improving text, rearranging graphs, adding images, and other specific corrections.

[0599] Input: Evaluation result

[0600] Output: Feedback

[0601] Step 6:

[0602] The server displays the generated feedback on the smart glasses display, allowing workers to check the feedback in real time and make corrections to the materials on-site.

[0603] Input: Feedback

[0604] Output: Feedback displayed on the smart glasses display

[0605] Step 7:

[0606] The user modifies the data based on the feedback provided through the smart glasses, and if necessary, re-acquires on-site data using the smart glasses and uploads the newly modified data to the server.

[0607] Input: Corrective instructions based on feedback

[0608] Output: Corrected material

[0609] By repeating this process, the document is adjusted until it meets the supervisor's evaluation criteria, allowing document creators to quickly create high-quality documents that reflect the situation on-site.

[0610] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0611] The system of the present invention aims to improve the process by which document creators can efficiently create documents that meet their superiors' evaluation criteria. In particular, by combining it with an emotion engine, it provides feedback that takes into account the user's emotional state.

[0612] 1. Uploading materials

[0613] The user uploads the documents (e.g., PowerPoint files or reports) they have created from their terminal to the system. The user selects the files using the system interface, clicks the upload button, and the terminal sends the selected files to the server.

[0614] 2. Receipt and storage of materials

[0615] The server receives the data file sent by the user and stores it in its internal temporary storage. The stored file is then subjected to analysis processing.

[0616] 3. Analysis of the data

[0617] The server analyzes the stored data files and breaks down their contents, using natural language processing (NLP) and image recognition technology to extract important elements such as text, graphs, and images.

[0618] 4. Extracting important elements

[0619] The server extracts important elements from the analyzed content (e.g., titles, headings, graphs, tables, key points, etc.), and records each important element in the database.

[0620] 5. Evaluation of materials

[0621] The server then evaluates the entire document based on the extracted important elements and in accordance with the supervisor's pre-trained evaluation criteria, including clarity, specificity, and design consistency.

[0622] 6. Generate feedback

[0623] The server generates specific feedback based on the evaluation results, detailing how and where the material should be improved, for example providing specific advice on improving text or instructions on rearranging graphs.

[0624] 7. Leveraging Emotional Engines

[0625] The server utilizes an emotion engine to recognize the user's emotions. The emotion engine analyzes the user's facial expressions and tone of voice to determine their current emotional state. This allows the server to determine whether the user is feeling stressed or has a high level of understanding.

[0626] 8. Providing Feedback

[0627] The server considers the results of the emotion engine and provides the generated feedback to the user. For example, if the user is stressed, the server will simplify the feedback, and if the user is calm, it will provide detailed instructions.

[0628] 9. Modification of Materials

[0629] The user then modifies the document based on the feedback received from the server. The modification work is carried out on the user's terminal, and the content of the document is changed while referring to the feedback.

[0630] 10. Re-uploading and Re-rating

[0631] The user then uploads the revised document back to the system. The server then repeats the process described above (steps 2 to 9), reassessing the document and providing new feedback. This process continues until the document meets the supervisor's evaluation criteria.

[0632] Specific examples

[0633] Example 1: Improving the title

[0634] A user uploads a document called "Annual Sales Report." Based on the server's evaluation criteria, the server returns feedback that the title is not specific. If the emotion engine analyzes the user's facial expression and recognizes that the user is stressed, it offers a simple suggestion for improvement. For example, it suggests revising the title to "Annual Sales Report 2023 – Q1 Overview."

[0635] Example 2: Improving graph visualization

[0636] A user uploads a slide showing sales data in the form of a bar graph. When the server responds with feedback that "there's too much data and it's difficult to understand," the emotion engine determines from the user's tone of voice that they are calm and provides detailed instructions for improvement, such as suggesting ways to highlight parts of the data or adding new sub-slides.

[0637] These methods enable document creators to create high-quality documents that satisfy their superiors in a short period of time. Each method and emotion engine of the system is designed to streamline the document creation process and reduce repetition of work.

[0638] The processing flow will be explained below.

[0639] Step 1: Upload your materials

[0640] Users select PowerPoint presentations or report files they have created on their own devices, and then click the upload button on the device interface to send the files to the system.

[0641] Step 2: Receive and save the file

[0642] The server receives the uploaded file and stores it in its internal storage. This process allows the server to proceed to the next step without losing any data.

[0643] Step 3: Analyze the material

[0644] The server analyzes the stored data files, extracting text content using natural language processing (NLP) technology and analyzing visual elements such as graphs and images using image recognition technology. This analysis allows the data content to be structured.

[0645] Step 4: Extracting important elements

[0646] The server extracts important elements from the analyzed data, such as titles, headings, graphs, tables, and key points, and records each extracted element in a database for use in subsequent evaluation processes.

[0647] Step 5: Evaluate the material

[0648] The server evaluates the document based on the extracted important elements. Based on the supervisor's evaluation criteria, which it has learned in advance, the server evaluates the entire document in terms of clarity, specificity, design consistency, etc. The results of this evaluation form the basis for generating feedback.

[0649] Step 6: Generate feedback

[0650] The server generates specific feedback based on the evaluation results. For example, if the title is not specific enough, it will generate an instruction to "use a more specific title." It also includes specific advice on graph visualization, such as "the amount of data is too large, please add highlights."

[0651] Step 7: Emotion Recognition with the Emotion Engine

[0652] The server uses an emotion engine to recognize the user's emotional state. When the user checks the feedback, the server analyzes their facial expressions and tone of voice via a camera and microphone to determine their stress level and emotional state.

[0653] Step 8: Feedback adjustment based on emotional information

[0654] The server adjusts the content and presentation of the generated feedback based on the user's emotional information recognized by the emotion engine. For example, if the user is feeling stressed, the server will simplify the feedback. Conversely, if the user is calm, the server will provide detailed instructions.

[0655] Step 9: Provide feedback

[0656] The server sends the adjusted feedback to the terminal and provides it to the user, who can check the feedback on the terminal and understand the necessary corrections.

[0657] Step 10: Modifying the material

[0658] The user then modifies the material based on the feedback provided, and the modifications are performed on the user's device, updating the material according to the specific improvements.

[0659] Step 11: Re-upload and re-evaluate

[0660] The user then uploads the revised document back to the system, and the server repeats steps 2 to 9 for the newly uploaded document. This process is repeated until the document meets the supervisor's evaluation criteria.

[0661] Specific examples

[0662] Example 1: Improving the title

[0663] A user uploads a document called "Annual Sales Report." The server analyzes and evaluates it, generating feedback that the title is not specific enough. The emotion engine recognizes the user's stress level and offers a simple suggestion to revise it to "Annual Sales Report 2023 – Q1 Overview."

[0664] Example 2: Improving graph visualization

[0665] A user uploads a slide displaying sales data in the form of a bar graph. The server evaluates the slide and returns feedback that the data is too large and difficult to understand. If the emotion engine recognizes that the user is calm, it offers detailed instructions such as highlighting some of the data and adding a new sub-slide.

[0666] This series of processes enables users to efficiently create high-quality documents that meet their superiors' expectations. The system utilizes emotion recognition by an emotion engine and flexibly adjusts feedback according to the user's emotional state, providing effective support for document creation.

[0667] Example 2

[0668] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0669] The traditional document creation process has the problem that it is difficult for document creators to efficiently create documents that meet their superiors' evaluation criteria. Furthermore, the complicated process of making corrections after receiving feedback is often inefficient and time-consuming. Another problem is that the feedback content is uniform, and appropriate feedback is not provided based on the user's emotional state.

[0670] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for inputting data created by a material creator, a means for analyzing the data and extracting important elements, a means for making an evaluation based on the extracted elements, a means for generating feedback based on the evaluation results, a means for providing the feedback to the material creator, and a means for recognizing the user's emotions and adjusting the content of the feedback. This enables the material creator to efficiently create materials that meet the supervisor's evaluation criteria and provide appropriate feedback according to the user's emotional state.

[0671] A "document creator" is a person whose role is to create reports, presentation materials, etc.

[0672] "Data" is a collection of information including text, images, graphs, tables, etc. created by the document creator.

[0673] "Input means" refers to a method or device for providing data created by a document creator to the system.

[0674] "Means of analysis" are the methods and techniques that a system uses to process data, understand its contents, and extract important elements.

[0675] "Key elements" are the main information needed for evaluation and feedback, such as the title, headings, graphs, tables, and key points within the document.

[0676] "Means of evaluation" are methods or algorithms for determining and evaluating the quality of data based on the extracted key elements.

[0677] "Means for generating feedback" refers to methods and techniques for providing improvements and advice to document creators based on the evaluation results.

[0678] The "means for providing" refers to a method or device for conveying the generated feedback to the material creator.

[0679] "Means for recognizing emotions and adjusting the content of feedback" refers to methods or techniques for recognizing the user's current emotional state and adjusting the content and level of detail of the feedback accordingly.

[0680] "Natural language processing" is a technology that enables computers to understand and process human language.

[0681] "Image recognition technology" is a technology that allows a computer to analyze image data and recognize the information contained within it.

[0682] The system of the present invention allows document creators to efficiently create documents that meet their superiors' evaluation criteria. In particular, by combining this system with an emotion engine, it is possible to provide feedback that takes into account the user's emotional state.

[0683] The system consists of the following main elements:

[0684] 1. Input Method

[0685] Users use an interface to upload created document files (such as PowerPoint files or reports) from their terminals to the system. The user selects the target file in the file selection dialog and clicks the upload button, and the terminal sends the selected file to the server.

[0686] 2. Analysis Methods

[0687] The server uses natural language processing (NLP) and image recognition technologies to analyze the received data files. Specifically, it uses Apache Tika to analyze the file contents, NLTK (Natural Language Toolkit) to extract text elements, and OpenCV to recognize images and graphs.

[0688] 3. Means of extracting important elements

[0689] The server extracts important elements from the analyzed content (titles, headings, graphs, tables, key points, etc.) and records them in a database. For example, it connects to an H2 database and stores the extracted titles and graph metadata in the database.

[0690] 4. Evaluation methods

[0691] The server evaluates the documents based on the extracted key elements, using a Scikit-learn model to score them based on pre-trained evaluation criteria, including clarity, specificity, and design consistency.

[0692] 5. Means of generating feedback

[0693] The server generates specific feedback based on the evaluation results. It uses the Jinja template engine to create an HTML feedback page with suggestions for improvement. This feedback details what parts of the material should be improved and how.

[0694] 6. Emotional awareness and regulation tools

[0695] The server uses an emotion engine to recognize the user's emotions and adjusts the feedback accordingly. Specifically, it uses OpenCV and librosa to analyze the user's facial expressions and tone of voice to determine their emotional state. For example, if the user is stressed, the server will simplify the feedback, but if they are calm, it will provide more detailed instructions.

[0696] 7. Means of Providing Feedback

[0697] The server provides the generated feedback to the user, for example, when the user views the feedback in a web browser, detailed or simplified instructions are displayed based on the dynamic response from the server.

[0698] Specific examples

[0699] Example 1: Improving the title

[0700] If a user uploads a document titled "Annual Sales Report," the server will evaluate it as "not specific enough" and suggest a more specific title, "Annual Sales Report 2023 – Q1 Overview." If the emotion engine analyzes the user's facial expressions and recognizes that they are feeling stressed, the server will offer simple suggestions for improvement.

[0701] Example 2: Improving graph visualization

[0702] If a user uploads a slide showing sales data in the form of a bar graph, the server will determine that there is too much data and that it is difficult to understand. The emotion engine will then determine if the user is calm and provide detailed instructions on how to improve the slide, such as how to highlight the data or add new sub-slides.

[0703] Prompt Sentence Examples

[0704] "Please improve the title of this PowerPoint file to make it more specific and descriptive."

[0705] "How can I change the data in this bar chart to make it more visually understandable?"

[0706] As a result, document creators can create high-quality documents that satisfy their superiors in a short period of time. Each method and emotion engine of the system is designed to streamline the document creation process and reduce repetition of work.

[0707] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0708] System program processing flow

[0709] Step 1:

[0710] A document file created by a user is uploaded from the terminal to the system. The user selects a file using the system interface and clicks the upload button. The terminal sends the file selected by the user to the server. The input here is the document file selected by the user, and the output is the file received by the server. Specifically, a file selection dialog is displayed, the user selects a file, and it is sent to the server via an Ajax request using JavaScript.

[0711] Step 2:

[0712] The server receives the file sent from the device and temporarily stores it in its internal storage. The input is the file sent from the device, and the output is the temporarily stored file and its metadata. Specifically, the server receives a secure HTTP request, temporarily stores the file in RAM, and simultaneously records metadata such as the file name and upload date and time in a database.

[0713] Step 3:

[0714] The server analyzes the stored data files and decomposes their contents. It performs the analysis using natural language processing (NLP) and image recognition technologies. The input is the stored data file, and the output is the analyzed text elements and image data. Specifically, it uses Apache Tika to extract text from the file, NLTK to further decompose the text, and OpenCV to analyze images and graphs.

[0715] Step 4:

[0716] The server extracts important elements from the analyzed content. Important elements include titles, headings, graphs, tables, key points, etc. The input is the analyzed text elements and image data, and the output is the extracted important elements. Specifically, the extracted elements are stored in an H2 database, and metadata is also recorded.

[0717] Step 5:

[0718] The server evaluates the materials based on the extracted important elements. The evaluation uses pre-trained evaluation criteria and scores using a Scikit-learn model. The input is the extracted important elements, and the output is the evaluation score and comments. Specifically, the server automatically calculates the score based on the evaluation criteria and records it in a database.

[0719] Step 6:

[0720] The server generates specific feedback based on the evaluation results. The feedback includes areas for improvement and advice. The input is the evaluation score and comments, and the output is the generated feedback. Specifically, it uses the Jinja template engine to create an HTML feedback page.

[0721] Step 7:

[0722] The server uses an emotion engine to recognize the user's emotions and adjust the feedback content. The input is the user's facial expression and voice data, and the output is adjusted feedback. Specifically, OpenCV and librosa are used to analyze the user's facial expressions and tone of voice in real time, and the feedback content is adjusted according to the user's emotional state.

[0723] Step 8:

[0724] The server provides the generated feedback to the user, who then modifies the document based on the feedback. The input is the adjusted feedback, and the output is the document modified by the user. Specifically, the user views the feedback in a web browser, modifies the document, and saves it as a new file.

[0725] Step 9:

[0726] The user re-uploads the revised material to the system. The server then repeats the process described above, providing a new evaluation and feedback. The input is the revised material file, and the output is a re-evaluated score and new feedback. Specifically, the user re-uploads, and the server begins a new evaluation process.

[0727] Step 10:

[0728] This process is repeated until the document meets the supervisor's evaluation criteria. The input is a document file that is repeatedly revised and reevaluated, and the output is a final document that meets the supervisor's evaluation criteria. Specifically, the cycle of feedback and revision is repeated until a satisfactory document is completed.

[0729] (Application example 2)

[0730] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0731] In the past, when document creators tried to efficiently create documents that met their superiors' evaluation criteria, the supervisor's specific evaluation criteria and areas for improvement in the document were often not clearly presented, forcing the creator to make repeated revisions. Furthermore, feedback that took into account the creator's emotional state was not provided, which increased work stress. This resulted in an inefficient document creation process and made it difficult to create high-quality documents in a short period of time.

[0732] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0733] In this invention, the server includes means for inputting data created by a material creator, means for analyzing the data and extracting important elements, means for evaluating based on the extracted elements, means for generating feedback based on the evaluation results, means for analyzing the user's emotional state using an emotion engine, means for adjusting the feedback according to the emotional state, and means for providing the feedback to the material creator. This not only enables the material creator to efficiently create high-quality materials that meet the supervisor's evaluation criteria, but also reduces work stress and provides more effective support by providing feedback that takes into account the material creator's emotional state.

[0734] "Document Creator" refers to the individual or organization that creates the document.

[0735] "Data" refers to the entire content, including information such as text, graphs, and images created by the document creator.

[0736] "Important elements" are particularly important parts of the material extracted through analysis, including titles, headings, graphs, tables, key points, etc.

[0737] "Evaluation" refers to the process of judging the quality of material based on extracted key elements.

[0738] "Feedback" refers to specific improvement suggestions and advice provided to the document creator based on the evaluation results.

[0739] An "emotion engine" refers to software or hardware that analyzes a user's facial expressions and tone of voice to determine their current emotional state.

[0740] "Emotional situation" refers to the user's current emotional state, such as stress level or level of understanding.

[0741] "Adjustment" refers to the process of changing the content and format of feedback depending on the emotional situation analyzed by the emotion engine.

[0742] The "document creation process" refers to the series of steps from creating a document to evaluation, feedback, and revision.

[0743] The system for realizing this invention is for a document creator to efficiently create documents that conform to the supervisor's evaluation criteria, and includes the following processes.

[0744] First, the user uploads the documents (e.g., PowerPoint files or reports) they have created from their terminal to the system. The user selects the files using the system interface and clicks the upload button. The terminal then sends the selected files to the server.

[0745] Next, the server receives the document file sent by the user and stores it in internal temporary storage. The stored file is then subjected to analysis. Natural language processing (NLP) and image recognition technologies are used for the analysis to extract important elements such as text, graphs, and images. NLP technologies such as "SpaCy" and "BERT" are used, while image recognition technologies such as "OpenCV" and "TensorFlow" can be used.

[0746] The server then extracts important elements (such as titles, headings, graphs, tables, and key points) from the analyzed content, and records each important element in a database.The server then evaluates the entire document based on the extracted important elements and in accordance with the supervisor's evaluation criteria, which it has learned in advance.Evaluation criteria include clarity, specificity, and design consistency.

[0747] Based on the evaluation results, the server generates specific feedback that details which parts of the material should be improved and how. For example, it provides specific advice for improving the text or instructions for rearranging graphs. The text generation model used in this process is a generative AI model such as GPT-3.

[0748] Furthermore, the server uses an emotion engine to analyze the user's emotional state. The emotion engine analyzes the user's facial expressions and tone of voice to determine their current emotional state. The emotion engine uses tools such as "Affectiva" and "Microsoft Azure Emotion API." This allows the server to determine whether the user is feeling stressed or has a high level of understanding.

[0749] When providing feedback, the system takes into account the results of the emotion engine and provides the generated feedback to the user. For example, if the user is feeling stressed, the system will simplify the feedback content, and conversely, if the user is calm, it will provide detailed instructions. In this way, feedback appropriate to the emotional state of the document creator is provided.

[0750] As a concrete example, consider the case where a user uploads a document titled "Annual Sales Report." Based on the server's evaluation criteria, the server returns feedback that the title is not specific. If the emotion engine analyzes the user's facial expression and recognizes that the user is stressed, it offers a simple suggestion for improvement. For example, it suggests revising the title to "Annual Sales Report 2023 – Q1 Overview."

[0751] Examples of prompts include:

[0752] "Please suggest ways to make the document titles concise and specific."

[0753] "What are some best practices for rearranging graphs for better visibility?"

[0754] In this way, the system can support the document creator in efficiently creating high-quality documents that meet the supervisor's evaluation criteria.

[0755] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0756] Step 1:

[0757] Upload user-created materials.

[0758] Input: User selection of file (e.g. PowerPoint or report file)

[0759] Specific operation: The user selects the document file through the system interface and clicks the upload button.

[0760] Output: The selected files are sent to the server.

[0761] Step 2:

[0762] The server receives the document file and stores it in temporary storage.

[0763] Input: Data file sent from the user's terminal

[0764] Specific operation: The server receives the sent file and stores it in its internal temporary storage.

[0765] Output: The saved file is the target for analysis processing.

[0766] Step 3:

[0767] The server analyzes the document files and extracts important elements.

[0768] Input: Data file saved in temporary storage

[0769] Specific operation: The server uses natural language processing (NLP) techniques (e.g., SpaCy and BERT) and image recognition techniques (e.g., OpenCV and TensorFlow) to extract important elements such as text, graphs, and images.

[0770] Data processing / calculation: Analysis of text and image data in documents

[0771] Output: The extracted important elements (titles, headings, graphs, tables, key points, etc.) are recorded in a database.

[0772] Step 4:

[0773] The server evaluates the entire document based on the extracted important elements.

[0774] Input: Key elements recorded in the database

[0775] Specific behavior: The server evaluates the clarity, specificity, and design consistency based on the supervisor's evaluation criteria that it has learned in advance.

[0776] Data processing / calculation: Assessing the quality of the material based on key factors

[0777] Output: The evaluation result is generated.

[0778] Step 5:

[0779] The server generates specific feedback based on the evaluation results.

[0780] Input: Evaluation result

[0781] Specific operation: The server uses a generative AI model (e.g., "GPT-3") to generate specific improvement suggestions for the material.

[0782] Data processing / calculation: Text generation based on evaluation results

[0783] Output: Specific feedback is generated.

[0784] Step 6:

[0785] The server uses an emotion engine to analyze the user's emotional state.

[0786] Input: User's facial expression data and tone of voice data

[0787] Specific operation: The server performs analysis using an emotion engine (e.g., "Affectiva" or "Microsoft Azure Emotion API").

[0788] Data processing / calculation: Analysis of user facial expressions and tone of voice

[0789] Output: The user's emotional state is determined.

[0790] Step 7:

[0791] The server adjusts the feedback depending on the emotional situation.

[0792] Input: Generated feedback and the user's emotional state

[0793] Specific behavior: The server simplifies the feedback if the user is stressed, and provides detailed instructions if the user is calm.

[0794] Data processing / calculation: Adjusting the content and format of feedback

[0795] Output: Feedback appropriate to the user's emotions is generated.

[0796] Step 8:

[0797] The server provides feedback to the user.

[0798] Input: Feedback appropriate to the user's emotions

[0799] Specific Actions: The server provides the generated feedback to the user.

[0800] Output: The feedback the user receives is displayed on the screen.

[0801] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0802] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0803] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0804] [Third embodiment]

[0805] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0806] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0807] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0808] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0809] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0810] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0811] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0812] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0813] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0814] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0815] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0816] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0817] The system of the present invention aims to enable a document creator to quickly and efficiently create documents that meet the supervisor's evaluation criteria. To achieve this aim, the system includes the following means.

[0818] 1. Uploading materials

[0819] The user uploads the materials (presentation files and reports) they have created to the system from their terminal. The user selects the material file using the system interface and provides the data by clicking the upload button.

[0820] 2. Analysis of the data

[0821] The server analyzes the received data using natural language processing (NLP) and image recognition technologies to extract important elements such as text, graphs, and images. This analysis provides a detailed understanding of the data's contents.

[0822] 3. Evaluation of materials

[0823] The server evaluates the document based on the extracted key elements. The evaluation is performed using an AI model that reflects the supervisor's evaluation criteria. The model judges the quality of the document from aspects such as clarity, specificity, and design.

[0824] 4. Generate feedback

[0825] The server generates specific feedback based on the evaluation results. The generated feedback details which parts of the material should be improved and how. The feedback includes specific correction instructions such as improving text, rearranging graphs, adding images, etc.

[0826] 5. Providing Feedback

[0827] The server provides the generated feedback to the terminal, and the user can see which parts of the material need to be revised through the feedback displayed on the terminal screen.

[0828] 6. Modification of Materials

[0829] The user modifies the document based on the provided feedback. The user then refers to the feedback and changes the content of the document to make it meet the manager's evaluation criteria.

[0830] 7. Re-upload and Re-rating

[0831] The user then uploads the revised document back to the system. The server then repeats the process, evaluating the document and providing new feedback. This process continues until the document meets the supervisor's evaluation criteria.

[0832] Specific examples

[0833] Example 1: Improving the title

[0834] A user uploads a document called "Annual Sales Report." Based on the server's evaluation criteria, the server returns feedback that the title is not specific. The user then corrects the title to "Annual Sales Report 2023 – Q1 Overview" and re-uploads the document.

[0835] Example 2: Improving graph visualization

[0836] A user uploads a slide showing sales data in the form of a bar graph. The server returns feedback that "there is too much data and it's difficult to understand." The user highlights some of the data and adds a sub-slide based on that. The revised slide is then uploaded again, and the next feedback is received.

[0837] These procedures enable document creators to create high-quality documents that satisfy their superiors in a short period of time. Each method in the system is designed to streamline the document creation process and reduce repetition of work.

[0838] The processing flow will be explained below.

[0839] Step 1: Upload your data

[0840] Users upload PowerPoint presentations or reports created on their own devices to the system as files. Users select the files from the system interface and click the upload button. The device then sends the selected files to the server.

[0841] Step 2: Receiving and storing materials

[0842] The server receives the document file sent by the user and stores the received file in temporary storage within the server.

[0843] Step 3: Analyze the material

[0844] The server analyzes the stored files and breaks down the content of the documents, using natural language processing (NLP) technology to analyze the text content and image recognition technology to extract visual elements such as images and graphs.

[0845] Step 4: Extracting important elements

[0846] The server extracts important elements from the document (e.g., title, headings, graphs, tables, key points, etc.) from the analyzed content. Each element is recorded in the database as an independent element.

[0847] Step 5: Evaluate the material

[0848] The server evaluates the entire document based on the extracted important elements, using specific indicators (e.g., clarity, specificity, and design consistency) based on the supervisor's evaluation criteria that it has learned in advance.

[0849] Step 6: Generate feedback

[0850] The server generates feedback based on the evaluation results, creating comments that point out areas for improvement or missing elements, and presents them in a format that is easy for the user to understand.

[0851] Step 7: Provide feedback

[0852] The server sends the generated feedback to the user's device, where the user can view the feedback on the device screen.

[0853] Step 8: Modify the material

[0854] Users can then modify their own documents based on the feedback they receive from the server, for example by making the title more specific or improving the layout of the graphs.

[0855] Step 9: Re-upload

[0856] The user then re-uploads the revised document to the system. The re-uploading procedure is the same as in step 1.

[0857] Step 10: Reassess and feedback iteratively

[0858] The server then repeats steps 2 to 7 for the newly uploaded materials. This process is repeated until the materials meet the supervisor's evaluation criteria. The user can continue to receive feedback and make revisions as many times as they like.

[0859] This series of steps enables the document creator to efficiently create documents that will satisfy their superiors.

[0860] Example 1

[0861] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0862] The goal of this project is to solve the problem of time-consuming and labor-intensive manual corrections and repetitive tasks that are required when document creators create documents that meet their superiors' evaluation criteria efficiently and quickly. Specifically, a system is needed that supports document creators in making effective corrections without hesitation to meet evaluation criteria such as whether the content of the document is specific, easy to understand, and well-designed.

[0863] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0864] In this invention, the server includes means for inputting data created by a document creator, means for analyzing the data and extracting important elements, means for making an evaluation based on the extracted elements, means for generating feedback based on the evaluation results, means for providing the feedback to the document creator, means for the document creator to correct the data based on the generated feedback, means for re-inputting the corrected data, and means for re-analyzing and re-evaluating the corrected data. This allows the document creator to make corrections quickly and accurately based on the feedback provided by the system, making it possible to efficiently create high-quality documents that meet the evaluation criteria of their superiors.

[0865] A "material creator" is a user who creates a material and uploads it to the system.

[0866] "Data" refers to document files such as presentation files and reports created by document creators.

[0867] "Means of input" refers to the interface that allows document creators to upload data into the system.

[0868] "Means of analysis" refers to the process by which the server analyzes the data received using natural language processing and image recognition technology to extract important elements.

[0869] "Important elements" refer to items necessary for evaluating materials such as text, graphs, and images in the data.

[0870] "Means of evaluation" refers to an AI model that evaluates the quality of materials based on extracted elements.

[0871] "Means of generating feedback" refers to the process of creating specific corrective instructions based on the evaluation results.

[0872] "Means for providing" refers to an interface for notifying the material creator of the generated feedback and displaying it.

[0873] "Measures to revise" refers to actions taken by the material creator to revise the material based on the feedback provided.

[0874] "Means for re-entering" refers to an interface for uploading corrected materials back into the system.

[0875] "Reanalysis and reevaluation" refers to the process of reanalyzing and evaluating the corrected data.

[0876] "Generative AI model" refers to an artificial intelligence model used to evaluate materials and generate feedback.

[0877] A "prompt" refers to a specific question or instruction that feeds data into a generative AI model.

[0878] The present invention relates to a system that enables document creators to efficiently and quickly create documents that meet their supervisor's evaluation criteria. To achieve this goal, the system includes a series of means, specifically, procedures for uploading documents, analyzing, evaluating, generating feedback, providing feedback, correcting, and reevaluating.

[0879] Hardware and Software

[0880] The hardware that realizes this system includes a server and user terminals. The server is equipped with a high-performance processor and large-capacity memory, and by incorporating a GPU in particular, the processing power of the AI ​​model can be improved. Terminals can be general-purpose personal computers or tablet terminals.

[0881] The following software is used:

[0882] Natural Language Processing (NLP) libraries: NLTK (Natural Language Toolkit), SpaCy

[0883] Image recognition library: OpenCV, TensorFlow

[0884] AI model frameworks: TensorFlow, PyTorch

[0885] Uploading materials

[0886] The user opens the system interface using a terminal and uploads the created document. This operation is completed by clicking the "Select File" button on the simple interface, selecting the document file, and then clicking the "Upload" button. The terminal sends the document file to the server using an HTTP request.

[0887] Analysis of data

[0888] The server uses natural language processing (NLP) and image recognition technologies to analyze the documents received. Specifically, the server uses NLTK and SpaCy to analyze the text, and OpenCV and TensorFlow to analyze graphs and images. This allows the server to extract important elements from the documents.

[0889] Evaluation of materials

[0890] The server evaluates the documents based on the analyzed data. This evaluation uses a generative AI model that reflects the supervisor's evaluation criteria. The AI ​​model used is a generative AI model (e.g., GPT-3), which evaluates the documents' clarity, specificity, design, etc.

[0891] Generate feedback

[0892] The server generates specific feedback based on the evaluation results. The generated feedback includes specific instructions on how to improve the material, such as improving the text, rearranging graphs, or adding images. The server uses a generative AI model to generate the feedback in natural language.

[0893] Providing feedback

[0894] The generated feedback is sent from the server to the device, where the user can check it. The feedback specifically indicates which slides in the document need to be revised and what kind of revisions are required. The user can review this and begin revising the document.

[0895] Corrections to the material

[0896] The user then modifies the document based on the provided feedback, such as by modifying the text, rearranging the graphs, and adding new images as needed, according to the feedback instructions. After modifying the document, the user then saves the modified document.

[0897] Re-upload and re-evaluation

[0898] The user then uploads the revised material back into the system, after which the server repeats the analysis and evaluation process described above and provides new feedback. This process is repeated until the material meets the supervisor's evaluation criteria.

[0899] Examples of concrete examples and prompts

[0900] Example 1: Improving the title

[0901] A user uploads a document called "Annual Sales Report." Based on the server's evaluation criteria, the server returns feedback that the title is not specific. The user corrects the title to "Annual Sales Report 2023 – Q1 Overview" and re-uploads it. Example prompt: "How can you make the title of this presentation more specific?"

[0902] Example 2: Improving graph visualization

[0903] A user uploads a slide showing sales data in the form of a bar graph. The server provides feedback that "there is too much data and it's difficult to understand." The user highlights some of the data and adds a sub-slide based on that. The revised slide is then uploaded again and the following feedback is received: Example prompt: "How can you modify this graph to make it easier to read?"

[0904] As described above, this system provides a series of functions to streamline the document creation process and is designed to enable document creators to quickly create high-quality documents.

[0905] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0906] Step 1: Upload your materials

[0907] The user opens the system interface using a terminal and selects the created document. The user clicks the "Select File" button, selects the document file, and then clicks the "Upload" button. The terminal sends the document file to the server using an HTTP request.

[0908] Input: User-created material file

[0909] Output: The document file sent to the server

[0910] Specific operation: When the user clicks the "Upload" button, the device sends the document file to the server. The file is sent as an attachment to the HTTP request.

[0911] Step 2: Analyze the data

[0912] The server analyzes the received file. To analyze, it first reads the file contents and separates the text from the images. It then uses natural language processing (NLP) technology to analyze the text portion, and image recognition technology to analyze the graphs and images.

[0913] Input: Data file sent to the server

[0914] Output: Extracted important elements (text data, image data)

[0915] Specific operation: The server uses NLTK and SpaCy to analyze the text and identify keywords and sentence structure, and OpenCV and TensorFlow to extract elements from graphs and images and obtain features.

[0916] Step 3: Evaluate the material

[0917] The server evaluates the document based on the analyzed data. This evaluation uses a generative AI model that reflects the supervisor's evaluation criteria. The generative AI model receives the analyzed data as input and evaluates the document based on that.

[0918] Input: Extracted important elements (text data, image data)

[0919] Output: Evaluation results (quality score and evaluation comments)

[0920] Specific operation: The server inputs the analysis results into the generative AI model and generates an evaluation result. The server compares the results with the evaluation criteria to assign a score and generate evaluation comments.

[0921] Step 4: Generate feedback

[0922] The server generates specific feedback based on the evaluation results, including specific instructions on how to improve the material.

[0923] Input: Evaluation results (quality score and evaluation comments)

[0924] Output: Generated feedback (correction instructions)

[0925] Specific behavior: Based on the evaluation results, the server uses a generative AI model to generate feedback, which includes specific correction instructions such as improving text, rearranging graphs, and adding images.

[0926] Step 5: Provide feedback

[0927] The server generates feedback and sends it to the device. The user can view the feedback on the device. The server formats the feedback data into structured data such as JSON format and sends it to the device.

[0928] Input: Generated feedback (correction instructions)

[0929] Output: Feedback provided to the terminal

[0930] Specific operation: The server sends feedback data to the terminal, and the terminal displays the received feedback on the screen and notifies the user.

[0931] Step 6: Modify the material

[0932] The user modifies the material based on the feedback provided, specifically by modifying the text, rearranging charts, or adding new images as directed by the feedback.

[0933] Input: Feedback provided

[0934] Output: Modified documentation file

[0935] Specific operation: The user makes corrections using document editing software (e.g., Microsoft PowerPoint, Google Slides) and saves the corrected document.

[0936] Step 7: Re-upload and re-evaluate

[0937] The user uploads the revised material back into the system, and the server receives it again, performs the analysis and evaluation process described above, and provides new feedback.

[0938] Input: Modified document file

[0939] Output: New evaluation results and new feedback

[0940] How it works: The user re-uploads the revised document, and the server analyzes and evaluates it again, generating and providing new feedback. This process is repeated until the document meets the supervisor's evaluation criteria.

[0941] (Application example 1)

[0942] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0943] Existing document creation support systems are primarily used in office environments, making it difficult to support document creation and reporting in the field. Furthermore, there was a lack of means to quickly analyze data acquired in the field and provide feedback, making it difficult for document creators to efficiently create high-quality documents. To solve these issues, a system that can seamlessly acquire and analyze data in the field and provide feedback is needed.

[0944] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0945] In this invention, the server includes means for inputting data created by a document creator, means for analyzing the data and extracting important elements, means for evaluating the data based on the extracted elements, means for generating feedback based on the evaluation results, means for providing the feedback to the document creator, means for using images and videos captured on-site as input, means for analyzing the data using natural language processing, image recognition technology, and OCR, and means for displaying the generated feedback on a smart device display. This makes it possible to instantly acquire and analyze data and provide feedback when creating documents on-site. As a result, document creators can quickly create high-quality documents that reflect the situation on-site.

[0946] A "document creator" is a person or organization that creates a presentation or report.

[0947] "Data" refers to a collection of information such as presentation files and reports created by document creators.

[0948] "Uploading" is the act of sending materials from a terminal to a server.

[0949] "Analysis" is the process of carefully examining and evaluating data to extract important elements.

[0950] "Key elements" are items of text, graphs, images, etc. that deserve special attention in the content of the document.

[0951] "Evaluation" is the act of judging the quality of material based on extracted important elements.

[0952] "Feedback" refers to specific comments based on the evaluation results, including suggestions for improvements and corrections to the materials.

[0953] The "site" is a place where actual work is carried out, such as a factory or a production line.

[0954] A "smart device" is a terminal that can connect to the Internet, such as smart glasses or a head-mounted display.

[0955] "OCR" stands for optical character recognition, a technology that extracts text from an image as digital data.

[0956] "Natural language processing" is a technology that allows computers to understand and analyze human language.

[0957] "Image recognition technology" is a technology that allows a computer to analyze images and understand their content.

[0958] A "display" is the display screen of a smart device, and is a device for visually checking information.

[0959] The system for realizing this invention includes the following programs. First, the document creator uses smart glasses on-site to capture images and videos. This device can be Google Glass or Microsoft HoloLens. The captured data is uploaded from the smart glasses to a server. The server receives this data and begins analysis.

[0960] This analysis process uses natural language processing (NLP), image recognition technology, and OCR technology. NLP is used to analyze the text information in the document and extract important elements. This includes understanding the content of the text and extracting keywords. Image recognition technology recognizes specific objects and situations from the image and extracts them as part of the document. OCR technology digitizes the text in the image so that it can be processed as data.

[0961] The server uses these technologies to analyze the data and evaluate the materials based on the extracted key elements. This evaluation is performed using a generative AI model that reflects the supervisor's evaluation criteria. This AI model evaluates the materials' clarity, specificity, design, and other criteria. Once the evaluation is complete, the server generates specific feedback.

[0962] The generated feedback is instantly displayed on the smart glasses' display, allowing workers to review the feedback and make on-site corrections to the materials. This feedback includes specific instructions such as suggestions for improving text, rearranging graphs, or adding images.

[0963] For example, if a worker discovers an abnormality on a production line and takes a photo of the abnormal area with smart glasses, the image is uploaded to the server. The server analyzes the image and evaluates the details of the abnormality. Feedback such as "Add a photo of the abnormality to the report" is then generated, allowing the worker to make corrections. As a result, high-quality documents that reflect the situation on-site are quickly created.

[0964] Examples of prompts for generative AI models include:

[0965] "This image shows an anomaly on the production line. Please use the system to analyze the content and generate feedback to create a report on the anomaly."

[0966] This invention allows document creators to instantly obtain documents on-site and analyze, evaluate, and provide feedback, making it possible to quickly create high-quality documents.

[0967] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0968] Step 1:

[0969] Users use smart glasses to capture images and videos needed for document creation on-site. Devices such as Google Glass and Microsoft HoloLens can be used for this purpose. The captured data is stored in the smart glasses' internal storage.

[0970] Input: Images and videos from the scene

[0971] Output: Data stored in the smart glasses storage

[0972] Step 2:

[0973] The user operates the smart glasses through an interface and uploads captured images and videos to a server, which is the process of transmitting data from the device to the server.

[0974] Input: Data stored in the smart glasses storage

[0975] Output: Data uploaded to the server

[0976] Step 3:

[0977] The server receives the uploaded data and begins analyzing it, first using OCR technology to extract text from the image, then using natural language processing (NLP) and image recognition technology to extract key elements from the image or video.

[0978] Input: Data uploaded to the server

[0979] Output: Extracted text data and image recognition results

[0980] Step 4:

[0981] The server evaluates the document based on the extracted key elements. This evaluation uses a generative AI model that reflects the supervisor's evaluation criteria. The model evaluates the document's clarity, specificity, design, and other criteria.

[0982] Input: Extracted text data and image recognition results

[0983] Output: Evaluation results

[0984] Step 5:

[0985] The server generates specific feedback based on the evaluation results, detailing how and where the material should be improved, including suggestions for improving text, rearranging graphs, adding images, and other specific corrections.

[0986] Input: Evaluation result

[0987] Output: Feedback

[0988] Step 6:

[0989] The server displays the generated feedback on the smart glasses display, allowing workers to check the feedback in real time and make corrections to the materials on-site.

[0990] Input: Feedback

[0991] Output: Feedback displayed on the smart glasses display

[0992] Step 7:

[0993] The user modifies the data based on the feedback provided through the smart glasses, and if necessary, re-acquires on-site data using the smart glasses and uploads the newly modified data to the server.

[0994] Input: Corrective instructions based on feedback

[0995] Output: Corrected material

[0996] By repeating this process, the document is adjusted until it meets the supervisor's evaluation criteria, allowing document creators to quickly create high-quality documents that reflect the situation on-site.

[0997] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0998] The system of the present invention aims to improve the process by which document creators can efficiently create documents that meet their superiors' evaluation criteria. In particular, by combining it with an emotion engine, it provides feedback that takes into account the user's emotional state.

[0999] 1. Uploading materials

[1000] The user uploads the documents (e.g., PowerPoint files or reports) they have created from their terminal to the system. The user selects the files using the system interface, clicks the upload button, and the terminal sends the selected files to the server.

[1001] 2. Receipt and storage of materials

[1002] The server receives the data file sent by the user and stores it in its internal temporary storage. The stored file is then subjected to analysis processing.

[1003] 3. Analysis of the data

[1004] The server analyzes the stored data files and breaks down their contents, using natural language processing (NLP) and image recognition technology to extract important elements such as text, graphs, and images.

[1005] 4. Extracting important elements

[1006] The server extracts important elements from the analyzed content (e.g., titles, headings, graphs, tables, key points, etc.), and records each important element in the database.

[1007] 5. Evaluation of materials

[1008] The server then evaluates the entire document based on the extracted important elements and in accordance with the supervisor's pre-trained evaluation criteria, including clarity, specificity, and design consistency.

[1009] 6. Generate feedback

[1010] The server generates specific feedback based on the evaluation results, detailing how and where the material should be improved, for example providing specific advice on improving text or instructions on rearranging graphs.

[1011] 7. Leveraging Emotional Engines

[1012] The server utilizes an emotion engine to recognize the user's emotions. The emotion engine analyzes the user's facial expressions and tone of voice to determine their current emotional state. This allows the server to determine whether the user is feeling stressed or has a high level of understanding.

[1013] 8. Providing Feedback

[1014] The server considers the results of the emotion engine and provides the generated feedback to the user. For example, if the user is stressed, the server will simplify the feedback, and if the user is calm, it will provide detailed instructions.

[1015] 9. Modification of Materials

[1016] The user then modifies the document based on the feedback received from the server. The modification work is carried out on the user's terminal, and the content of the document is changed while referring to the feedback.

[1017] 10. Re-uploading and Re-rating

[1018] The user then uploads the revised document back to the system. The server then repeats the process described above (steps 2 to 9), reassessing the document and providing new feedback. This process continues until the document meets the supervisor's evaluation criteria.

[1019] Specific examples

[1020] Example 1: Improving the title

[1021] A user uploads a document called "Annual Sales Report." Based on the server's evaluation criteria, the server returns feedback that the title is not specific. If the emotion engine analyzes the user's facial expression and recognizes that the user is stressed, it offers a simple suggestion for improvement. For example, it suggests revising the title to "Annual Sales Report 2023 – Q1 Overview."

[1022] Example 2: Improving graph visualization

[1023] A user uploads a slide showing sales data in the form of a bar graph. When the server responds with feedback that "there's too much data and it's difficult to understand," the emotion engine determines from the user's tone of voice that they are calm and provides detailed instructions for improvement, such as suggesting ways to highlight parts of the data or adding new sub-slides.

[1024] These methods enable document creators to create high-quality documents that satisfy their superiors in a short period of time. Each method and emotion engine of the system is designed to streamline the document creation process and reduce repetition of work.

[1025] The processing flow will be explained below.

[1026] Step 1: Upload your materials

[1027] Users select PowerPoint presentations or report files they have created on their own devices, and then click the upload button on the device interface to send the files to the system.

[1028] Step 2: Receive and save the file

[1029] The server receives the uploaded file and stores it in its internal storage. This process allows the server to proceed to the next step without losing any data.

[1030] Step 3: Analyze the material

[1031] The server analyzes the stored data files, extracting text content using natural language processing (NLP) technology and analyzing visual elements such as graphs and images using image recognition technology. This analysis allows the data content to be structured.

[1032] Step 4: Extracting important elements

[1033] The server extracts important elements from the analyzed data, such as titles, headings, graphs, tables, and key points, and records each extracted element in a database for use in subsequent evaluation processes.

[1034] Step 5: Evaluate the material

[1035] The server evaluates the document based on the extracted important elements. Based on the supervisor's evaluation criteria, which it has learned in advance, the server evaluates the entire document in terms of clarity, specificity, design consistency, etc. The results of this evaluation form the basis for generating feedback.

[1036] Step 6: Generate feedback

[1037] The server generates specific feedback based on the evaluation results. For example, if the title is not specific enough, it will generate an instruction to "use a more specific title." It also includes specific advice on graph visualization, such as "the amount of data is too large, please add highlights."

[1038] Step 7: Emotion Recognition with the Emotion Engine

[1039] The server uses an emotion engine to recognize the user's emotional state. When the user checks the feedback, the server analyzes their facial expressions and tone of voice via a camera and microphone to determine their stress level and emotional state.

[1040] Step 8: Feedback adjustment based on emotional information

[1041] The server adjusts the content and presentation of the generated feedback based on the user's emotional information recognized by the emotion engine. For example, if the user is feeling stressed, the server will simplify the feedback. Conversely, if the user is calm, the server will provide detailed instructions.

[1042] Step 9: Provide feedback

[1043] The server sends the adjusted feedback to the terminal and provides it to the user, who can check the feedback on the terminal and understand the necessary corrections.

[1044] Step 10: Modifying the material

[1045] The user then modifies the material based on the feedback provided, and the modifications are performed on the user's device, updating the material according to the specific improvements.

[1046] Step 11: Re-upload and re-evaluate

[1047] The user then uploads the revised document back to the system, and the server repeats steps 2 to 9 for the newly uploaded document. This process is repeated until the document meets the supervisor's evaluation criteria.

[1048] Specific examples

[1049] Example 1: Improving the title

[1050] A user uploads a document called "Annual Sales Report." The server analyzes and evaluates it, generating feedback that the title is not specific enough. The emotion engine recognizes the user's stress level and offers a simple suggestion to revise it to "Annual Sales Report 2023 – Q1 Overview."

[1051] Example 2: Improving graph visualization

[1052] A user uploads a slide displaying sales data in the form of a bar graph. The server evaluates the slide and returns feedback that the data is too large and difficult to understand. If the emotion engine recognizes that the user is calm, it offers detailed instructions such as highlighting some of the data and adding a new sub-slide.

[1053] This series of processes enables users to efficiently create high-quality documents that meet their superiors' expectations. The system utilizes emotion recognition by an emotion engine and flexibly adjusts feedback according to the user's emotional state, providing effective support for document creation.

[1054] Example 2

[1055] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1056] The traditional document creation process has the problem that it is difficult for document creators to efficiently create documents that meet their superiors' evaluation criteria. Furthermore, the complicated process of making corrections after receiving feedback is often inefficient and time-consuming. Another problem is that the feedback content is uniform, and appropriate feedback is not provided based on the user's emotional state.

[1057] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for inputting data created by a material creator, a means for analyzing the data and extracting important elements, a means for making an evaluation based on the extracted elements, a means for generating feedback based on the evaluation results, a means for providing the feedback to the material creator, and a means for recognizing the user's emotions and adjusting the content of the feedback. This enables the material creator to efficiently create materials that meet the supervisor's evaluation criteria and provide appropriate feedback according to the user's emotional state.

[1058] A "document creator" is a person whose role is to create reports, presentation materials, etc.

[1059] "Data" is a collection of information including text, images, graphs, tables, etc. created by the document creator.

[1060] "Input means" refers to a method or device for providing data created by a document creator to the system.

[1061] "Means of analysis" are the methods and techniques that a system uses to process data, understand its contents, and extract important elements.

[1062] "Key elements" are the main information needed for evaluation and feedback, such as the title, headings, graphs, tables, and key points within the document.

[1063] "Means of evaluation" are methods or algorithms for determining and evaluating the quality of data based on the extracted key elements.

[1064] "Means for generating feedback" refers to methods and techniques for providing improvements and advice to document creators based on the evaluation results.

[1065] The "means for providing" refers to a method or device for conveying the generated feedback to the material creator.

[1066] "Means for recognizing emotions and adjusting the content of feedback" refers to methods or techniques for recognizing the user's current emotional state and adjusting the content and level of detail of the feedback accordingly.

[1067] "Natural language processing" is a technology that enables computers to understand and process human language.

[1068] "Image recognition technology" is a technology that allows a computer to analyze image data and recognize the information contained within it.

[1069] The system of the present invention allows document creators to efficiently create documents that meet their superiors' evaluation criteria. In particular, by combining this system with an emotion engine, it is possible to provide feedback that takes into account the user's emotional state.

[1070] The system consists of the following main elements:

[1071] 1. Input Method

[1072] Users use an interface to upload created document files (such as PowerPoint files or reports) from their terminals to the system. The user selects the target file in the file selection dialog and clicks the upload button, and the terminal sends the selected file to the server.

[1073] 2. Analysis Methods

[1074] The server uses natural language processing (NLP) and image recognition technologies to analyze the received data files. Specifically, it uses Apache Tika to analyze the file contents, NLTK (Natural Language Toolkit) to extract text elements, and OpenCV to recognize images and graphs.

[1075] 3. Means of extracting important elements

[1076] The server extracts important elements from the analyzed content (titles, headings, graphs, tables, key points, etc.) and records them in a database. For example, it connects to an H2 database and stores the extracted titles and graph metadata in the database.

[1077] 4. Evaluation methods

[1078] The server evaluates the documents based on the extracted key elements, using a Scikit-learn model to score them based on pre-trained evaluation criteria, including clarity, specificity, and design consistency.

[1079] 5. Means of generating feedback

[1080] The server generates specific feedback based on the evaluation results. It uses the Jinja template engine to create an HTML feedback page with suggestions for improvement. This feedback details what parts of the material should be improved and how.

[1081] 6. Emotional awareness and regulation tools

[1082] The server uses an emotion engine to recognize the user's emotions and adjusts the feedback accordingly. Specifically, it uses OpenCV and librosa to analyze the user's facial expressions and tone of voice to determine their emotional state. For example, if the user is stressed, the server will simplify the feedback, but if they are calm, it will provide more detailed instructions.

[1083] 7. Means of Providing Feedback

[1084] The server provides the generated feedback to the user, for example, when the user views the feedback in a web browser, detailed or simplified instructions are displayed based on the dynamic response from the server.

[1085] Specific examples

[1086] Example 1: Improving the title

[1087] If a user uploads a document titled "Annual Sales Report," the server will evaluate it as "not specific enough" and suggest a more specific title, "Annual Sales Report 2023 – Q1 Overview." If the emotion engine analyzes the user's facial expressions and recognizes that they are feeling stressed, the server will offer simple suggestions for improvement.

[1088] Example 2: Improving graph visualization

[1089] If a user uploads a slide showing sales data in the form of a bar graph, the server will determine that there is too much data and that it is difficult to understand. The emotion engine will then determine if the user is calm and provide detailed instructions on how to improve the slide, such as how to highlight the data or add new sub-slides.

[1090] Prompt Sentence Examples

[1091] "Please improve the title of this PowerPoint file to make it more specific and descriptive."

[1092] "How can I change the data in this bar chart to make it more visually understandable?"

[1093] As a result, document creators can create high-quality documents that satisfy their superiors in a short period of time. Each method and emotion engine of the system is designed to streamline the document creation process and reduce repetition of work.

[1094] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1095] System program processing flow

[1096] Step 1:

[1097] A document file created by a user is uploaded from the terminal to the system. The user selects a file using the system interface and clicks the upload button. The terminal sends the file selected by the user to the server. The input here is the document file selected by the user, and the output is the file received by the server. Specifically, a file selection dialog is displayed, the user selects a file, and it is sent to the server via an Ajax request using JavaScript.

[1098] Step 2:

[1099] The server receives the file sent from the device and temporarily stores it in its internal storage. The input is the file sent from the device, and the output is the temporarily stored file and its metadata. Specifically, the server receives a secure HTTP request, temporarily stores the file in RAM, and simultaneously records metadata such as the file name and upload date and time in a database.

[1100] Step 3:

[1101] The server analyzes the stored data files and decomposes their contents. It performs the analysis using natural language processing (NLP) and image recognition technologies. The input is the stored data file, and the output is the analyzed text elements and image data. Specifically, it uses Apache Tika to extract text from the file, NLTK to further decompose the text, and OpenCV to analyze images and graphs.

[1102] Step 4:

[1103] The server extracts important elements from the analyzed content. Important elements include titles, headings, graphs, tables, key points, etc. The input is the analyzed text elements and image data, and the output is the extracted important elements. Specifically, the extracted elements are stored in an H2 database, and metadata is also recorded.

[1104] Step 5:

[1105] The server evaluates the materials based on the extracted important elements. The evaluation uses pre-trained evaluation criteria and scores using a Scikit-learn model. The input is the extracted important elements, and the output is the evaluation score and comments. Specifically, the server automatically calculates the score based on the evaluation criteria and records it in a database.

[1106] Step 6:

[1107] The server generates specific feedback based on the evaluation results. The feedback includes areas for improvement and advice. The input is the evaluation score and comments, and the output is the generated feedback. Specifically, it uses the Jinja template engine to create an HTML feedback page.

[1108] Step 7:

[1109] The server uses an emotion engine to recognize the user's emotions and adjust the feedback content. The input is the user's facial expression and voice data, and the output is adjusted feedback. Specifically, OpenCV and librosa are used to analyze the user's facial expressions and tone of voice in real time, and the feedback content is adjusted according to the user's emotional state.

[1110] Step 8:

[1111] The server provides the generated feedback to the user, who then modifies the document based on the feedback. The input is the adjusted feedback, and the output is the document modified by the user. Specifically, the user views the feedback in a web browser, modifies the document, and saves it as a new file.

[1112] Step 9:

[1113] The user re-uploads the revised material to the system. The server then repeats the process described above, providing a new evaluation and feedback. The input is the revised material file, and the output is a re-evaluated score and new feedback. Specifically, the user re-uploads, and the server begins a new evaluation process.

[1114] Step 10:

[1115] This process is repeated until the document meets the supervisor's evaluation criteria. The input is a document file that is repeatedly revised and reevaluated, and the output is a final document that meets the supervisor's evaluation criteria. Specifically, the cycle of feedback and revision is repeated until a satisfactory document is completed.

[1116] (Application example 2)

[1117] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1118] In the past, when document creators tried to efficiently create documents that met their superiors' evaluation criteria, the supervisor's specific evaluation criteria and areas for improvement in the document were often not clearly presented, forcing the creator to make repeated revisions. Furthermore, feedback that took into account the creator's emotional state was not provided, which increased work stress. This resulted in an inefficient document creation process and made it difficult to create high-quality documents in a short period of time.

[1119] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1120] In this invention, the server includes means for inputting data created by a material creator, means for analyzing the data and extracting important elements, means for evaluating based on the extracted elements, means for generating feedback based on the evaluation results, means for analyzing the user's emotional state using an emotion engine, means for adjusting the feedback according to the emotional state, and means for providing the feedback to the material creator. This not only enables the material creator to efficiently create high-quality materials that meet the supervisor's evaluation criteria, but also reduces work stress and provides more effective support by providing feedback that takes into account the material creator's emotional state.

[1121] "Document Creator" refers to the individual or organization that creates the document.

[1122] "Data" refers to the entire content, including information such as text, graphs, and images created by the document creator.

[1123] "Important elements" are particularly important parts of the material extracted through analysis, including titles, headings, graphs, tables, key points, etc.

[1124] "Evaluation" refers to the process of judging the quality of material based on extracted key elements.

[1125] "Feedback" refers to specific improvement suggestions and advice provided to the document creator based on the evaluation results.

[1126] An "emotion engine" refers to software or hardware that analyzes a user's facial expressions and tone of voice to determine their current emotional state.

[1127] "Emotional situation" refers to the user's current emotional state, such as stress level or level of understanding.

[1128] "Adjustment" refers to the process of changing the content and format of feedback depending on the emotional situation analyzed by the emotion engine.

[1129] The "document creation process" refers to the series of steps from creating a document to evaluation, feedback, and revision.

[1130] The system for realizing this invention is for a document creator to efficiently create documents that conform to the supervisor's evaluation criteria, and includes the following processes.

[1131] First, the user uploads the documents (e.g., PowerPoint files or reports) they have created from their terminal to the system. The user selects the files using the system interface and clicks the upload button. The terminal then sends the selected files to the server.

[1132] Next, the server receives the document file sent by the user and stores it in internal temporary storage. The stored file is then subjected to analysis. Natural language processing (NLP) and image recognition technologies are used for the analysis to extract important elements such as text, graphs, and images. NLP technologies such as "SpaCy" and "BERT" are used, while image recognition technologies such as "OpenCV" and "TensorFlow" can be used.

[1133] The server then extracts important elements (such as titles, headings, graphs, tables, and key points) from the analyzed content, and records each important element in a database.The server then evaluates the entire document based on the extracted important elements and in accordance with the supervisor's evaluation criteria, which it has learned in advance.Evaluation criteria include clarity, specificity, and design consistency.

[1134] Based on the evaluation results, the server generates specific feedback that details which parts of the material should be improved and how. For example, it provides specific advice for improving the text or instructions for rearranging graphs. The text generation model used in this process is a generative AI model such as GPT-3.

[1135] Furthermore, the server uses an emotion engine to analyze the user's emotional state. The emotion engine analyzes the user's facial expressions and tone of voice to determine their current emotional state. The emotion engine uses tools such as "Affectiva" and "Microsoft Azure Emotion API." This allows the server to determine whether the user is feeling stressed or has a high level of understanding.

[1136] When providing feedback, the system takes into account the results of the emotion engine and provides the generated feedback to the user. For example, if the user is feeling stressed, the system will simplify the feedback content, and conversely, if the user is calm, it will provide detailed instructions. In this way, feedback appropriate to the emotional state of the document creator is provided.

[1137] As a concrete example, consider the case where a user uploads a document titled "Annual Sales Report." Based on the server's evaluation criteria, the server returns feedback that the title is not specific. If the emotion engine analyzes the user's facial expression and recognizes that the user is stressed, it offers a simple suggestion for improvement. For example, it suggests revising the title to "Annual Sales Report 2023 – Q1 Overview."

[1138] Examples of prompts include:

[1139] "Please suggest ways to make the document titles concise and specific."

[1140] "What are some best practices for rearranging graphs for better visibility?"

[1141] In this way, the system can support the document creator in efficiently creating high-quality documents that meet the supervisor's evaluation criteria.

[1142] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1143] Step 1:

[1144] Upload user-created materials.

[1145] Input: User selection of file (e.g. PowerPoint or report file)

[1146] Specific operation: The user selects the document file through the system interface and clicks the upload button.

[1147] Output: The selected files are sent to the server.

[1148] Step 2:

[1149] The server receives the document file and stores it in temporary storage.

[1150] Input: Data file sent from the user's terminal

[1151] Specific operation: The server receives the sent file and stores it in its internal temporary storage.

[1152] Output: The saved file is the target for analysis processing.

[1153] Step 3:

[1154] The server analyzes the document files and extracts important elements.

[1155] Input: Data file saved in temporary storage

[1156] Specific operation: The server uses natural language processing (NLP) techniques (e.g., SpaCy and BERT) and image recognition techniques (e.g., OpenCV and TensorFlow) to extract important elements such as text, graphs, and images.

[1157] Data processing / calculation: Analysis of text and image data in documents

[1158] Output: The extracted important elements (titles, headings, graphs, tables, key points, etc.) are recorded in a database.

[1159] Step 4:

[1160] The server evaluates the entire document based on the extracted important elements.

[1161] Input: Key elements recorded in the database

[1162] Specific behavior: The server evaluates the clarity, specificity, and design consistency based on the supervisor's evaluation criteria that it has learned in advance.

[1163] Data processing / calculation: Assessing the quality of the material based on key factors

[1164] Output: The evaluation result is generated.

[1165] Step 5:

[1166] The server generates specific feedback based on the evaluation results.

[1167] Input: Evaluation result

[1168] Specific operation: The server uses a generative AI model (e.g., "GPT-3") to generate specific improvement suggestions for the material.

[1169] Data processing / calculation: Text generation based on evaluation results

[1170] Output: Specific feedback is generated.

[1171] Step 6:

[1172] The server uses an emotion engine to analyze the user's emotional state.

[1173] Input: User's facial expression data and tone of voice data

[1174] Specific operation: The server performs analysis using an emotion engine (e.g., "Affectiva" or "Microsoft Azure Emotion API").

[1175] Data processing / calculation: Analysis of user facial expressions and tone of voice

[1176] Output: The user's emotional state is determined.

[1177] Step 7:

[1178] The server adjusts the feedback depending on the emotional situation.

[1179] Input: Generated feedback and the user's emotional state

[1180] Specific behavior: The server simplifies the feedback if the user is stressed, and provides detailed instructions if the user is calm.

[1181] Data processing / calculation: Adjusting the content and format of feedback

[1182] Output: Feedback appropriate to the user's emotions is generated.

[1183] Step 8:

[1184] The server provides feedback to the user.

[1185] Input: Feedback appropriate to the user's emotions

[1186] Specific Actions: The server provides the generated feedback to the user.

[1187] Output: The feedback the user receives is displayed on the screen.

[1188] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1189] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1190] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1191] [Fourth embodiment]

[1192] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1193] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1194] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1195] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1196] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1197] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1198] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1199] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1200] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1201] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1202] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1203] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1204] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1205] The system of the present invention aims to enable a document creator to quickly and efficiently create documents that meet the supervisor's evaluation criteria. To achieve this aim, the system includes the following means.

[1206] 1. Uploading materials

[1207] The user uploads the materials (presentation files and reports) they have created to the system from their terminal. The user selects the material file using the system interface and provides the data by clicking the upload button.

[1208] 2. Analysis of the data

[1209] The server analyzes the received data using natural language processing (NLP) and image recognition technologies to extract important elements such as text, graphs, and images. This analysis provides a detailed understanding of the data's contents.

[1210] 3. Evaluation of materials

[1211] The server evaluates the document based on the extracted key elements. The evaluation is performed using an AI model that reflects the supervisor's evaluation criteria. The model judges the quality of the document from aspects such as clarity, specificity, and design.

[1212] 4. Generate feedback

[1213] The server generates specific feedback based on the evaluation results. The generated feedback details which parts of the material should be improved and how. The feedback includes specific correction instructions such as improving text, rearranging graphs, adding images, etc.

[1214] 5. Providing Feedback

[1215] The server provides the generated feedback to the terminal, and the user can see which parts of the material need to be revised through the feedback displayed on the terminal screen.

[1216] 6. Modification of Materials

[1217] The user modifies the document based on the provided feedback. The user then refers to the feedback and changes the content of the document to make it meet the manager's evaluation criteria.

[1218] 7. Re-upload and Re-rating

[1219] The user then uploads the revised document back to the system. The server then repeats the process, evaluating the document and providing new feedback. This process continues until the document meets the supervisor's evaluation criteria.

[1220] Specific examples

[1221] Example 1: Improving the title

[1222] A user uploads a document called "Annual Sales Report." Based on the server's evaluation criteria, the server returns feedback that the title is not specific. The user then corrects the title to "Annual Sales Report 2023 – Q1 Overview" and re-uploads the document.

[1223] Example 2: Improving graph visualization

[1224] A user uploads a slide showing sales data in the form of a bar graph. The server returns feedback that "there is too much data and it's difficult to understand." The user highlights some of the data and adds a sub-slide based on that. The revised slide is then uploaded again, and the next feedback is received.

[1225] These procedures enable document creators to create high-quality documents that satisfy their superiors in a short period of time. Each method in the system is designed to streamline the document creation process and reduce repetition of work.

[1226] The processing flow will be explained below.

[1227] Step 1: Upload your data

[1228] Users upload PowerPoint presentations or reports created on their own devices to the system as files. Users select the files from the system interface and click the upload button. The device then sends the selected files to the server.

[1229] Step 2: Receiving and storing materials

[1230] The server receives the document file sent by the user and stores the received file in temporary storage within the server.

[1231] Step 3: Analyze the material

[1232] The server analyzes the stored files and breaks down the content of the documents, using natural language processing (NLP) technology to analyze the text content and image recognition technology to extract visual elements such as images and graphs.

[1233] Step 4: Extracting important elements

[1234] The server extracts important elements from the document (e.g., title, headings, graphs, tables, key points, etc.) from the analyzed content. Each element is recorded in the database as an independent element.

[1235] Step 5: Evaluate the material

[1236] The server evaluates the entire document based on the extracted important elements, using specific indicators (e.g., clarity, specificity, and design consistency) based on the supervisor's evaluation criteria that it has learned in advance.

[1237] Step 6: Generate feedback

[1238] The server generates feedback based on the evaluation results, creating comments that point out areas for improvement or missing elements, and presents them in a format that is easy for the user to understand.

[1239] Step 7: Provide feedback

[1240] The server sends the generated feedback to the user's device, where the user can view the feedback on the device screen.

[1241] Step 8: Modify the material

[1242] Users can then modify their own documents based on the feedback they receive from the server, for example by making the title more specific or improving the layout of the graphs.

[1243] Step 9: Re-upload

[1244] The user then re-uploads the revised document to the system. The re-uploading procedure is the same as in step 1.

[1245] Step 10: Reassess and feedback iteratively

[1246] The server then repeats steps 2 to 7 for the newly uploaded materials. This process is repeated until the materials meet the supervisor's evaluation criteria. The user can continue to receive feedback and make revisions as many times as they like.

[1247] This series of steps enables the document creator to efficiently create documents that will satisfy their superiors.

[1248] Example 1

[1249] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1250] The goal of this project is to solve the problem of time-consuming and labor-intensive manual corrections and repetitive tasks that are required when document creators create documents that meet their superiors' evaluation criteria efficiently and quickly. Specifically, a system is needed that supports document creators in making effective corrections without hesitation to meet evaluation criteria such as whether the content of the document is specific, easy to understand, and well-designed.

[1251] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1252] In this invention, the server includes means for inputting data created by a document creator, means for analyzing the data and extracting important elements, means for making an evaluation based on the extracted elements, means for generating feedback based on the evaluation results, means for providing the feedback to the document creator, means for the document creator to correct the data based on the generated feedback, means for re-inputting the corrected data, and means for re-analyzing and re-evaluating the corrected data. This allows the document creator to make corrections quickly and accurately based on the feedback provided by the system, making it possible to efficiently create high-quality documents that meet the evaluation criteria of their superiors.

[1253] A "material creator" is a user who creates a material and uploads it to the system.

[1254] "Data" refers to document files such as presentation files and reports created by document creators.

[1255] "Means of input" refers to the interface that allows document creators to upload data into the system.

[1256] "Means of analysis" refers to the process by which the server analyzes the data received using natural language processing and image recognition technology to extract important elements.

[1257] "Important elements" refer to items necessary for evaluating materials such as text, graphs, and images in the data.

[1258] "Means of evaluation" refers to an AI model that evaluates the quality of materials based on extracted elements.

[1259] "Means of generating feedback" refers to the process of creating specific corrective instructions based on the evaluation results.

[1260] "Means for providing" refers to an interface for notifying the material creator of the generated feedback and displaying it.

[1261] "Measures to revise" refers to actions taken by the material creator to revise the material based on the feedback provided.

[1262] "Means for re-entering" refers to an interface for uploading corrected materials back into the system.

[1263] "Reanalysis and reevaluation" refers to the process of reanalyzing and evaluating the corrected data.

[1264] "Generative AI model" refers to an artificial intelligence model used to evaluate materials and generate feedback.

[1265] A "prompt" refers to a specific question or instruction that feeds data into a generative AI model.

[1266] The present invention relates to a system that enables document creators to efficiently and quickly create documents that meet their supervisor's evaluation criteria. To achieve this goal, the system includes a series of means, specifically, procedures for uploading documents, analyzing, evaluating, generating feedback, providing feedback, correcting, and reevaluating.

[1267] Hardware and Software

[1268] The hardware that realizes this system includes a server and user terminals. The server is equipped with a high-performance processor and large-capacity memory, and by incorporating a GPU in particular, the processing power of the AI ​​model can be improved. Terminals can be general-purpose personal computers or tablet terminals.

[1269] The following software is used:

[1270] Natural Language Processing (NLP) libraries: NLTK (Natural Language Toolkit), SpaCy

[1271] Image recognition library: OpenCV, TensorFlow

[1272] AI model frameworks: TensorFlow, PyTorch

[1273] Uploading materials

[1274] The user opens the system interface using a terminal and uploads the created document. This operation is completed by clicking the "Select File" button on the simple interface, selecting the document file, and then clicking the "Upload" button. The terminal sends the document file to the server using an HTTP request.

[1275] Analysis of data

[1276] The server uses natural language processing (NLP) and image recognition technologies to analyze the documents received. Specifically, the server uses NLTK and SpaCy to analyze the text, and OpenCV and TensorFlow to analyze graphs and images. This allows the server to extract important elements from the documents.

[1277] Evaluation of materials

[1278] The server evaluates the documents based on the analyzed data. This evaluation uses a generative AI model that reflects the supervisor's evaluation criteria. The AI ​​model used is a generative AI model (e.g., GPT-3), which evaluates the documents' clarity, specificity, design, etc.

[1279] Generate feedback

[1280] The server generates specific feedback based on the evaluation results. The generated feedback includes specific instructions on how to improve the material, such as improving the text, rearranging graphs, or adding images. The server uses a generative AI model to generate the feedback in natural language.

[1281] Providing feedback

[1282] The generated feedback is sent from the server to the device, where the user can check it. The feedback specifically indicates which slides in the document need to be revised and what kind of revisions are required. The user can review this and begin revising the document.

[1283] Corrections to the material

[1284] The user then modifies the document based on the provided feedback, such as by modifying the text, rearranging the graphs, and adding new images as needed, according to the feedback instructions. After modifying the document, the user then saves the modified document.

[1285] Re-upload and re-evaluation

[1286] The user then uploads the revised material back into the system, after which the server repeats the analysis and evaluation process described above and provides new feedback. This process is repeated until the material meets the supervisor's evaluation criteria.

[1287] Examples of concrete examples and prompts

[1288] Example 1: Improving the title

[1289] A user uploads a document called "Annual Sales Report." Based on the server's evaluation criteria, the server returns feedback that the title is not specific. The user corrects the title to "Annual Sales Report 2023 – Q1 Overview" and re-uploads it. Example prompt: "How can you make the title of this presentation more specific?"

[1290] Example 2: Improving graph visualization

[1291] A user uploads a slide showing sales data in the form of a bar graph. The server provides feedback that "there is too much data and it's difficult to understand." The user highlights some of the data and adds a sub-slide based on that. The revised slide is then uploaded again and the following feedback is received: Example prompt: "How can you modify this graph to make it easier to read?"

[1292] As described above, this system provides a series of functions to streamline the document creation process and is designed to enable document creators to quickly create high-quality documents.

[1293] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1294] Step 1: Upload your materials

[1295] The user opens the system interface using a terminal and selects the created document. The user clicks the "Select File" button, selects the document file, and then clicks the "Upload" button. The terminal sends the document file to the server using an HTTP request.

[1296] Input: User-created material file

[1297] Output: The document file sent to the server

[1298] Specific operation: When the user clicks the "Upload" button, the device sends the document file to the server. The file is sent as an attachment to the HTTP request.

[1299] Step 2: Analyze the data

[1300] The server analyzes the received file. To analyze, it first reads the file contents and separates the text from the images. It then uses natural language processing (NLP) technology to analyze the text portion, and image recognition technology to analyze the graphs and images.

[1301] Input: Data file sent to the server

[1302] Output: Extracted important elements (text data, image data)

[1303] Specific operation: The server uses NLTK and SpaCy to analyze the text and identify keywords and sentence structure, and OpenCV and TensorFlow to extract elements from graphs and images and obtain features.

[1304] Step 3: Evaluate the material

[1305] The server evaluates the document based on the analyzed data. This evaluation uses a generative AI model that reflects the supervisor's evaluation criteria. The generative AI model receives the analyzed data as input and evaluates the document based on that.

[1306] Input: Extracted important elements (text data, image data)

[1307] Output: Evaluation results (quality score and evaluation comments)

[1308] Specific operation: The server inputs the analysis results into the generative AI model and generates an evaluation result. The server compares the results with the evaluation criteria to assign a score and generate evaluation comments.

[1309] Step 4: Generate feedback

[1310] The server generates specific feedback based on the evaluation results, including specific instructions on how to improve the material.

[1311] Input: Evaluation results (quality score and evaluation comments)

[1312] Output: Generated feedback (correction instructions)

[1313] Specific behavior: Based on the evaluation results, the server uses a generative AI model to generate feedback, which includes specific correction instructions such as improving text, rearranging graphs, and adding images.

[1314] Step 5: Provide feedback

[1315] The server generates feedback and sends it to the device. The user can view the feedback on the device. The server formats the feedback data into structured data such as JSON format and sends it to the device.

[1316] Input: Generated feedback (correction instructions)

[1317] Output: Feedback provided to the terminal

[1318] Specific operation: The server sends feedback data to the terminal, and the terminal displays the received feedback on the screen and notifies the user.

[1319] Step 6: Modify the material

[1320] The user modifies the material based on the feedback provided, specifically by modifying the text, rearranging charts, or adding new images as directed by the feedback.

[1321] Input: Feedback provided

[1322] Output: Modified documentation file

[1323] Specific operation: The user makes corrections using document editing software (e.g., Microsoft PowerPoint, Google Slides) and saves the corrected document.

[1324] Step 7: Re-upload and re-evaluate

[1325] The user uploads the revised material back into the system, and the server receives it again, performs the analysis and evaluation process described above, and provides new feedback.

[1326] Input: Modified document file

[1327] Output: New evaluation results and new feedback

[1328] How it works: The user re-uploads the revised document, and the server analyzes and evaluates it again, generating and providing new feedback. This process is repeated until the document meets the supervisor's evaluation criteria.

[1329] (Application example 1)

[1330] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1331] Existing document creation support systems are primarily used in office environments, making it difficult to support document creation and reporting in the field. Furthermore, there was a lack of means to quickly analyze data acquired in the field and provide feedback, making it difficult for document creators to efficiently create high-quality documents. To solve these issues, a system that can seamlessly acquire and analyze data in the field and provide feedback is needed.

[1332] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1333] In this invention, the server includes means for inputting data created by a document creator, means for analyzing the data and extracting important elements, means for evaluating the data based on the extracted elements, means for generating feedback based on the evaluation results, means for providing the feedback to the document creator, means for using images and videos captured on-site as input, means for analyzing the data using natural language processing, image recognition technology, and OCR, and means for displaying the generated feedback on a smart device display. This makes it possible to instantly acquire and analyze data and provide feedback when creating documents on-site. As a result, document creators can quickly create high-quality documents that reflect the situation on-site.

[1334] A "document creator" is a person or organization that creates a presentation or report.

[1335] "Data" refers to a collection of information such as presentation files and reports created by document creators.

[1336] "Uploading" is the act of sending materials from a terminal to a server.

[1337] "Analysis" is the process of carefully examining and evaluating data to extract important elements.

[1338] "Key elements" are items of text, graphs, images, etc. that deserve special attention in the content of the document.

[1339] "Evaluation" is the act of judging the quality of material based on extracted important elements.

[1340] "Feedback" refers to specific comments based on the evaluation results, including suggestions for improvements and corrections to the materials.

[1341] The "site" is a place where actual work is carried out, such as a factory or a production line.

[1342] A "smart device" is a terminal that can connect to the Internet, such as smart glasses or a head-mounted display.

[1343] "OCR" stands for optical character recognition, a technology that extracts text from an image as digital data.

[1344] "Natural language processing" is a technology that allows computers to understand and analyze human language.

[1345] "Image recognition technology" is a technology that allows a computer to analyze images and understand their content.

[1346] A "display" is the display screen of a smart device, and is a device for visually checking information.

[1347] The system for realizing this invention includes the following programs. First, the document creator uses smart glasses on-site to capture images and videos. This device can be Google Glass or Microsoft HoloLens. The captured data is uploaded from the smart glasses to a server. The server receives this data and begins analysis.

[1348] This analysis process uses natural language processing (NLP), image recognition technology, and OCR technology. NLP is used to analyze the text information in the document and extract important elements. This includes understanding the content of the text and extracting keywords. Image recognition technology recognizes specific objects and situations from the image and extracts them as part of the document. OCR technology digitizes the text in the image so that it can be processed as data.

[1349] The server uses these technologies to analyze the data and evaluate the materials based on the extracted key elements. This evaluation is performed using a generative AI model that reflects the supervisor's evaluation criteria. This AI model evaluates the materials' clarity, specificity, design, and other criteria. Once the evaluation is complete, the server generates specific feedback.

[1350] The generated feedback is instantly displayed on the smart glasses' display, allowing workers to review the feedback and make on-site corrections to the materials. This feedback includes specific instructions such as suggestions for improving text, rearranging graphs, or adding images.

[1351] For example, if a worker discovers an abnormality on a production line and takes a photo of the abnormal area with smart glasses, the image is uploaded to the server. The server analyzes the image and evaluates the details of the abnormality. Feedback such as "Add a photo of the abnormality to the report" is then generated, allowing the worker to make corrections. As a result, high-quality documents that reflect the situation on-site are quickly created.

[1352] Examples of prompts for generative AI models include:

[1353] "This image shows an anomaly on the production line. Please use the system to analyze the content and generate feedback to create a report on the anomaly."

[1354] This invention allows document creators to instantly obtain documents on-site and analyze, evaluate, and provide feedback, making it possible to quickly create high-quality documents.

[1355] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1356] Step 1:

[1357] Users use smart glasses to capture images and videos needed for document creation on-site. Devices such as Google Glass and Microsoft HoloLens can be used for this purpose. The captured data is stored in the smart glasses' internal storage.

[1358] Input: Images and videos from the scene

[1359] Output: Data stored in the smart glasses storage

[1360] Step 2:

[1361] The user operates the smart glasses through an interface and uploads captured images and videos to a server, which is the process of transmitting data from the device to the server.

[1362] Input: Data stored in the smart glasses storage

[1363] Output: Data uploaded to the server

[1364] Step 3:

[1365] The server receives the uploaded data and begins analyzing it, first using OCR technology to extract text from the image, then using natural language processing (NLP) and image recognition technology to extract key elements from the image or video.

[1366] Input: Data uploaded to the server

[1367] Output: Extracted text data and image recognition results

[1368] Step 4:

[1369] The server evaluates the document based on the extracted key elements. This evaluation uses a generative AI model that reflects the supervisor's evaluation criteria. The model evaluates the document's clarity, specificity, design, and other criteria.

[1370] Input: Extracted text data and image recognition results

[1371] Output: Evaluation results

[1372] Step 5:

[1373] The server generates specific feedback based on the evaluation results, detailing how and where the material should be improved, including suggestions for improving text, rearranging graphs, adding images, and other specific corrections.

[1374] Input: Evaluation result

[1375] Output: Feedback

[1376] Step 6:

[1377] The server displays the generated feedback on the smart glasses display, allowing workers to check the feedback in real time and make corrections to the materials on-site.

[1378] Input: Feedback

[1379] Output: Feedback displayed on the smart glasses display

[1380] Step 7:

[1381] The user modifies the data based on the feedback provided through the smart glasses, and if necessary, re-acquires on-site data using the smart glasses and uploads the newly modified data to the server.

[1382] Input: Corrective instructions based on feedback

[1383] Output: Corrected material

[1384] By repeating this process, the document is adjusted until it meets the supervisor's evaluation criteria, allowing document creators to quickly create high-quality documents that reflect the situation on-site.

[1385] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1386] The system of the present invention aims to improve the process by which document creators can efficiently create documents that meet their superiors' evaluation criteria. In particular, by combining it with an emotion engine, it provides feedback that takes into account the user's emotional state.

[1387] 1. Uploading materials

[1388] The user uploads the documents (e.g., PowerPoint files or reports) they have created from their terminal to the system. The user selects the files using the system interface, clicks the upload button, and the terminal sends the selected files to the server.

[1389] 2. Receipt and storage of materials

[1390] The server receives the data file sent by the user and stores it in its internal temporary storage. The stored file is then subjected to analysis processing.

[1391] 3. Analysis of the data

[1392] The server analyzes the stored data files and breaks down their contents, using natural language processing (NLP) and image recognition technology to extract important elements such as text, graphs, and images.

[1393] 4. Extracting important elements

[1394] The server extracts important elements from the analyzed content (e.g., titles, headings, graphs, tables, key points, etc.), and records each important element in the database.

[1395] 5. Evaluation of materials

[1396] The server then evaluates the entire document based on the extracted important elements and in accordance with the supervisor's pre-trained evaluation criteria, including clarity, specificity, and design consistency.

[1397] 6. Generate feedback

[1398] The server generates specific feedback based on the evaluation results, detailing how and where the material should be improved, for example providing specific advice on improving text or instructions on rearranging graphs.

[1399] 7. Leveraging Emotional Engines

[1400] The server utilizes an emotion engine to recognize the user's emotions. The emotion engine analyzes the user's facial expressions and tone of voice to determine their current emotional state. This allows the server to determine whether the user is feeling stressed or has a high level of understanding.

[1401] 8. Providing Feedback

[1402] The server considers the results of the emotion engine and provides the generated feedback to the user. For example, if the user is stressed, the server will simplify the feedback, and if the user is calm, it will provide detailed instructions.

[1403] 9. Modification of Materials

[1404] The user then modifies the document based on the feedback received from the server. The modification work is carried out on the user's terminal, and the content of the document is changed while referring to the feedback.

[1405] 10. Re-uploading and Re-rating

[1406] The user then uploads the revised document back to the system. The server then repeats the process described above (steps 2 to 9), reassessing the document and providing new feedback. This process continues until the document meets the supervisor's evaluation criteria.

[1407] Specific examples

[1408] Example 1: Improving the title

[1409] A user uploads a document called "Annual Sales Report." Based on the server's evaluation criteria, the server returns feedback that the title is not specific. If the emotion engine analyzes the user's facial expression and recognizes that the user is stressed, it offers a simple suggestion for improvement. For example, it suggests revising the title to "Annual Sales Report 2023 – Q1 Overview."

[1410] Example 2: Improving graph visualization

[1411] A user uploads a slide showing sales data in the form of a bar graph. When the server responds with feedback that "there's too much data and it's difficult to understand," the emotion engine determines from the user's tone of voice that they are calm and provides detailed instructions for improvement, such as suggesting ways to highlight parts of the data or adding new sub-slides.

[1412] These methods enable document creators to create high-quality documents that satisfy their superiors in a short period of time. Each method and emotion engine of the system is designed to streamline the document creation process and reduce repetition of work.

[1413] The processing flow will be explained below.

[1414] Step 1: Upload your materials

[1415] Users select PowerPoint presentations or report files they have created on their own devices, and then click the upload button on the device interface to send the files to the system.

[1416] Step 2: Receive and save the file

[1417] The server receives the uploaded file and stores it in its internal storage. This process allows the server to proceed to the next step without losing any data.

[1418] Step 3: Analyze the material

[1419] The server analyzes the stored data files, extracting text content using natural language processing (NLP) technology and analyzing visual elements such as graphs and images using image recognition technology. This analysis allows the data content to be structured.

[1420] Step 4: Extracting important elements

[1421] The server extracts important elements from the analyzed data, such as titles, headings, graphs, tables, and key points, and records each extracted element in a database for use in subsequent evaluation processes.

[1422] Step 5: Evaluate the material

[1423] The server evaluates the document based on the extracted important elements. Based on the supervisor's evaluation criteria, which it has learned in advance, the server evaluates the entire document in terms of clarity, specificity, design consistency, etc. The results of this evaluation form the basis for generating feedback.

[1424] Step 6: Generate feedback

[1425] The server generates specific feedback based on the evaluation results. For example, if the title is not specific enough, it will generate an instruction to "use a more specific title." It also includes specific advice on graph visualization, such as "the amount of data is too large, please add highlights."

[1426] Step 7: Emotion Recognition with the Emotion Engine

[1427] The server uses an emotion engine to recognize the user's emotional state. When the user checks the feedback, the server analyzes their facial expressions and tone of voice via a camera and microphone to determine their stress level and emotional state.

[1428] Step 8: Feedback adjustment based on emotional information

[1429] The server adjusts the content and presentation of the generated feedback based on the user's emotional information recognized by the emotion engine. For example, if the user is feeling stressed, the server will simplify the feedback. Conversely, if the user is calm, the server will provide detailed instructions.

[1430] Step 9: Provide feedback

[1431] The server sends the adjusted feedback to the terminal and provides it to the user, who can check the feedback on the terminal and understand the necessary corrections.

[1432] Step 10: Modifying the material

[1433] The user then modifies the material based on the feedback provided, and the modifications are performed on the user's device, updating the material according to the specific improvements.

[1434] Step 11: Re-upload and re-evaluate

[1435] The user then uploads the revised document back to the system, and the server repeats steps 2 to 9 for the newly uploaded document. This process is repeated until the document meets the supervisor's evaluation criteria.

[1436] Specific examples

[1437] Example 1: Improving the title

[1438] A user uploads a document called "Annual Sales Report." The server analyzes and evaluates it, generating feedback that the title is not specific enough. The emotion engine recognizes the user's stress level and offers a simple suggestion to revise it to "Annual Sales Report 2023 – Q1 Overview."

[1439] Example 2: Improving graph visualization

[1440] A user uploads a slide displaying sales data in the form of a bar graph. The server evaluates the slide and returns feedback that the data is too large and difficult to understand. If the emotion engine recognizes that the user is calm, it offers detailed instructions such as highlighting some of the data and adding a new sub-slide.

[1441] This series of processes enables users to efficiently create high-quality documents that meet their superiors' expectations. The system utilizes emotion recognition by an emotion engine and flexibly adjusts feedback according to the user's emotional state, providing effective support for document creation.

[1442] Example 2

[1443] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1444] The traditional document creation process has the problem that it is difficult for document creators to efficiently create documents that meet their superiors' evaluation criteria. Furthermore, the complicated process of making corrections after receiving feedback is often inefficient and time-consuming. Another problem is that the feedback content is uniform, and appropriate feedback is not provided based on the user's emotional state.

[1445] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for inputting data created by a material creator, a means for analyzing the data and extracting important elements, a means for making an evaluation based on the extracted elements, a means for generating feedback based on the evaluation results, a means for providing the feedback to the material creator, and a means for recognizing the user's emotions and adjusting the content of the feedback. This enables the material creator to efficiently create materials that meet the supervisor's evaluation criteria and provide appropriate feedback according to the user's emotional state.

[1446] A "document creator" is a person whose role is to create reports, presentation materials, etc.

[1447] "Data" is a collection of information including text, images, graphs, tables, etc. created by the document creator.

[1448] "Input means" refers to a method or device for providing data created by a document creator to the system.

[1449] "Means of analysis" are the methods and techniques that a system uses to process data, understand its contents, and extract important elements.

[1450] "Key elements" are the main information needed for evaluation and feedback, such as the title, headings, graphs, tables, and key points within the document.

[1451] "Means of evaluation" are methods or algorithms for determining and evaluating the quality of data based on the extracted key elements.

[1452] "Means for generating feedback" refers to methods and techniques for providing improvements and advice to document creators based on the evaluation results.

[1453] The "means for providing" refers to a method or device for conveying the generated feedback to the material creator.

[1454] "Means for recognizing emotions and adjusting the content of feedback" refers to methods or techniques for recognizing the user's current emotional state and adjusting the content and level of detail of the feedback accordingly.

[1455] "Natural language processing" is a technology that enables computers to understand and process human language.

[1456] "Image recognition technology" is a technology that allows a computer to analyze image data and recognize the information contained within it.

[1457] The system of the present invention allows document creators to efficiently create documents that meet their superiors' evaluation criteria. In particular, by combining this system with an emotion engine, it is possible to provide feedback that takes into account the user's emotional state.

[1458] The system consists of the following main elements:

[1459] 1. Input Method

[1460] Users use an interface to upload created document files (such as PowerPoint files or reports) from their terminals to the system. The user selects the target file in the file selection dialog and clicks the upload button, and the terminal sends the selected file to the server.

[1461] 2. Analysis Methods

[1462] The server uses natural language processing (NLP) and image recognition technologies to analyze the received data files. Specifically, it uses Apache Tika to analyze the file contents, NLTK (Natural Language Toolkit) to extract text elements, and OpenCV to recognize images and graphs.

[1463] 3. Means of extracting important elements

[1464] The server extracts important elements from the analyzed content (titles, headings, graphs, tables, key points, etc.) and records them in a database. For example, it connects to an H2 database and stores the extracted titles and graph metadata in the database.

[1465] 4. Evaluation methods

[1466] The server evaluates the documents based on the extracted key elements, using a Scikit-learn model to score them based on pre-trained evaluation criteria, including clarity, specificity, and design consistency.

[1467] 5. Means of generating feedback

[1468] The server generates specific feedback based on the evaluation results. It uses the Jinja template engine to create an HTML feedback page with suggestions for improvement. This feedback details what parts of the material should be improved and how.

[1469] 6. Emotional awareness and regulation tools

[1470] The server uses an emotion engine to recognize the user's emotions and adjusts the feedback accordingly. Specifically, it uses OpenCV and librosa to analyze the user's facial expressions and tone of voice to determine their emotional state. For example, if the user is stressed, the server will simplify the feedback, but if they are calm, it will provide more detailed instructions.

[1471] 7. Means of Providing Feedback

[1472] The server provides the generated feedback to the user, for example, when the user views the feedback in a web browser, detailed or simplified instructions are displayed based on the dynamic response from the server.

[1473] Specific examples

[1474] Example 1: Improving the title

[1475] If a user uploads a document titled "Annual Sales Report," the server will evaluate it as "not specific enough" and suggest a more specific title, "Annual Sales Report 2023 – Q1 Overview." If the emotion engine analyzes the user's facial expressions and recognizes that they are feeling stressed, the server will offer simple suggestions for improvement.

[1476] Example 2: Improving graph visualization

[1477] If a user uploads a slide showing sales data in the form of a bar graph, the server will determine that there is too much data and that it is difficult to understand. The emotion engine will then determine if the user is calm and provide detailed instructions on how to improve the slide, such as how to highlight the data or add new sub-slides.

[1478] Prompt Sentence Examples

[1479] "Please improve the title of this PowerPoint file to make it more specific and descriptive."

[1480] "How can I change the data in this bar chart to make it more visually understandable?"

[1481] As a result, document creators can create high-quality documents that satisfy their superiors in a short period of time. Each method and emotion engine of the system is designed to streamline the document creation process and reduce repetition of work.

[1482] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1483] System program processing flow

[1484] Step 1:

[1485] A document file created by a user is uploaded from the terminal to the system. The user selects a file using the system interface and clicks the upload button. The terminal sends the file selected by the user to the server. The input here is the document file selected by the user, and the output is the file received by the server. Specifically, a file selection dialog is displayed, the user selects a file, and it is sent to the server via an Ajax request using JavaScript.

[1486] Step 2:

[1487] The server receives the file sent from the device and temporarily stores it in its internal storage. The input is the file sent from the device, and the output is the temporarily stored file and its metadata. Specifically, the server receives a secure HTTP request, temporarily stores the file in RAM, and simultaneously records metadata such as the file name and upload date and time in a database.

[1488] Step 3:

[1489] The server analyzes the stored data files and decomposes their contents. It performs the analysis using natural language processing (NLP) and image recognition technologies. The input is the stored data file, and the output is the analyzed text elements and image data. Specifically, it uses Apache Tika to extract text from the file, NLTK to further decompose the text, and OpenCV to analyze images and graphs.

[1490] Step 4:

[1491] The server extracts important elements from the analyzed content. Important elements include titles, headings, graphs, tables, key points, etc. The input is the analyzed text elements and image data, and the output is the extracted important elements. Specifically, the extracted elements are stored in an H2 database, and metadata is also recorded.

[1492] Step 5:

[1493] The server evaluates the materials based on the extracted important elements. The evaluation uses pre-trained evaluation criteria and scores using a Scikit-learn model. The input is the extracted important elements, and the output is the evaluation score and comments. Specifically, the server automatically calculates the score based on the evaluation criteria and records it in a database.

[1494] Step 6:

[1495] The server generates specific feedback based on the evaluation results. The feedback includes areas for improvement and advice. The input is the evaluation score and comments, and the output is the generated feedback. Specifically, it uses the Jinja template engine to create an HTML feedback page.

[1496] Step 7:

[1497] The server uses an emotion engine to recognize the user's emotions and adjust the feedback content. The input is the user's facial expression and voice data, and the output is adjusted feedback. Specifically, OpenCV and librosa are used to analyze the user's facial expressions and tone of voice in real time, and the feedback content is adjusted according to the user's emotional state.

[1498] Step 8:

[1499] The server provides the generated feedback to the user, who then modifies the document based on the feedback. The input is the adjusted feedback, and the output is the document modified by the user. Specifically, the user views the feedback in a web browser, modifies the document, and saves it as a new file.

[1500] Step 9:

[1501] The user re-uploads the revised material to the system. The server then repeats the process described above, providing a new evaluation and feedback. The input is the revised material file, and the output is a re-evaluated score and new feedback. Specifically, the user re-uploads, and the server begins a new evaluation process.

[1502] Step 10:

[1503] This process is repeated until the document meets the supervisor's evaluation criteria. The input is a document file that is repeatedly revised and reevaluated, and the output is a final document that meets the supervisor's evaluation criteria. Specifically, the cycle of feedback and revision is repeated until a satisfactory document is completed.

[1504] (Application example 2)

[1505] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1506] In the past, when document creators tried to efficiently create documents that met their superiors' evaluation criteria, the supervisor's specific evaluation criteria and areas for improvement in the document were often not clearly presented, forcing the creator to make repeated revisions. Furthermore, feedback that took into account the creator's emotional state was not provided, which increased work stress. This resulted in an inefficient document creation process and made it difficult to create high-quality documents in a short period of time.

[1507] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1508] In this invention, the server includes means for inputting data created by a material creator, means for analyzing the data and extracting important elements, means for evaluating based on the extracted elements, means for generating feedback based on the evaluation results, means for analyzing the user's emotional state using an emotion engine, means for adjusting the feedback according to the emotional state, and means for providing the feedback to the material creator. This not only enables the material creator to efficiently create high-quality materials that meet the supervisor's evaluation criteria, but also reduces work stress and provides more effective support by providing feedback that takes into account the material creator's emotional state.

[1509] "Document Creator" refers to the individual or organization that creates the document.

[1510] "Data" refers to the entire content, including information such as text, graphs, and images created by the document creator.

[1511] "Important elements" are particularly important parts of the material extracted through analysis, including titles, headings, graphs, tables, key points, etc.

[1512] "Evaluation" refers to the process of judging the quality of material based on extracted key elements.

[1513] "Feedback" refers to specific improvement suggestions and advice provided to the document creator based on the evaluation results.

[1514] An "emotion engine" refers to software or hardware that analyzes a user's facial expressions and tone of voice to determine their current emotional state.

[1515] "Emotional situation" refers to the user's current emotional state, such as stress level or level of understanding.

[1516] "Adjustment" refers to the process of changing the content and format of feedback depending on the emotional situation analyzed by the emotion engine.

[1517] The "document creation process" refers to the series of steps from creating a document to evaluation, feedback, and revision.

[1518] The system for realizing this invention is for a document creator to efficiently create documents that conform to the supervisor's evaluation criteria, and includes the following processes.

[1519] First, the user uploads the documents (e.g., PowerPoint files or reports) they have created from their terminal to the system. The user selects the files using the system interface and clicks the upload button. The terminal then sends the selected files to the server.

[1520] Next, the server receives the document file sent by the user and stores it in internal temporary storage. The stored file is then subjected to analysis. Natural language processing (NLP) and image recognition technologies are used for the analysis to extract important elements such as text, graphs, and images. NLP technologies such as "SpaCy" and "BERT" are used, while image recognition technologies such as "OpenCV" and "TensorFlow" can be used.

[1521] The server then extracts important elements (such as titles, headings, graphs, tables, and key points) from the analyzed content, and records each important element in a database.The server then evaluates the entire document based on the extracted important elements and in accordance with the supervisor's evaluation criteria, which it has learned in advance.Evaluation criteria include clarity, specificity, and design consistency.

[1522] Based on the evaluation results, the server generates specific feedback that details which parts of the material should be improved and how. For example, it provides specific advice for improving the text or instructions for rearranging graphs. The text generation model used in this process is a generative AI model such as GPT-3.

[1523] Furthermore, the server uses an emotion engine to analyze the user's emotional state. The emotion engine analyzes the user's facial expressions and tone of voice to determine their current emotional state. The emotion engine uses tools such as "Affectiva" and "Microsoft Azure Emotion API." This allows the server to determine whether the user is feeling stressed or has a high level of understanding.

[1524] When providing feedback, the system takes into account the results of the emotion engine and provides the generated feedback to the user. For example, if the user is feeling stressed, the system will simplify the feedback content, and conversely, if the user is calm, it will provide detailed instructions. In this way, feedback appropriate to the emotional state of the document creator is provided.

[1525] As a concrete example, consider the case where a user uploads a document titled "Annual Sales Report." Based on the server's evaluation criteria, the server returns feedback that the title is not specific. If the emotion engine analyzes the user's facial expression and recognizes that the user is stressed, it offers a simple suggestion for improvement. For example, it suggests revising the title to "Annual Sales Report 2023 – Q1 Overview."

[1526] Examples of prompts include:

[1527] "Please suggest ways to make the document titles concise and specific."

[1528] "What are some best practices for rearranging graphs for better visibility?"

[1529] In this way, the system can support the document creator in efficiently creating high-quality documents that meet the supervisor's evaluation criteria.

[1530] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1531] Step 1:

[1532] Upload user-created materials.

[1533] Input: User selection of file (e.g. PowerPoint or report file)

[1534] Specific operation: The user selects the document file through the system interface and clicks the upload button.

[1535] Output: The selected files are sent to the server.

[1536] Step 2:

[1537] The server receives the document file and stores it in temporary storage.

[1538] Input: Data file sent from the user's terminal

[1539] Specific operation: The server receives the sent file and stores it in its internal temporary storage.

[1540] Output: The saved file is the target for analysis processing.

[1541] Step 3:

[1542] The server analyzes the document files and extracts important elements.

[1543] Input: Data file saved in temporary storage

[1544] Specific operation: The server uses natural language processing (NLP) techniques (e.g., SpaCy and BERT) and image recognition techniques (e.g., OpenCV and TensorFlow) to extract important elements such as text, graphs, and images.

[1545] Data processing / calculation: Analysis of text and image data in documents

[1546] Output: The extracted important elements (titles, headings, graphs, tables, key points, etc.) are recorded in a database.

[1547] Step 4:

[1548] The server evaluates the entire document based on the extracted important elements.

[1549] Input: Key elements recorded in the database

[1550] Specific behavior: The server evaluates the clarity, specificity, and design consistency based on the supervisor's evaluation criteria that it has learned in advance.

[1551] Data processing / calculation: Assessing the quality of the material based on key factors

[1552] Output: The evaluation result is generated.

[1553] Step 5:

[1554] The server generates specific feedback based on the evaluation results.

[1555] Input: Evaluation result

[1556] Specific operation: The server uses a generative AI model (e.g., "GPT-3") to generate specific improvement suggestions for the material.

[1557] Data processing / calculation: Text generation based on evaluation results

[1558] Output: Specific feedback is generated.

[1559] Step 6:

[1560] The server uses an emotion engine to analyze the user's emotional state.

[1561] Input: User's facial expression data and tone of voice data

[1562] Specific operation: The server performs analysis using an emotion engine (e.g., "Affectiva" or "Microsoft Azure Emotion API").

[1563] Data processing / calculation: Analysis of user facial expressions and tone of voice

[1564] Output: The user's emotional state is determined.

[1565] Step 7:

[1566] The server adjusts the feedback depending on the emotional situation.

[1567] Input: Generated feedback and the user's emotional state

[1568] Specific behavior: The server simplifies the feedback if the user is stressed, and provides detailed instructions if the user is calm.

[1569] Data processing / calculation: Adjusting the content and format of feedback

[1570] Output: Feedback appropriate to the user's emotions is generated.

[1571] Step 8:

[1572] The server provides feedback to the user.

[1573] Input: Feedback appropriate to the user's emotions

[1574] Specific Actions: The server provides the generated feedback to the user.

[1575] Output: The feedback the user receives is displayed on the screen.

[1576] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1577] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1578] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1579] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1580] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1581] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1582] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1583] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1584] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1585] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1586] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1587] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1588] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1589] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1590] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1591] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1592] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1593] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1594] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1595] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1596] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1597] The following is further disclosed regarding the above embodiment.

[1598] (Claim 1)

[1599] a means for inputting data created by the document creator;

[1600] means for analyzing the data and extracting important elements;

[1601] A means for performing an evaluation based on the extracted elements;

[1602] a means for generating feedback based on the evaluation results;

[1603] The system includes means for providing said feedback to the material creator.

[1604] (Claim 2)

[1605] The data creator corrects the data based on the feedback,

[1606] 10. The system of claim 1, further comprising means for re-entering corrected data.

[1607] (Claim 3)

[1608] The data is analyzed using natural language processing and image recognition technology.

[1609] 10. The system of claim 1.

[1610] "Example 1"

[1611] (Claim 1)

[1612] a means for inputting data created by the document creator;

[1613] means for analyzing the data and extracting important elements;

[1614] A means for performing an evaluation based on the extracted elements;

[1615] a means for generating feedback based on the evaluation results;

[1616] means for providing said feedback to the material creator;

[1617] a means for the document creator to modify the data based on the generated feedback;

[1618] A means for re-entering the corrected data;

[1619] The system further comprises means for reanalyzing and reevaluating the corrected data.

[1620] (Claim 2)

[1621] and means for providing a prompt for the document creator to input data into the generative AI model.

[1622] 10. The system of claim 1.

[1623] (Claim 3)

[1624] The data is analyzed using natural language processing and image recognition technology.

[1625] 10. The system of claim 1.

[1626] "Application Example 1"

[1627] (Claim 1)

[1628] a means for inputting data created by the document creator;

[1629] means for analyzing the data and extracting important elements;

[1630] A means for performing an evaluation based on the extracted elements;

[1631] a means for generating feedback based on the evaluation results;

[1632] means for providing said feedback to the material creator;

[1633] a means for using images or video captured in the field as input;

[1634] A means for analyzing the data using natural language processing, image recognition technology and OCR;

[1635] The system includes means for displaying the generated feedback on a display of the smart device.

[1636] (Claim 2)

[1637] The data creator corrects the data based on the feedback,

[1638] 10. The system of claim 1, further comprising means for re-entering corrected data.

[1639] (Claim 3)

[1640] We use natural language processing and image recognition technology to analyze the data.

[1641] 10. The system of claim 1, wherein the generated feedback is displayed on a display of the smart device.

[1642] "Example 2: Combining Emotion Engines"

[1643] (Claim 1)

[1644] a means for inputting data created by the document creator;

[1645] means for analyzing the data and extracting important elements;

[1646] A means for performing an evaluation based on the extracted elements;

[1647] a means for generating feedback based on the evaluation results;

[1648] means for providing said feedback to the material creator;

[1649] A system including a means for recognizing a user's emotions and adjusting the content of the feedback.

[1650] (Claim 2)

[1651] 2. The system according to claim 1, further comprising means for the material creator to modify the data based on the feedback and re-enter the modified data.

[1652] (Claim 3)

[1653] The system according to claim 1, wherein the data is analyzed using natural language processing and image recognition techniques.

[1654] "Application example 2 when combining emotion engines"

[1655] (Claim 1)

[1656] a means for inputting data created by the document creator;

[1657] means for analyzing the data and extracting important elements;

[1658] A means for performing an evaluation based on the extracted elements;

[1659] a means for generating feedback based on the evaluation results;

[1660] means for analyzing the emotional state of a user using an emotion engine;

[1661] a means of adjusting feedback depending on emotional situations;

[1662] The system includes means for providing said feedback to the material creator.

[1663] (Claim 2)

[1664] 2. The system according to claim 1, further comprising means for the material creator to modify the data based on the feedback and re-enter the modified data.

[1665] (Claim 3)

[1666] The system according to claim 1, wherein the data is analyzed using natural language processing and image recognition techniques. [Explanation of symbols]

[1667] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a means for inputting data created by the document creator; means for analyzing the data and extracting important elements; A means for performing an evaluation based on the extracted elements; a means for generating feedback based on the evaluation results; The system includes means for providing said feedback to the material creator.

2. The data creator corrects the data based on the feedback, 2. The system of claim 1, further comprising means for re-entering corrected data.

3. The data is analyzed using natural language processing and image recognition technology. The system of claim 1 .

Images