system

Abstract

We provide the system. [Solution] A means of learning necessary items and formats by analyzing existing documents, A means of providing learned templates when creating a new document, A means for comparing documents in multiple different formats and detecting differences, A means of generating annotations based on legal requirements and guidelines and suggesting their appropriate location within a document, An information processing system that includes means for verifying the legal requirements of generated documents and providing real-time warnings.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In document creation and proofreading, when comparing documents in different formats, it takes a lot of time and effort. Especially in the case of long documents, visual confirmation is indispensable, which leads to a decrease in work efficiency. In addition, for documents that are handled for the first time, the necessary requirements are not clear, and it is difficult to make appropriate judgments even by referring to reference materials. Furthermore, from a legal perspective, it may not be possible to respond promptly when there are detailed corrections that require confirmation. In addition, when cooperation with multiple related departments is required, the confirmation work is delayed, and there is a problem that efficient document creation is hindered.

Means for Solving the Problems

[0005] This invention solves the above problems by providing an information processing system with automatic analysis and proofreading functions that support various document formats. This system has the function to analyze existing documents and learn necessary items and formats, and provides templates when creating new documents. It also has the function to automatically compare documents of different formats and detect differences, and can automatically generate annotations based on legal requirements and guidelines and suggest their appropriate placement within the document. Furthermore, it is a system that supports efficient and accurate document creation by checking the legal requirements of the generated document in real time and issuing warnings.

[0006] "Existing documents" refer to written documents and papers that have already been created and are entered into the system for learning and analysis.

[0007] "Format" refers to the standards and frameworks related to the structure and layout of a document, and is an important element that affects the appearance and arrangement of a document.

[0008] A "template" refers to a basic framework or model used when creating a document, and is a fundamental document format for creating documents efficiently.

[0009] "Different formats" refers to the different structures and designs of various file formats, including various document formats such as PDF, Word, and Excel.

[0010] "Differences" refer to the elements or parts that differ between documents being compared, and primarily serve to indicate inconsistencies or changes in content.

[0011] An "annotation" is an explanation or comment added to a specific part of a document, and it is important information added based on legal requirements or guidelines.

[0012] "Legal requirements" refer to standards and conditions stipulated by laws and regulations, and function as rules that must be followed when creating documents.

[0013] An "information processing system" is a computer system that has a set of functions that provide value to the user through data input, processing, and output. [Brief explanation of the drawing]

[0014] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14]A sequence diagram showing the processing flow of a data processing system in Application Example 2 when combined with an emotion engine.

Embodiments for Carrying Out the Invention

[0015] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0018] In the following embodiments, a numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0019] In the following embodiments, a numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0020] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0022] [First Embodiment]

[0023] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0024] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0027] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0030] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0034] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0035] This invention provides a specific form for efficiently creating and reviewing documents in different formats using an information processing system that analyzes and learns from existing documents.

[0036] Document analysis and template generation

[0037] When the server receives an existing document uploaded by a user, it automatically analyzes its format and structure. This analysis process uses natural language processing technology to identify the necessary items and their placement within the document. Based on the analysis results, a template is generated and provided to the user's terminal. This template can be used intuitively when creating a new document, enabling efficient document creation.

[0038] Comparing documents in different formats

[0039] When a user uploads multiple files in different formats to the server, the server analyzes the files and extracts text data. This allows for comparison of content even across different formats such as PDFs and images. The server automatically detects differences in content and displays the results on the user's device. Users can easily check the differences on their device and accurately identify areas that need correction.

[0040] Annotation generation and legal verification

[0041] The server analyzes user-created advertisements and documents and automatically generates annotations based on relevant legal requirements and guidelines. These annotations are displayed on the device along with suggestions for their appropriate placement within the document. Users can then make necessary corrections and additions based on these suggestions. Furthermore, the system checks in real time whether the created document meets legal requirements and displays warnings on the device, thus preventing legal risks.

[0042] Multilingual support

[0043] This system features multilingual capabilities, supporting document creation for different cultural contexts. When a user selects a specific language, the server provides templates based on corresponding laws and guidelines. This enables users to accurately and efficiently create documents related to international contracts and transactions.

[0044] Thus, the present invention aims to provide a system for efficiently creating and proofreading documents in various formats, thereby reducing the workload on users while ensuring legal accuracy.

[0045] The following describes the processing flow.

[0046] Step 1:

[0047] Users upload existing documents to the server via their device. Uploaded documents can be in various formats, including Word and PDF.

[0048] Step 2:

[0049] The server analyzes the received document using natural language processing techniques. This process extracts the necessary items, formatting, and sentence structure from the document.

[0050] Step 3:

[0051] The server generates a document template based on the analysis results. This template includes items and formats extracted from existing documents.

[0052] Step 4:

[0053] The generated template is sent to the device, and the user begins creating a new document based on it. The template includes a suggestion feature to support more efficient document creation.

[0054] Step 5:

[0055] When a user uploads multiple files in different formats to the server via their device, the server converts those files into text data and performs analysis. Text can also be extracted from image files using OCR technology.

[0056] Step 6:

[0057] The server compares the analyzed text data and detects differences. The detected difference information is sent to the terminal and displayed visually to the user.

[0058] Step 7:

[0059] Users can view the differences on their devices, identify and correct the parts that need fixing. The corrections are saved to the server as needed.

[0060] Step 8:

[0061] The server analyzes the user's document and generates annotations based on legal requirements and guidelines. These annotations are displayed on the terminal along with suggestions for their appropriate placement within the document.

[0062] Step 9:

[0063] Users review the annotations and suggestions provided on their devices and make any necessary corrections or additions. This ensures that the document is legally compliant in terms of both format and content.

[0064] Step 10:

[0065] The server checks in real time whether the generated documents meet legal requirements and displays warnings on the terminal as needed. This allows users to mitigate legal risks.

[0066] Step 11:

[0067] When a user creates a multilingual document, the server provides an appropriate template based on the selected language. This helps ensure that documents are accurately created for different cultural contexts.

[0068] (Example 1)

[0069] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0070] Traditionally, efficiently creating, comparing, and reviewing documents in different formats required advanced expertise and manual adjustments. Furthermore, creating multilingual documents presented challenges in understanding and accurately reflecting the norms of each cultural sphere. Additionally, the process of verifying that documents met legal requirements was cumbersome, making real-time verification difficult.

[0071] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0072] In this invention, the server includes means for analyzing existing information to learn necessary elements and formats, means for generating instructions based on normative requirements and guidelines to suggest appropriate locations within the information, and means for multilingual support to assist in generating information for different social spheres. This enables users to efficiently create documents in different formats, verify legal requirements in real time, and generate international documents through multilingual support.

[0073] "Existing information" refers to documents and data created in the past that the system will analyze.

[0074] An "element" refers to an important part or component within a document or data, which is identified during analysis.

[0075] "Format" refers to the way documents and data are represented and their layout, and it forms the basis of the templates that a system learns and provides.

[0076] "Information in different formats" refers to different methods of representing information, such as PDFs, Word documents, and images.

[0077] "Difference" refers to differences in content or format between pieces of information being compared, and is something that the system detects.

[0078] "Normative requirements" refer to requirements stipulated in laws and guidelines, and are necessary to verify that a document meets these standards.

[0079] "Instructions" refer to advice and warnings generated based on normative requirements and guidelines.

[0080] "Appropriate location" refers to the point in the document where instructions or annotations should be added or modified, as suggested by the system.

[0081] "Social sphere" refers to the cultural and legal framework within a country or region, and should be considered when creating multilingual documents.

[0082] "Multilingual support" includes features that assist in creating and translating documents in different languages.

[0083] The information processing system implementing this invention is server-centric and, in cooperation with user terminals, streamlines the document creation and proofreading processes. When a user uploads a document from their terminal to the server, the server analyzes the received document and learns the necessary elements and format. For the analysis, natural language processing technology is applied, using Python's NLTK and SpaCy as specific libraries. This makes it possible to identify important components within a document and understand its format.

[0084] The server generates templates that can be used when creating new documents based on the analysis results and provides them to the terminal. Users can easily create documents using these templates. For example, when creating a contract, it is possible to upload an existing contract and receive a template based on its format.

[0085] Furthermore, if a user uploads multiple files in different formats, the server analyzes the files, extracts text data, and compares their contents. It can clearly detect differences in content even between files of different formats, such as PDF and Word documents. Libraries used for this purpose include PyPDF2 and Tesseract OCR.

[0086] The server also utilizes a generative AI model to generate instructions based on the legal requirements and guidelines of the document, suggesting their appropriate placement within the document. Users receive this information in real time and can verify whether the document meets legal standards. If there are legal issues, the server displays a warning on the terminal, allowing the user to take appropriate action.

[0087] Furthermore, this system features multilingual capabilities, supporting document creation for different social regions. When a user selects a specific language on their terminal, the server provides templates based on corresponding standards, enabling the accurate generation of international documents.

[0088] A concrete example of a prompt message is, "Analyze the uploaded business contract and add any necessary legal annotations." In this way, the present invention aims to provide a system for efficiently creating and reviewing documents of different formats, reducing the user's workload while ensuring legal accuracy.

[0089] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0090] Step 1:

[0091] The user uploads a document from their device to the server. An existing document file (PDF, Word, image, etc.) is used as input. The server verifies the file format in order to analyze the received document.

[0092] Step 2:

[0093] The server analyzes the received document and extracts the necessary elements and formatting. It uses Python's NLTK and SpaCy to tokenize the document and perform syntactic analysis. The input is the entire document file, and the output is data regarding the semantic structure of the items and sentences within the document. This data is used to prepare for the generation of a new template.

[0094] Step 3:

[0095] The server generates a template usable for document creation based on the analysis results. The template includes an arrangement based on the document's structure and format. The input is the item data extracted in step 2, and the output is the template provided to the user. The user receives the template on their terminal and can begin creating a new document.

[0096] Step 4:

[0097] Users upload files in different formats to the server. The server receives them and extracts text data using PyPDF2 or Tesseract OCR. The input consists of multiple files in different formats, and the output is the extracted text data. The server then prepares to compare the contents using this data.

[0098] Step 5:

[0099] The server compares the extracted text data and detects differences between documents. The input is the text data obtained in step 4, and the output is a report on the differences. The differences are displayed on the user's terminal, allowing the user to make appropriate corrections.

[0100] Step 6:

[0101] The server utilizes an AI model to generate instructions based on the legal requirements and guidelines of the document. The input is analyzed document data, and the output is annotation information. The annotations suggest appropriate locations within the document and are displayed on the user's device.

[0102] Step 7:

[0103] The server verifies whether the created document meets legal standards and provides real-time warnings. The input is the final document data, and the output is warning information regarding legal risks. The user reviews the warnings and adjusts the document as needed.

[0104] Step 8:

[0105] When a user selects a specific language, the server provides a template corresponding to that language. The input is the language information selected by the user, and the output is a multilingual template. This allows users to accurately create documents for different social contexts.

[0106] (Application Example 1)

[0107] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0108] In creating advertising documents, it is essential to efficiently analyze existing documents and easily adhere to the format and legal requirements of newly created documents. However, creating documents in different formats and languages ​​is prone to minor errors and legal non-compliance, resulting in wasted time and costs.

[0109] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0110] In this invention, the server includes means for analyzing existing documents to learn necessary items and formats, means for providing learned templates when creating new documents, and means for analyzing advertising documents to generate advertising templates. This enables efficient analysis and creation of advertising documents.

[0111] "Means for learning necessary items and formats by analyzing existing documents" refers to a technical process that analyzes uploaded documents using natural language processing technology, extracts important items and structures from the documents, and generates templates based on these.

[0112] "A means of providing learned templates when creating a new document" refers to a technology that utilizes information from analyzed documents to generate and provide templates as guidelines for format and structure when a user creates a new document.

[0113] "Methods for comparing multiple documents in different formats and detecting differences" refers to technologies that analyze the text data contained in documents of different formats and automatically identify and display the differences in content.

[0114] "Means for generating annotations based on legal requirements and guidelines and suggesting their appropriate placement within a document" refers to technology that analyzes document content, automatically generates annotations in accordance with relevant legal standards and guidelines, and suggests their placement.

[0115] "A means of verifying the legal requirements of generated documents and providing real-time warnings" refers to a technology that checks the content of created documents in real time and immediately warns of any parts that may violate laws and regulations.

[0116] "Methods for analyzing advertising documents and generating advertising templates" refers to technologies that analyze existing advertising documents and automatically create templates for new advertisements based on their constituent elements.

[0117] This invention is primarily implemented through an information processing system centered on a server, terminal, and user. Upon receiving an existing document from a user, the server analyzes the document's content using a natural language processing library (e.g., spaCy). This analysis includes identifying the document's structure and necessary items, and generating a template. The generated template is provided to the user's terminal and used when creating a new document.

[0118] The server also receives documents in multiple different formats from the user, analyzes them, and compares their contents. This comparison process detects the differences between documents and displays them on the terminal, making it easy for the user to understand what needs to be corrected.

[0119] Furthermore, when creating advertising documents, the server automatically generates annotations regarding the legal requirements that the document content must comply with and suggests appropriate placement on the device. This allows users to proactively prevent legal risks. In addition, through its multilingual support function, the server provides appropriate templates when creating documents for different cultural regions.

[0120] As a concrete example, when an advertising agency agent creates a new ad on a smartphone, they upload an existing ad document, and the server automatically analyzes that document and presents a new ad template. The user can then easily create a new ad based on that template. In this process, a prompt message such as "Please analyze this document and generate an ad template" is used.

[0121] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0122] Step 1:

[0123] The user uploads an existing document to the server. The server receives the uploaded document and parses its format and structure. The input to this process is the document data provided by the user, and the output is the document format and the identification of necessary items. A natural language processing library is used to analyze each element in the document and obtain the basic data for template generation.

[0124] Step 2:

[0125] The server generates a template based on the analysis results obtained in Step 1 and sends it to the terminal. The input is the analysis results, and the output is template data. The server automatically generates an intuitive template that allows the user to easily create documents using the analysis data.

[0126] Step 3:

[0127] When a user creates a new document, they utilize a template provided on the terminal. The user creates a new document based on the submitted template. At this stage, the input is template information, and the output is a completed new document. Using the editing functions on the terminal, the structure of the new document is quickly created according to the generated template.

[0128] Step 4:

[0129] The server receives multiple documents in different formats uploaded by users, analyzes them, and detects differences. The input is multiple documents in different formats, and the output is the differences between the documents. Text data is extracted from the documents, and a comparison algorithm is used to clarify the differences in content.

[0130] Step 5:

[0131] The server generates annotations regarding legal requirements for the created advertising document and suggests appropriate placement on the terminal. The input is the advertising document, and the output is annotations based on legal requirements and their placement suggestions. The server operates an annotation generation engine that reflects legal standards and adds appropriate annotation information to the document.

[0132] Step 6:

[0133] When users create documents for different cultural contexts, the server provides multilingual templates. The input is the user's language selection information, and the output is a template corresponding to the specific language. The server uses a language database to generate templates that apply guidelines based on the selected language.

[0134] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0135] This invention optimizes document creation and editing processes based on the user's emotional state by combining an emotion engine with an existing information processing system, and embodiments thereof will be described below.

[0136] Emotion recognition and interface adjustment

[0137] The server analyzes the user's emotions in real time through the device's camera and voice input device as the user creates or proofreads documents. This analysis is performed using an emotion engine, which identifies the user's emotions from data such as facial expressions, voice tone, and typing speed. Based on the recognized emotions, the device adjusts its interface and suggestion functions to support the user in continuing their work most effectively.

[0138] Using emotional history

[0139] The server stores user sentiment data for a certain period and personalizes document creation templates and annotation suggestions based on past sentiment history. This feature allows the device to provide more appropriate suggestions that take into account the user's past sentiment tendencies.

[0140] Real-time sentiment analysis and adjustment

[0141] When a user's emotions change while they are creating a document, the server immediately analyzes this and makes adjustments accordingly. For example, if it detects that the user is feeling stressed, the device can increase the frequency of suggestions or change to a more relaxing interface. These adjustments aim to improve user efficiency and satisfaction.

[0142] Specific example

[0143] When a user is creating a legal document, if the device detects the user's confused expression, the emotion engine analyzes the emotion and reports the increased stress level to the server. In response, the server begins providing step-by-step guidance based on templates, enhancing automatic completion and advice on legal requirements. On the other hand, if the user is working calmly, the system optimizes the work environment by reducing the number of suggestions or gently changing the interface's color scheme.

[0144] These features allow the present invention to support efficient and accurate document creation while recognizing the user's emotions. As a result, users can reduce work stress and create higher-quality documents.

[0145] The following describes the processing flow.

[0146] Step 1:

[0147] The user begins creating or proofreading a document via the device. At this time, the device starts recording the user's facial expressions and voice data using sensors such as a camera and microphone.

[0148] Step 2:

[0149] The device transmits sensor data it records to the server in real time. The transmitted data includes the user's facial movements and voice tone.

[0150] Step 3:

[0151] The server analyzes the received data using an emotion engine to identify the user's current emotional state. This analysis uses natural language processing and image recognition technologies to quantify the stress, anxiety, concentration, and other emotions the user is likely experiencing.

[0152] Step 4:

[0153] Based on the analysis results, the server generates a plan to design a document environment that is appropriate for the user's emotional state. The plan includes methods for modifying the interface and adjusting the suggestion function.

[0154] Step 5:

[0155] Based on the created plan, the device will modify the interface in response to the user's emotions. These modifications may include adjusting the color scheme and rearranging UI elements.

[0156] Step 6:

[0157] Simultaneously, the device dynamically adjusts its suggestion function based on information obtained from the server. For example, if the user is confused, it provides more detailed guides and frequent suggestions; conversely, if the user is focused, it minimizes suggestions.

[0158] Step 7:

[0159] As the user continues working, emotional data is collected again and sent to the server. This process is repeated in real time, and the server constantly provides feedback based on the most up-to-date emotions.

[0160] Step 8:

[0161] After the task is completed, the server analyzes the accumulated emotional data and provides feedback to the terminal that visualizes the user's emotional fluctuations. This allows the user to reflect on their emotional state during the task and use that feedback to improve their next task.

[0162] (Example 2)

[0163] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0164] In modern information processing systems, there is a growing need to create documents efficiently and accurately, but a lack of consideration for the emotional state of users often leads to stressful situations. Furthermore, when documents are created in diverse cultural contexts and languages, standard templates and guidelines are insufficient. There is a need to address these challenges and provide information processing systems that enable users to create documents more comfortably and efficiently.

[0165] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0166] In this invention, the server includes means for analyzing existing information to learn necessary items and formats, means for comparing multiple different formats of information to detect differences, and means for analyzing emotional states based on emotional data. This enables the adjustment of the interface according to the user's emotional state and multilingual support according to cultural background and language.

[0167] "Information" is a collection of data that is recognized as having value and meaning through organization and analysis.

[0168] "Format" refers to the specific methods and patterns by which information and data are put together, and it describes the structure of a document or its content.

[0169] A "template" is a standardized framework or template used when creating a specific type of information.

[0170] "Difference" refers to the differences or distinct characteristics found between two or more pieces of information or data that are being compared.

[0171] "Emotional data" refers to related information such as facial expressions and voice that is collected to identify the emotional state of the user.

[0172] "Analysis" is the operation or process of breaking down information or data and investigating its properties in detail for better understanding.

[0173] An "interface" refers to the means, such as screens, operating methods, and input devices, that a user uses to interact with a system.

[0174] "Multilingual support" refers to a function that assists in creating and processing information in the appropriate language in situations where different languages ​​are used.

[0175] This invention optimizes document creation in an information processing system according to the user's emotional state. It primarily utilizes a terminal, a server, and an emotion engine.

[0176] Hardware and software configuration

[0177] The device is equipped with a camera and microphone to capture the user's facial expressions and voice in real time. This data is sent to a server, where an emotion engine analyzes it. The emotion engine uses a generative AI model to identify the user's emotions, which are then used to adjust the interface.

[0178] The server is responsible for changing interface colors and adjusting suggestion functions based on emotion data from the emotion engine. Furthermore, the server stores the user's emotion history and customizes document creation templates based on past emotions.

[0179] Specific example

[0180] If a user appears confused while creating a legal document, the device captures their facial expression with its camera and sends it to the server. The server analyzes this data using an emotion engine, and if it determines that the user is experiencing stress, it provides the user with a template guide. This template guide supports the user by displaying step-by-step instructions and presenting the specific requirements of legal documents.

[0181] Example of a prompt

[0182] "How can we instantly enhance template guides if users experience stress while creating legal documents?"

[0183] This system provides advanced support to ensure users can continue working comfortably, while also offering an efficient document creation environment.

[0184] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0185] Step 1:

[0186] The device acquires the user's face and voice data. Specifically, it captures the user's facial expressions with a camera and records their voice tone with a microphone. This input data is sent to the server in real time.

[0187] Step 2:

[0188] The server passes the facial and voice data received from the terminal to the emotion engine. The emotion engine uses a generative AI model to analyze the data and identify the user's emotional state. In this process, it analyzes facial features and phonemes from the input data and outputs an emotion label.

[0189] Step 3:

[0190] The server adjusts the interface based on emotion labels from the emotion engine. Specifically, if the entered emotion label is "stress," the device presents a more relaxing interface to the user and enhances the template guide. The user experience is improved by changing the screen's color scheme to blue and increasing the frequency of suggestion features.

[0191] Step 4:

[0192] The server stores user sentiment data and interface adjustment history in a database. Based on this stored history, it is possible to personalize templates and suggestions that will be useful when creating documents later. Input data includes sentiment labels and their corresponding actions, which are then used to generate optimized output in subsequent sessions.

[0193] (Application Example 2)

[0194] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0195] Modern information processing systems lack sufficient support for users to efficiently create and review documents. In particular, there is a need for interface adjustments and optimization of suggestion functions that reflect the user's emotional state during document creation. Furthermore, features for purchasing support and spending management based on user emotions are also lacking, making it difficult to provide personalized services that meet user needs.

[0196] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0197] In this invention, the server includes means for analyzing existing documents to learn necessary items and formats, means for analyzing the user's emotional state and adjusting the interface and suggestion functions based on that emotion, and means for providing expenditure management functions based on emotion analysis to support the user's purchasing decisions. This enables appropriate interface adjustments and suggestions according to the user's emotional state, resulting in efficient and personalized document creation and purchasing support.

[0198] A "document" is a collection of text and data used to organize and represent information.

[0199] "Emotional state" refers to the state of a user's psychological and mental reactions and mood, and is analyzed through facial expressions, tone of voice, input speed, etc.

[0200] An "interface" is a point of contact or layout that allows a user and an information processing system to interact, and is designed to make it easier for users to access the system.

[0201] A "suggestion function" is a feature provided to assist users in their operations by offering relevant information and advice on the next course of action.

[0202] The "spending management function" is a feature that analyzes the user's spending habits and financial situation, and provides advice and warnings to optimize spending.

[0203] This invention relates to an information processing system that analyzes a user's emotional state and adjusts the interface and suggested functions based on this analysis. The server analyzes the user's facial expressions, voice tone, and input speed through terminal cameras such as smartphones and voice input devices, and uses an emotion engine to identify the user's emotional state from this data. The emotional data obtained through the analysis is processed in real time using an appropriate algorithm.

[0204] In terms of hardware, a camera is used to analyze the user's facial expressions, and a microphone is used to analyze the tone of their voice. In terms of software, an image processing library such as OpenCV is used for facial expression analysis, and a specific emotion analysis model such as EmotionEngine is used for emotion analysis. This allows for interface adjustments depending on whether the user is stressed or relaxed.

[0205] As a concrete example, consider a situation where a user is online shopping. If the device detects that the user is relaxed, the server will improve the user's shopping experience by setting the interface's color scheme to a more subdued tone and refraining from providing spending management advice based on purchase history. Conversely, if the server detects that the user is stressed, it will frequently display suggestions to encourage careful spending.

[0206] An example of a prompt for a generative AI model is, "How can we provide a more enjoyable shopping experience when the user is relaxed?" This enables appropriate user support based on their emotions.

[0207] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0208] Step 1:

[0209] The device collects the user's facial expressions and voice. It receives facial image data from the smartphone's camera and audio data from the microphone as input. The device then prepares to send this data to the server.

[0210] Step 2:

[0211] The server receives data from the camera and microphone and performs emotion analysis using EmotionEngine. It uses facial image data and audio data as input, and identifies the user's emotional state (relaxed, stressed, etc.) as output emotion data.

[0212] Step 3:

[0213] The server adjusts the interface based on the user's emotional state. It uses emotional data as input and generates settings to optimize the interface's color scheme and layout as output. The server uses warm colors when the user is relaxed and suggests a simpler design when the user is stressed.

[0214] Step 4:

[0215] The server adjusts its suggestion function based on the user's emotional state. It uses emotional data as input and determines appropriate suggestions (product recommendations, purchase advice, etc.) as output. When the user is emotionally calm, it makes subtle suggestions, while when they are stressed, it emphasizes specific spending advice.

[0216] Step 5:

[0217] The user receives interface settings and suggestions sent from the server. The terminal displays this to the user and prepares to assist the user's actions. This allows the user to experience a personalized interface and make appropriate purchasing decisions.

[0218] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0219] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0220] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0221] [Second Embodiment]

[0222] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0223] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0224] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0225] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0226] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0227] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0228] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0229] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0230] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0231] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0232] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0233] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0234] This invention provides a specific form for efficiently creating and reviewing documents in different formats using an information processing system that analyzes and learns from existing documents.

[0235] Document analysis and template generation

[0236] When the server receives an existing document uploaded by a user, it automatically analyzes its format and structure. This analysis process uses natural language processing technology to identify the necessary items and their placement within the document. Based on the analysis results, a template is generated and provided to the user's terminal. This template can be used intuitively when creating a new document, enabling efficient document creation.

[0237] Comparing documents in different formats

[0238] When a user uploads multiple files in different formats to the server, the server analyzes the files and extracts text data. This allows for comparison of content even across different formats such as PDFs and images. The server automatically detects differences in content and displays the results on the user's device. Users can easily check the differences on their device and accurately identify areas that need correction.

[0239] Annotation generation and legal verification

[0240] The server analyzes user-created advertisements and documents and automatically generates annotations based on relevant legal requirements and guidelines. These annotations are displayed on the device along with suggestions for their appropriate placement within the document. Users can then make necessary corrections and additions based on these suggestions. Furthermore, the system checks in real time whether the created document meets legal requirements and displays warnings on the device, thus preventing legal risks.

[0241] Multilingual support

[0242] This system features multilingual capabilities, supporting document creation for different cultural contexts. When a user selects a specific language, the server provides templates based on corresponding laws and guidelines. This enables users to accurately and efficiently create documents related to international contracts and transactions.

[0243] Thus, the present invention aims to provide a system for efficiently creating and proofreading documents in various formats, thereby reducing the workload on users while ensuring legal accuracy.

[0244] The following describes the processing flow.

[0245] Step 1:

[0246] Users upload existing documents to the server via their device. Uploaded documents can be in various formats, including Word and PDF.

[0247] Step 2:

[0248] The server analyzes the received document using natural language processing techniques. This process extracts the necessary items, formatting, and sentence structure from the document.

[0249] Step 3:

[0250] The server generates a document template based on the analysis results. This template includes items and formats extracted from existing documents.

[0251] Step 4:

[0252] The generated template is sent to the device, and the user begins creating a new document based on it. The template includes a suggestion feature to support more efficient document creation.

[0253] Step 5:

[0254] When a user uploads multiple files in different formats to the server via their device, the server converts those files into text data and performs analysis. Text can also be extracted from image files using OCR technology.

[0255] Step 6:

[0256] The server compares the analyzed text data and detects differences. The detected difference information is sent to the terminal and displayed visually to the user.

[0257] Step 7:

[0258] Users can view the differences on their devices, identify and correct the parts that need fixing. The corrections are saved to the server as needed.

[0259] Step 8:

[0260] The server analyzes the user's document and generates annotations based on legal requirements and guidelines. These annotations are displayed on the terminal along with suggestions for their appropriate placement within the document.

[0261] Step 9:

[0262] Users review the annotations and suggestions provided on their devices and make any necessary corrections or additions. This ensures that the document is legally compliant in terms of both format and content.

[0263] Step 10:

[0264] The server checks in real time whether the generated documents meet legal requirements and displays warnings on the terminal as needed. This allows users to mitigate legal risks.

[0265] Step 11:

[0266] When a user creates a multilingual document, the server provides an appropriate template based on the selected language. This helps ensure that documents are accurately created for different cultural contexts.

[0267] (Example 1)

[0268] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0269] Traditionally, efficiently creating, comparing, and reviewing documents in different formats required advanced expertise and manual adjustments. Furthermore, creating multilingual documents presented challenges in understanding and accurately reflecting the norms of each cultural sphere. Additionally, the process of verifying that documents met legal requirements was cumbersome, making real-time verification difficult.

[0270] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0271] In this invention, the server includes means for analyzing existing information to learn necessary elements and formats, means for generating instructions based on normative requirements and guidelines to suggest appropriate locations within the information, and means for multilingual support to assist in generating information for different social spheres. This enables users to efficiently create documents in different formats, verify legal requirements in real time, and generate international documents through multilingual support.

[0272] "Existing information" refers to documents and data created in the past that the system will analyze.

[0273] An "element" refers to an important part or component within a document or data, which is identified during analysis.

[0274] "Format" refers to the way documents and data are represented and their layout, and it forms the basis of the templates that a system learns and provides.

[0275] "Information in different formats" refers to different methods of representing information, such as PDFs, Word documents, and images.

[0276] "Difference" refers to differences in content or format between pieces of information being compared, and is something that the system detects.

[0277] "Normative requirements" refer to requirements stipulated in laws and guidelines, and are necessary to verify that a document meets these standards.

[0278] "Instructions" refer to advice and warnings generated based on normative requirements and guidelines.

[0279] "Appropriate location" refers to the point in the document where instructions or annotations should be added or modified, as suggested by the system.

[0280] "Social sphere" refers to the cultural and legal framework within a country or region, and should be considered when creating multilingual documents.

[0281] "Multilingual support" includes features that assist in creating and translating documents in different languages.

[0282] The information processing system implementing this invention is server-centric and, in cooperation with user terminals, streamlines the document creation and proofreading processes. When a user uploads a document from their terminal to the server, the server analyzes the received document and learns the necessary elements and format. For the analysis, natural language processing technology is applied, using Python's NLTK and SpaCy as specific libraries. This makes it possible to identify important components within a document and understand its format.

[0283] The server generates templates that can be used when creating new documents based on the analysis results and provides them to the terminal. Users can easily create documents using these templates. For example, when creating a contract, it is possible to upload an existing contract and receive a template based on its format.

[0284] Furthermore, when the user uploads multiple files in different formats, the server analyzes the files to extract text data and compares the contents. Even between different formats, such as PDF and Word files, differences in content can be clearly detected. Libraries such as PyPDF2 and Tesseract OCR are used for this purpose.

[0285] The server also utilizes a generative AI model to generate instructions based on legal requirements and guidelines for the document and propose appropriate positions within the document. The user can receive this information in real time and check whether the document meets legal standards. If there are legal issues, a warning is displayed from the server to the terminal, and the user can take appropriate action.

[0286] Furthermore, this system has a multilingual support function and supports document creation for different communities. When the user selects a specific language on the terminal, the server provides a template based on the corresponding norms and can accurately generate international documents.

[0287] A specific example of the prompt text is "Analyze the uploaded business contract and add necessary legal annotations." In this way, the present invention aims to provide a system for efficiently creating and reviewing documents in different formats, reducing the user's workload while ensuring legal accuracy.

[0288] The flow of the specific process in Example 1 will be described using FIG. 11.

[0289] Step 1:

[0290] The user uploads a document from the terminal to the server. As input, existing document files (such as PDF, Word, images, etc.) are used. The server checks the format of the file in order to analyze the received document.

[0291] Step 2:

[0292] The server analyzes the received document and extracts the necessary elements and formatting. It uses Python's NLTK and SpaCy to tokenize the document and perform syntactic analysis. The input is the entire document file, and the output is data regarding the semantic structure of the items and sentences within the document. This data is used to prepare for the generation of a new template.

[0293] Step 3:

[0294] The server generates a template usable for document creation based on the analysis results. The template includes an arrangement based on the document's structure and format. The input is the item data extracted in step 2, and the output is the template provided to the user. The user receives the template on their terminal and can begin creating a new document.

[0295] Step 4:

[0296] Users upload files in different formats to the server. The server receives them and extracts text data using PyPDF2 or Tesseract OCR. The input consists of multiple files in different formats, and the output is the extracted text data. The server then prepares to compare the contents using this data.

[0297] Step 5:

[0298] The server compares the extracted text data and detects differences between documents. The input is the text data obtained in step 4, and the output is a report on the differences. The differences are displayed on the user's terminal, allowing the user to make appropriate corrections.

[0299] Step 6:

[0300] The server utilizes a generated AI model to produce instructions based on the legal requirements and guidelines of the document. The input is analyzed document data, and the output is annotation information. The annotations suggest the appropriate location within the document and are displayed on the user's terminal.

[0301] Step 7:

[0302] The server checks whether the created document meets legal standards and issues real-time warnings. The input is the final document data, and the output is warning information regarding legal risks. The user checks the warning content and adjusts the document as necessary.

[0303] Step 8:

[0304] When the user selects a specific language, the server provides a template corresponding to that language. The input is the language information selected by the user, and the output is a multi-language compatible template. This enables the user to accurately create documents for different communities.

[0305] (Application Example 1)

[0306] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0307] In creating advertising documents, it is required to efficiently analyze existing documents and easily comply with the formats and legal requirements of newly created documents. However, in creating documents in different formats and languages, minute errors and legal non-compliance are likely to occur, resulting in a problem of wasted time and cost.

[0308] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0309] In this invention, the server includes means for analyzing existing documents to learn necessary items and formats, means for providing the learned template when creating a new document, and means for analyzing advertising documents to generate an advertising template. This enables efficient analysis and creation of advertising documents.

[0310] "Means for learning necessary items and formats by analyzing existing documents" refers to a technical process that analyzes uploaded documents using natural language processing technology, extracts important items and structures from the documents, and generates templates based on these.

[0311] "A means of providing learned templates when creating a new document" refers to a technology that utilizes information from analyzed documents to generate and provide templates as guidelines for format and structure when a user creates a new document.

[0312] "Methods for comparing multiple documents in different formats and detecting differences" refers to technologies that analyze the text data contained in documents of different formats and automatically identify and display the differences in content.

[0313] "Means for generating annotations based on legal requirements and guidelines and suggesting their appropriate placement within a document" refers to technology that analyzes document content, automatically generates annotations in accordance with relevant legal standards and guidelines, and suggests their placement.

[0314] "A means of verifying the legal requirements of generated documents and providing real-time warnings" refers to a technology that checks the content of created documents in real time and immediately warns of any parts that may violate laws and regulations.

[0315] "Methods for analyzing advertising documents and generating advertising templates" refers to technologies that analyze existing advertising documents and automatically create templates for new advertisements based on their constituent elements.

[0316] This invention is primarily implemented through an information processing system centered on a server, terminal, and user. Upon receiving an existing document from a user, the server analyzes the document's content using a natural language processing library (e.g., spaCy). This analysis includes identifying the document's structure and necessary items, and generating a template. The generated template is provided to the user's terminal and used when creating a new document.

[0317] The server also receives documents in multiple different formats from the user, analyzes them, and compares their contents. This comparison process detects the differences between documents and displays them on the terminal, making it easy for the user to understand what needs to be corrected.

[0318] Furthermore, when creating advertising documents, the server automatically generates annotations regarding the legal requirements that the document content must comply with and suggests appropriate placement on the device. This allows users to proactively prevent legal risks. In addition, through its multilingual support function, the server provides appropriate templates when creating documents for different cultural regions.

[0319] As a concrete example, when an advertising agency agent creates a new ad on a smartphone, they upload an existing ad document, and the server automatically analyzes that document and presents a new ad template. The user can then easily create a new ad based on that template. In this process, a prompt message such as "Please analyze this document and generate an ad template" is used.

[0320] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0321] Step 1:

[0322] The user uploads an existing document to the server. The server receives the uploaded document and parses its format and structure. The input to this process is the document data provided by the user, and the output is the document format and the identification of necessary items. A natural language processing library is used to analyze each element in the document and obtain the basic data for template generation.

[0323] Step 2:

[0324] The server generates a template based on the analysis results obtained in Step 1 and sends it to the terminal. The input is the analysis results, and the output is template data. The server automatically generates an intuitive template that allows the user to easily create documents using the analysis data.

[0325] Step 3:

[0326] When a user creates a new document, they utilize a template provided on the terminal. The user creates a new document based on the submitted template. At this stage, the input is template information, and the output is a completed new document. Using the editing functions on the terminal, the structure of the new document is quickly created according to the generated template.

[0327] Step 4:

[0328] The server receives multiple documents in different formats uploaded by users, analyzes them, and detects differences. The input is multiple documents in different formats, and the output is the differences between the documents. Text data is extracted from the documents, and a comparison algorithm is used to clarify the differences in content.

[0329] Step 5:

[0330] The server generates annotations regarding legal requirements for the created advertising document and suggests appropriate placement on the terminal. The input is the advertising document, and the output is annotations based on legal requirements and their placement suggestions. The server operates an annotation generation engine that reflects legal standards and adds appropriate annotation information to the document.

[0331] Step 6:

[0332] When users create documents for different cultural contexts, the server provides multilingual templates. The input is the user's language selection information, and the output is a template corresponding to the specific language. The server uses a language database to generate templates that apply guidelines based on the selected language.

[0333] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0334] This invention optimizes document creation and editing processes based on the user's emotional state by combining an emotion engine with an existing information processing system, and embodiments thereof will be described below.

[0335] Emotion recognition and interface adjustment

[0336] The server analyzes the user's emotions in real time through the device's camera and voice input device as the user creates or proofreads documents. This analysis is performed using an emotion engine, which identifies the user's emotions from data such as facial expressions, voice tone, and typing speed. Based on the recognized emotions, the device adjusts its interface and suggestion functions to support the user in continuing their work most effectively.

[0337] Using emotional history

[0338] The server stores user sentiment data for a certain period and personalizes document creation templates and annotation suggestions based on past sentiment history. This feature allows the device to provide more appropriate suggestions that take into account the user's past sentiment tendencies.

[0339] Real-time sentiment analysis and adjustment

[0340] When a user's emotions change while they are creating a document, the server immediately analyzes this and makes adjustments accordingly. For example, if it detects that the user is feeling stressed, the device can increase the frequency of suggestions or change to a more relaxing interface. These adjustments aim to improve user efficiency and satisfaction.

[0341] Specific example

[0342] When a user is creating a legal document, if the device detects the user's confused expression, the emotion engine analyzes the emotion and reports the increased stress level to the server. In response, the server begins providing step-by-step guidance based on templates, enhancing automatic completion and advice on legal requirements. On the other hand, if the user is working calmly, the system optimizes the work environment by reducing the number of suggestions or gently changing the interface's color scheme.

[0343] These features allow the present invention to support efficient and accurate document creation while recognizing the user's emotions. As a result, users can reduce work stress and create higher-quality documents.

[0344] The following describes the processing flow.

[0345] Step 1:

[0346] The user begins creating or proofreading a document via the device. At this time, the device starts recording the user's facial expressions and voice data using sensors such as a camera and microphone.

[0347] Step 2:

[0348] The device transmits sensor data it records to the server in real time. The transmitted data includes the user's facial movements and voice tone.

[0349] Step 3:

[0350] The server analyzes the received data using an emotion engine to identify the user's current emotional state. This analysis uses natural language processing and image recognition technologies to quantify the stress, anxiety, concentration, and other emotions the user is likely experiencing.

[0351] Step 4:

[0352] Based on the analysis results, the server generates a plan to design a document environment that is appropriate for the user's emotional state. The plan includes methods for modifying the interface and adjusting the suggestion function.

[0353] Step 5:

[0354] Based on the created plan, the device will modify the interface in response to the user's emotions. These modifications may include adjusting the color scheme and rearranging UI elements.

[0355] Step 6:

[0356] Simultaneously, the device dynamically adjusts its suggestion function based on information obtained from the server. For example, if the user is confused, it provides more detailed guides and frequent suggestions; conversely, if the user is focused, it minimizes suggestions.

[0357] Step 7:

[0358] As the user continues working, emotional data is collected again and sent to the server. This process is repeated in real time, and the server constantly provides feedback based on the most up-to-date emotions.

[0359] Step 8:

[0360] After the task is completed, the server analyzes the accumulated emotional data and provides feedback to the terminal that visualizes the user's emotional fluctuations. This allows the user to reflect on their emotional state during the task and use that feedback to improve their next task.

[0361] (Example 2)

[0362] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0363] In modern information processing systems, there is a growing need to create documents efficiently and accurately, but a lack of consideration for the emotional state of users often leads to stressful situations. Furthermore, when documents are created in diverse cultural contexts and languages, standard templates and guidelines are insufficient. There is a need to address these challenges and provide information processing systems that enable users to create documents more comfortably and efficiently.

[0364] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0365] In this invention, the server includes means for analyzing existing information to learn necessary items and formats, means for comparing multiple different formats of information to detect differences, and means for analyzing emotional states based on emotional data. This enables the adjustment of the interface according to the user's emotional state and multilingual support according to cultural background and language.

[0366] "Information" is a collection of data that is recognized as having value and meaning through organization and analysis.

[0367] "Format" refers to the specific methods and patterns by which information and data are put together, and it describes the structure of a document or its content.

[0368] A "template" is a standardized framework or template used when creating a specific type of information.

[0369] "Difference" refers to the differences or distinct characteristics found between two or more pieces of information or data that are being compared.

[0370] "Emotional data" refers to related information such as facial expressions and voice that is collected to identify the emotional state of the user.

[0371] "Analysis" is the operation or process of breaking down information or data and investigating its properties in detail for better understanding.

[0372] An "interface" refers to the means, such as screens, operating methods, and input devices, that a user uses to interact with a system.

[0373] "Multilingual support" refers to a function that assists in creating and processing information in the appropriate language in situations where different languages ​​are used.

[0374] This invention optimizes document creation in an information processing system according to the user's emotional state. It primarily utilizes a terminal, a server, and an emotion engine.

[0375] Hardware and software configuration

[0376] The device is equipped with a camera and microphone to capture the user's facial expressions and voice in real time. This data is sent to a server, where an emotion engine analyzes it. The emotion engine uses a generative AI model to identify the user's emotions, which are then used to adjust the interface.

[0377] The server is responsible for changing interface colors and adjusting suggestion functions based on emotion data from the emotion engine. Furthermore, the server stores the user's emotion history and customizes document creation templates based on past emotions.

[0378] Specific example

[0379] If a user appears confused while creating a legal document, the device captures their facial expression with its camera and sends it to the server. The server analyzes this data using an emotion engine, and if it determines that the user is experiencing stress, it provides the user with a template guide. This template guide supports the user by displaying step-by-step instructions and presenting the specific requirements of legal documents.

[0380] Example of a prompt

[0381] "How can we instantly enhance template guides if users experience stress while creating legal documents?"

[0382] This system provides advanced support to ensure users can continue working comfortably, while also offering an efficient document creation environment.

[0383] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0384] Step 1:

[0385] The device acquires the user's face and voice data. Specifically, it captures the user's facial expressions with a camera and records their voice tone with a microphone. This input data is sent to the server in real time.

[0386] Step 2:

[0387] The server passes the facial and voice data received from the terminal to the emotion engine. The emotion engine uses a generative AI model to analyze the data and identify the user's emotional state. In this process, it analyzes facial features and phonemes from the input data and outputs an emotion label.

[0388] Step 3:

[0389] The server adjusts the interface based on emotion labels from the emotion engine. Specifically, if the entered emotion label is "stress," the device presents a more relaxing interface to the user and enhances the template guide. The user experience is improved by changing the screen's color scheme to blue and increasing the frequency of suggestion features.

[0390] Step 4:

[0391] The server stores user sentiment data and interface adjustment history in a database. Based on this stored history, it is possible to personalize templates and suggestions that will be useful when creating documents later. Input data includes sentiment labels and their corresponding actions, which are then used to generate optimized output in subsequent sessions.

[0392] (Application Example 2)

[0393] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0394] Modern information processing systems lack sufficient support for users to efficiently create and review documents. In particular, there is a need for interface adjustments and optimization of suggestion functions that reflect the user's emotional state during document creation. Furthermore, features for purchasing support and spending management based on user emotions are also lacking, making it difficult to provide personalized services that meet user needs.

[0395] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0396] In this invention, the server includes means for analyzing existing documents to learn necessary items and formats, means for analyzing the user's emotional state and adjusting the interface and suggestion functions based on that emotion, and means for providing expenditure management functions based on emotion analysis to support the user's purchasing decisions. This enables appropriate interface adjustments and suggestions according to the user's emotional state, resulting in efficient and personalized document creation and purchasing support.

[0397] A "document" is a collection of text and data used to organize and represent information.

[0398] "Emotional state" refers to the state of a user's psychological and mental reactions and mood, and is analyzed through facial expressions, tone of voice, input speed, etc.

[0399] An "interface" is a point of contact or layout that allows a user and an information processing system to interact, and is designed to make it easier for users to access the system.

[0400] A "suggestion function" is a feature provided to assist users in their operations by offering relevant information and advice on the next course of action.

[0401] The "spending management function" is a feature that analyzes the user's spending habits and financial situation, and provides advice and warnings to optimize spending.

[0402] This invention relates to an information processing system that analyzes a user's emotional state and adjusts the interface and suggested functions based on this analysis. The server analyzes the user's facial expressions, voice tone, and input speed through terminal cameras such as smartphones and voice input devices, and uses an emotion engine to identify the user's emotional state from this data. The emotional data obtained through the analysis is processed in real time using an appropriate algorithm.

[0403] In terms of hardware, a camera is used to analyze the user's facial expressions, and a microphone is used to analyze the tone of their voice. In terms of software, an image processing library such as OpenCV is used for facial expression analysis, and a specific emotion analysis model such as EmotionEngine is used for emotion analysis. This allows for interface adjustments depending on whether the user is stressed or relaxed.

[0404] As a concrete example, consider a situation where a user is online shopping. If the device detects that the user is relaxed, the server will improve the user's shopping experience by setting the interface's color scheme to a more subdued tone and refraining from providing spending management advice based on purchase history. Conversely, if the server detects that the user is stressed, it will frequently display suggestions to encourage careful spending.

[0405] An example of a prompt for a generative AI model is, "How can we provide a more enjoyable shopping experience when the user is relaxed?" This enables appropriate user support based on their emotions.

[0406] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0407] Step 1:

[0408] The device collects the user's facial expressions and voice. It receives facial image data from the smartphone's camera and audio data from the microphone as input. The device then prepares to send this data to the server.

[0409] Step 2:

[0410] The server receives data from the camera and microphone and performs emotion analysis using EmotionEngine. It uses facial image data and audio data as input, and identifies the user's emotional state (relaxed, stressed, etc.) as output emotion data.

[0411] Step 3:

[0412] The server adjusts the interface based on the user's emotional state. It uses emotional data as input and generates settings to optimize the interface's color scheme and layout as output. The server uses warm colors when the user is relaxed and suggests a simpler design when the user is stressed.

[0413] Step 4:

[0414] The server adjusts its suggestion function based on the user's emotional state. It uses emotional data as input and determines appropriate suggestions (product recommendations, purchase advice, etc.) as output. When the user is emotionally calm, it makes subtle suggestions, while when they are stressed, it emphasizes specific spending advice.

[0415] Step 5:

[0416] The user receives interface settings and suggestions sent from the server. The terminal displays this to the user and prepares to assist the user's actions. This allows the user to experience a personalized interface and make appropriate purchasing decisions.

[0417] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0418] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0419] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0420] [Third Embodiment]

[0421] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0422] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0423] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0424] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0425] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0426] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0427] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0428] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0429] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0430] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0431] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0432] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0433] This invention provides a specific form for efficiently creating and reviewing documents in different formats using an information processing system that analyzes and learns from existing documents.

[0434] Document analysis and template generation

[0435] When the server receives an existing document uploaded by a user, it automatically analyzes its format and structure. This analysis process uses natural language processing technology to identify the necessary items and their placement within the document. Based on the analysis results, a template is generated and provided to the user's terminal. This template can be used intuitively when creating a new document, enabling efficient document creation.

[0436] Comparing documents in different formats

[0437] When a user uploads multiple files in different formats to the server, the server analyzes the files and extracts text data. This allows for comparison of content even across different formats such as PDFs and images. The server automatically detects differences in content and displays the results on the user's device. Users can easily check the differences on their device and accurately identify areas that need correction.

[0438] Annotation generation and legal verification

[0439] The server analyzes user-created advertisements and documents and automatically generates annotations based on relevant legal requirements and guidelines. These annotations are displayed on the device along with suggestions for their appropriate placement within the document. Users can then make necessary corrections and additions based on these suggestions. Furthermore, the system checks in real time whether the created document meets legal requirements and displays warnings on the device, thus preventing legal risks.

[0440] Multilingual support

[0441] This system features multilingual capabilities, supporting document creation for different cultural contexts. When a user selects a specific language, the server provides templates based on corresponding laws and guidelines. This enables users to accurately and efficiently create documents related to international contracts and transactions.

[0442] Thus, the present invention aims to provide a system for efficiently creating and proofreading documents in various formats, thereby reducing the workload on users while ensuring legal accuracy.

[0443] The following describes the processing flow.

[0444] Step 1:

[0445] Users upload existing documents to the server via their device. Uploaded documents can be in various formats, including Word and PDF.

[0446] Step 2:

[0447] The server analyzes the received document using natural language processing techniques. This process extracts the necessary items, formatting, and sentence structure from the document.

[0448] Step 3:

[0449] The server generates a document template based on the analysis results. This template includes items and formats extracted from existing documents.

[0450] Step 4:

[0451] The generated template is sent to the device, and the user begins creating a new document based on it. The template includes a suggestion feature to support more efficient document creation.

[0452] Step 5:

[0453] When a user uploads multiple files in different formats to the server via their device, the server converts those files into text data and performs analysis. Text can also be extracted from image files using OCR technology.

[0454] Step 6:

[0455] The server compares the analyzed text data and detects differences. The detected difference information is sent to the terminal and displayed visually to the user.

[0456] Step 7:

[0457] Users can view the differences on their devices, identify and correct the parts that need fixing. The corrections are saved to the server as needed.

[0458] Step 8:

[0459] The server analyzes the user's document and generates annotations based on legal requirements and guidelines. These annotations are displayed on the terminal along with suggestions for their appropriate placement within the document.

[0460] Step 9:

[0461] Users review the annotations and suggestions provided on their devices and make any necessary corrections or additions. This ensures that the document is legally compliant in terms of both format and content.

[0462] Step 10:

[0463] The server checks in real time whether the generated documents meet legal requirements and displays warnings on the terminal as needed. This allows users to mitigate legal risks.

[0464] Step 11:

[0465] When a user creates a multilingual document, the server provides an appropriate template based on the selected language. This helps ensure that documents are accurately created for different cultural contexts.

[0466] (Example 1)

[0467] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0468] Traditionally, efficiently creating, comparing, and reviewing documents in different formats required advanced expertise and manual adjustments. Furthermore, creating multilingual documents presented challenges in understanding and accurately reflecting the norms of each cultural sphere. Additionally, the process of verifying that documents met legal requirements was cumbersome, making real-time verification difficult.

[0469] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0470] In this invention, the server includes means for analyzing existing information to learn necessary elements and formats, means for generating instructions based on normative requirements and guidelines to suggest appropriate locations within the information, and means for multilingual support to assist in generating information for different social spheres. This enables users to efficiently create documents in different formats, verify legal requirements in real time, and generate international documents through multilingual support.

[0471] "Existing information" refers to documents and data created in the past that the system will analyze.

[0472] An "element" refers to an important part or component within a document or data, which is identified during analysis.

[0473] "Format" refers to the way documents and data are represented and their layout, and it forms the basis of the templates that a system learns and provides.

[0474] "Information in different formats" refers to different methods of representing information, such as PDFs, Word documents, and images.

[0475] "Difference" refers to differences in content or format between pieces of information being compared, and is something that the system detects.

[0476] "Normative requirements" refer to requirements stipulated in laws and guidelines, and are necessary to verify that a document meets these standards.

[0477] "Instructions" refer to advice and warnings generated based on normative requirements and guidelines.

[0478] "Appropriate location" refers to the point in the document where instructions or annotations should be added or modified, as suggested by the system.

[0479] "Social sphere" refers to the cultural and legal framework within a country or region, and should be considered when creating multilingual documents.

[0480] "Multilingual support" includes features that assist in creating and translating documents in different languages.

[0481] The information processing system implementing this invention is server-centric and, in cooperation with user terminals, streamlines the document creation and proofreading processes. When a user uploads a document from their terminal to the server, the server analyzes the received document and learns the necessary elements and format. For the analysis, natural language processing technology is applied, using Python's NLTK and SpaCy as specific libraries. This makes it possible to identify important components within a document and understand its format.

[0482] The server generates templates that can be used when creating new documents based on the analysis results and provides them to the terminal. Users can easily create documents using these templates. For example, when creating a contract, it is possible to upload an existing contract and receive a template based on its format.

[0483] Furthermore, if a user uploads multiple files in different formats, the server analyzes the files, extracts text data, and compares their contents. It can clearly detect differences in content even between files of different formats, such as PDF and Word documents. Libraries used for this purpose include PyPDF2 and Tesseract OCR.

[0484] The server also utilizes a generative AI model to generate instructions based on the legal requirements and guidelines of the document, suggesting their appropriate placement within the document. Users receive this information in real time and can verify whether the document meets legal standards. If there are legal issues, the server displays a warning on the terminal, allowing the user to take appropriate action.

[0485] Furthermore, this system features multilingual capabilities, supporting document creation for different social regions. When a user selects a specific language on their terminal, the server provides templates based on corresponding standards, enabling the accurate generation of international documents.

[0486] A concrete example of a prompt message is, "Analyze the uploaded business contract and add any necessary legal annotations." In this way, the present invention aims to provide a system for efficiently creating and reviewing documents of different formats, reducing the user's workload while ensuring legal accuracy.

[0487] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0488] Step 1:

[0489] The user uploads a document from their device to the server. An existing document file (PDF, Word, image, etc.) is used as input. The server verifies the file format in order to analyze the received document.

[0490] Step 2:

[0491] The server analyzes the received document and extracts the necessary elements and formatting. It uses Python's NLTK and SpaCy to tokenize the document and perform syntactic analysis. The input is the entire document file, and the output is data regarding the semantic structure of the items and sentences within the document. This data is used to prepare for the generation of a new template.

[0492] Step 3:

[0493] The server generates a template usable for document creation based on the analysis results. The template includes an arrangement based on the document's structure and format. The input is the item data extracted in step 2, and the output is the template provided to the user. The user receives the template on their terminal and can begin creating a new document.

[0494] Step 4:

[0495] Users upload files in different formats to the server. The server receives them and extracts text data using PyPDF2 or Tesseract OCR. The input consists of multiple files in different formats, and the output is the extracted text data. The server then prepares to compare the contents using this data.

[0496] Step 5:

[0497] The server compares the extracted text data and detects differences between documents. The input is the text data obtained in step 4, and the output is a report on the differences. The differences are displayed on the user's terminal, allowing the user to make appropriate corrections.

[0498] Step 6:

[0499] The server utilizes a generated AI model to produce instructions based on the legal requirements and guidelines of the document. The input is analyzed document data, and the output is annotation information. The annotations suggest the appropriate location within the document and are displayed on the user's terminal.

[0500] Step 7:

[0501] The server verifies whether the created document meets legal standards and provides real-time warnings. The input is the final document data, and the output is warning information regarding legal risks. The user reviews the warnings and adjusts the document as needed.

[0502] Step 8:

[0503] When a user selects a specific language, the server provides a template corresponding to that language. The input is the language information selected by the user, and the output is a multilingual template. This allows users to accurately create documents for different social contexts.

[0504] (Application Example 1)

[0505] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0506] In creating advertising documents, it is essential to efficiently analyze existing documents and easily adhere to the format and legal requirements of newly created documents. However, creating documents in different formats and languages ​​is prone to minor errors and legal non-compliance, resulting in wasted time and costs.

[0507] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0508] In this invention, the server includes means for analyzing existing documents to learn necessary items and formats, means for providing learned templates when creating new documents, and means for analyzing advertising documents to generate advertising templates. This enables efficient analysis and creation of advertising documents.

[0509] "Means for learning necessary items and formats by analyzing existing documents" refers to a technical process that analyzes uploaded documents using natural language processing technology, extracts important items and structures from the documents, and generates templates based on these.

[0510] "A means of providing learned templates when creating a new document" refers to a technology that utilizes information from analyzed documents to generate and provide templates as guidelines for format and structure when a user creates a new document.

[0511] "Methods for comparing multiple documents in different formats and detecting differences" refers to technologies that analyze the text data contained in documents of different formats and automatically identify and display the differences in content.

[0512] "Means for generating annotations based on legal requirements and guidelines and suggesting their appropriate placement within a document" refers to technology that analyzes document content, automatically generates annotations in accordance with relevant legal standards and guidelines, and suggests their placement.

[0513] "A means of verifying the legal requirements of generated documents and providing real-time warnings" refers to a technology that checks the content of created documents in real time and immediately warns of any parts that may violate laws and regulations.

[0514] "Methods for analyzing advertising documents and generating advertising templates" refers to technologies that analyze existing advertising documents and automatically create templates for new advertisements based on their constituent elements.

[0515] This invention is primarily implemented through an information processing system centered on a server, terminal, and user. Upon receiving an existing document from a user, the server analyzes the document's content using a natural language processing library (e.g., spaCy). This analysis includes identifying the document's structure and necessary items, and generating a template. The generated template is provided to the user's terminal and used when creating a new document.

[0516] The server also receives documents in multiple different formats from the user, analyzes them, and compares their contents. This comparison process detects the differences between documents and displays them on the terminal, making it easy for the user to understand what needs to be corrected.

[0517] Furthermore, when creating advertising documents, the server automatically generates annotations regarding the legal requirements that the document content must comply with and suggests appropriate placement on the device. This allows users to proactively prevent legal risks. In addition, through its multilingual support function, the server provides appropriate templates when creating documents for different cultural regions.

[0518] As a concrete example, when an advertising agency agent creates a new ad on a smartphone, they upload an existing ad document, and the server automatically analyzes that document and presents a new ad template. The user can then easily create a new ad based on that template. In this process, a prompt message such as "Please analyze this document and generate an ad template" is used.

[0519] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0520] Step 1:

[0521] The user uploads an existing document to the server. The server receives the uploaded document and parses its format and structure. The input to this process is the document data provided by the user, and the output is the document format and the identification of necessary items. A natural language processing library is used to analyze each element in the document and obtain the basic data for template generation.

[0522] Step 2:

[0523] The server generates a template based on the analysis results obtained in Step 1 and sends it to the terminal. The input is the analysis results, and the output is template data. The server automatically generates an intuitive template that allows the user to easily create documents using the analysis data.

[0524] Step 3:

[0525] When a user creates a new document, they utilize a template provided on the terminal. The user creates a new document based on the submitted template. At this stage, the input is template information, and the output is a completed new document. Using the editing functions on the terminal, the structure of the new document is quickly created according to the generated template.

[0526] Step 4:

[0527] The server receives multiple documents in different formats uploaded by users, analyzes them, and detects differences. The input is multiple documents in different formats, and the output is the differences between the documents. Text data is extracted from the documents, and a comparison algorithm is used to clarify the differences in content.

[0528] Step 5:

[0529] The server generates annotations regarding legal requirements for the created advertising document and suggests appropriate placement on the terminal. The input is the advertising document, and the output is annotations based on legal requirements and their placement suggestions. The server operates an annotation generation engine that reflects legal standards and adds appropriate annotation information to the document.

[0530] Step 6:

[0531] When users create documents for different cultural contexts, the server provides multilingual templates. The input is the user's language selection information, and the output is a template corresponding to the specific language. The server uses a language database to generate templates that apply guidelines based on the selected language.

[0532] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0533] This invention optimizes document creation and editing processes based on the user's emotional state by combining an emotion engine with an existing information processing system, and embodiments thereof will be described below.

[0534] Emotion recognition and interface adjustment

[0535] The server analyzes the user's emotions in real time through the device's camera and voice input device as the user creates or proofreads documents. This analysis is performed using an emotion engine, which identifies the user's emotions from data such as facial expressions, voice tone, and typing speed. Based on the recognized emotions, the device adjusts its interface and suggestion functions to support the user in continuing their work most effectively.

[0536] Using emotional history

[0537] The server stores user sentiment data for a certain period and personalizes document creation templates and annotation suggestions based on past sentiment history. This feature allows the device to provide more appropriate suggestions that take into account the user's past sentiment tendencies.

[0538] Real-time sentiment analysis and adjustment

[0539] When a user's emotions change while they are creating a document, the server immediately analyzes this and makes adjustments accordingly. For example, if it detects that the user is feeling stressed, the device can increase the frequency of suggestions or change to a more relaxing interface. These adjustments aim to improve user efficiency and satisfaction.

[0540] Specific example

[0541] When a user is creating a legal document, if the device detects the user's confused expression, the emotion engine analyzes the emotion and reports the increased stress level to the server. In response, the server begins providing step-by-step guidance based on templates, enhancing automatic completion and advice on legal requirements. On the other hand, if the user is working calmly, the system optimizes the work environment by reducing the number of suggestions or gently changing the interface's color scheme.

[0542] These features allow the present invention to support efficient and accurate document creation while recognizing the user's emotions. As a result, users can reduce work stress and create higher-quality documents.

[0543] The following describes the processing flow.

[0544] Step 1:

[0545] The user begins creating or proofreading a document via the device. At this time, the device starts recording the user's facial expressions and voice data using sensors such as a camera and microphone.

[0546] Step 2:

[0547] The device transmits sensor data it records to the server in real time. The transmitted data includes the user's facial movements and voice tone.

[0548] Step 3:

[0549] The server analyzes the received data using an emotion engine to identify the user's current emotional state. This analysis uses natural language processing and image recognition technologies to quantify the stress, anxiety, concentration, and other emotions the user is likely experiencing.

[0550] Step 4:

[0551] Based on the analysis results, the server generates a plan to design a document environment that is appropriate for the user's emotional state. The plan includes methods for modifying the interface and adjusting the suggestion function.

[0552] Step 5:

[0553] Based on the created plan, the device will modify the interface in response to the user's emotions. These modifications may include adjusting the color scheme and rearranging UI elements.

[0554] Step 6:

[0555] Simultaneously, the device dynamically adjusts its suggestion function based on information obtained from the server. For example, if the user is confused, it provides more detailed guides and frequent suggestions; conversely, if the user is focused, it minimizes suggestions.

[0556] Step 7:

[0557] As the user continues working, emotional data is collected again and sent to the server. This process is repeated in real time, and the server constantly provides feedback based on the most up-to-date emotions.

[0558] Step 8:

[0559] After the task is completed, the server analyzes the accumulated emotional data and provides feedback to the terminal that visualizes the user's emotional fluctuations. This allows the user to reflect on their emotional state during the task and use that feedback to improve their next task.

[0560] (Example 2)

[0561] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0562] In modern information processing systems, there is a growing need to create documents efficiently and accurately, but a lack of consideration for the emotional state of users often leads to stressful situations. Furthermore, when documents are created in diverse cultural contexts and languages, standard templates and guidelines are insufficient. There is a need to address these challenges and provide information processing systems that enable users to create documents more comfortably and efficiently.

[0563] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0564] In this invention, the server includes means for analyzing existing information to learn necessary items and formats, means for comparing multiple different formats of information to detect differences, and means for analyzing emotional states based on emotional data. This enables the adjustment of the interface according to the user's emotional state and multilingual support according to cultural background and language.

[0565] "Information" is a collection of data that is recognized as having value and meaning through organization and analysis.

[0566] "Format" refers to the specific methods and patterns by which information and data are put together, and it describes the structure of a document or its content.

[0567] A "template" is a standardized framework or template used when creating a specific type of information.

[0568] "Difference" refers to the differences or distinct characteristics found between two or more pieces of information or data that are being compared.

[0569] "Emotional data" refers to related information such as facial expressions and voice that is collected to identify the emotional state of the user.

[0570] "Analysis" is the operation or process of breaking down information or data and investigating its properties in detail for better understanding.

[0571] An "interface" refers to the means, such as screens, operating methods, and input devices, that a user uses to interact with a system.

[0572] "Multilingual support" refers to a function that assists in creating and processing information in the appropriate language in situations where different languages ​​are used.

[0573] This invention optimizes document creation in an information processing system according to the user's emotional state. It primarily utilizes a terminal, a server, and an emotion engine.

[0574] Hardware and software configuration

[0575] The device is equipped with a camera and microphone to capture the user's facial expressions and voice in real time. This data is sent to a server, where an emotion engine analyzes it. The emotion engine uses a generative AI model to identify the user's emotions, which are then used to adjust the interface.

[0576] The server is responsible for changing interface colors and adjusting suggestion functions based on emotion data from the emotion engine. Furthermore, the server stores the user's emotion history and customizes document creation templates based on past emotions.

[0577] Specific example

[0578] If a user appears confused while creating a legal document, the device captures their facial expression with its camera and sends it to the server. The server analyzes this data using an emotion engine, and if it determines that the user is experiencing stress, it provides the user with a template guide. This template guide supports the user by displaying step-by-step instructions and presenting the specific requirements of legal documents.

[0579] Example of a prompt

[0580] "How can we instantly enhance template guides if users experience stress while creating legal documents?"

[0581] This system provides advanced support to ensure users can continue working comfortably, while also offering an efficient document creation environment.

[0582] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0583] Step 1:

[0584] The device acquires the user's face and voice data. Specifically, it captures the user's facial expressions with a camera and records their voice tone with a microphone. This input data is sent to the server in real time.

[0585] Step 2:

[0586] The server passes the facial and voice data received from the terminal to the emotion engine. The emotion engine uses a generative AI model to analyze the data and identify the user's emotional state. In this process, it analyzes facial features and phonemes from the input data and outputs an emotion label.

[0587] Step 3:

[0588] The server adjusts the interface based on emotion labels from the emotion engine. Specifically, if the entered emotion label is "stress," the device presents a more relaxing interface to the user and enhances the template guide. The user experience is improved by changing the screen's color scheme to blue and increasing the frequency of suggestion features.

[0589] Step 4:

[0590] The server stores user sentiment data and interface adjustment history in a database. Based on this stored history, it is possible to personalize templates and suggestions that will be useful when creating documents later. Input data includes sentiment labels and their corresponding actions, which are then used to generate optimized output in subsequent sessions.

[0591] (Application Example 2)

[0592] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0593] Modern information processing systems lack sufficient support for users to efficiently create and review documents. In particular, there is a need for interface adjustments and optimization of suggestion functions that reflect the user's emotional state during document creation. Furthermore, features for purchasing support and spending management based on user emotions are also lacking, making it difficult to provide personalized services that meet user needs.

[0594] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0595] In this invention, the server includes means for analyzing existing documents to learn necessary items and formats, means for analyzing the user's emotional state and adjusting the interface and suggestion functions based on that emotion, and means for providing expenditure management functions based on emotion analysis to support the user's purchasing decisions. This enables appropriate interface adjustments and suggestions according to the user's emotional state, resulting in efficient and personalized document creation and purchasing support.

[0596] A "document" is a collection of text and data used to organize and represent information.

[0597] "Emotional state" refers to the state of a user's psychological and mental reactions and mood, which is analyzed through facial expressions, tone of voice, input speed, and other factors.

[0598] An "interface" is a point of contact or layout that allows a user and an information processing system to interact, and is designed to make it easier for users to access the system.

[0599] A "suggestion function" is a feature provided to assist users in their operations by offering relevant information and advice on the next course of action.

[0600] The "spending management function" is a feature that analyzes the user's spending habits and financial situation, and provides advice and warnings to optimize spending.

[0601] This invention relates to an information processing system that analyzes a user's emotional state and adjusts the interface and suggested functions based on this analysis. The server analyzes the user's facial expressions, voice tone, and input speed through terminal cameras such as smartphones and voice input devices, and uses an emotion engine to identify the user's emotional state from this data. The emotional data obtained through the analysis is processed in real time using an appropriate algorithm.

[0602] In terms of hardware, a camera is used to analyze the user's facial expressions, and a microphone is used to analyze the tone of their voice. In terms of software, an image processing library such as OpenCV is used for facial expression analysis, and a specific emotion analysis model such as EmotionEngine is used for emotion analysis. This allows for interface adjustments depending on whether the user is stressed or relaxed.

[0603] As a concrete example, consider a situation where a user is online shopping. If the device detects that the user is relaxed, the server will improve the user's shopping experience by setting the interface's color scheme to a more subdued tone and refraining from providing spending management advice based on purchase history. Conversely, if the server detects that the user is stressed, it will frequently display suggestions to encourage careful spending.

[0604] An example of a prompt for a generative AI model is, "How can we provide a more enjoyable shopping experience when the user is relaxed?" This enables appropriate user support based on their emotions.

[0605] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0606] Step 1:

[0607] The device collects the user's facial expressions and voice. It receives facial image data from the smartphone's camera and audio data from the microphone as input. The device then prepares to send this data to the server.

[0608] Step 2:

[0609] The server receives data from the camera and microphone and performs emotion analysis using EmotionEngine. It uses facial image data and audio data as input, and identifies the user's emotional state (relaxed, stressed, etc.) as output emotion data.

[0610] Step 3:

[0611] The server adjusts the interface based on the user's emotional state. It uses emotional data as input and generates settings to optimize the interface's color scheme and layout as output. The server uses warm colors when the user is relaxed and suggests a simpler design when the user is stressed.

[0612] Step 4:

[0613] The server adjusts its suggestion function based on the user's emotional state. It uses emotional data as input and determines appropriate suggestions (product recommendations, purchase advice, etc.) as output. When the user is emotionally calm, it makes subtle suggestions, while when they are stressed, it emphasizes specific spending advice.

[0614] Step 5:

[0615] The user receives interface settings and suggestions sent from the server. The terminal displays this to the user and prepares to assist the user's actions. This allows the user to experience a personalized interface and make appropriate purchasing decisions.

[0616] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0617] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0618] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0619] [Fourth Embodiment]

[0620] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0621] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0622] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0623] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0624] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0625] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0626] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0627] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0628] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0629] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0630] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0631] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0632] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0633] This invention provides a specific form for efficiently creating and reviewing documents in different formats using an information processing system that analyzes and learns from existing documents.

[0634] Document analysis and template generation

[0635] When the server receives an existing document uploaded by a user, it automatically analyzes its format and structure. This analysis process uses natural language processing technology to identify the necessary items and their placement within the document. Based on the analysis results, a template is generated and provided to the user's terminal. This template can be used intuitively when creating a new document, enabling efficient document creation.

[0636] Comparing documents in different formats

[0637] When a user uploads multiple files in different formats to the server, the server analyzes the files and extracts text data. This allows for comparison of content even across different formats such as PDFs and images. The server automatically detects differences in content and displays the results on the user's device. Users can easily check the differences on their device and accurately identify areas that need correction.

[0638] Annotation generation and legal verification

[0639] The server analyzes user-created advertisements and documents and automatically generates annotations based on relevant legal requirements and guidelines. These annotations are displayed on the device along with suggestions for their appropriate placement within the document. Users can then make necessary corrections and additions based on these suggestions. Furthermore, the system checks in real time whether the created document meets legal requirements and displays warnings on the device, thus preventing legal risks.

[0640] Multilingual support

[0641] This system features multilingual capabilities, supporting document creation for different cultural contexts. When a user selects a specific language, the server provides templates based on corresponding laws and guidelines. This enables users to accurately and efficiently create documents related to international contracts and transactions.

[0642] Thus, the present invention aims to provide a system for efficiently creating and proofreading documents in various formats, thereby reducing the workload on users while ensuring legal accuracy.

[0643] The following describes the processing flow.

[0644] Step 1:

[0645] Users upload existing documents to the server via their device. Uploaded documents can be in various formats, including Word and PDF.

[0646] Step 2:

[0647] The server analyzes the received document using natural language processing techniques. This process extracts the necessary items, formatting, and sentence structure from the document.

[0648] Step 3:

[0649] The server generates a document template based on the analysis results. This template includes items and formats extracted from existing documents.

[0650] Step 4:

[0651] The generated template is sent to the device, and the user begins creating a new document based on it. The template includes a suggestion feature to support more efficient document creation.

[0652] Step 5:

[0653] When a user uploads multiple files in different formats to the server via their device, the server converts those files into text data and performs analysis. Text can also be extracted from image files using OCR technology.

[0654] Step 6:

[0655] The server compares the analyzed text data and detects differences. The detected difference information is sent to the terminal and displayed visually to the user.

[0656] Step 7:

[0657] Users can view the differences on their devices, identify and correct the parts that need fixing. The corrections are saved to the server as needed.

[0658] Step 8:

[0659] The server analyzes the user's document and generates annotations based on legal requirements and guidelines. These annotations are displayed on the terminal along with suggestions for their appropriate placement within the document.

[0660] Step 9:

[0661] Users review the annotations and suggestions provided on their devices and make any necessary corrections or additions. This ensures that the document is legally compliant in terms of both format and content.

[0662] Step 10:

[0663] The server checks in real time whether the generated documents meet legal requirements and displays warnings on the terminal as needed. This allows users to mitigate legal risks.

[0664] Step 11:

[0665] When a user creates a multilingual document, the server provides an appropriate template based on the selected language. This helps ensure that documents are accurately created for different cultural contexts.

[0666] (Example 1)

[0667] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0668] Traditionally, efficiently creating, comparing, and reviewing documents in different formats required advanced expertise and manual adjustments. Furthermore, creating multilingual documents presented challenges in understanding and accurately reflecting the norms of each cultural sphere. Additionally, the process of verifying that documents met legal requirements was cumbersome, making real-time verification difficult.

[0669] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0670] In this invention, the server includes means for analyzing existing information to learn necessary elements and formats, means for generating instructions based on normative requirements and guidelines to suggest appropriate locations within the information, and means for multilingual support to assist in generating information for different social spheres. This enables users to efficiently create documents in different formats, verify legal requirements in real time, and generate international documents through multilingual support.

[0671] "Existing information" refers to documents and data created in the past that the system will analyze.

[0672] An "element" refers to an important part or component within a document or data, which is identified during analysis.

[0673] "Format" refers to the way documents and data are represented and their layout, and it forms the basis of the templates that a system learns and provides.

[0674] "Information in different formats" refers to different methods of representing information, such as PDFs, Word documents, and images.

[0675] "Difference" refers to differences in content or format between pieces of information being compared, and is something that the system detects.

[0676] "Normative requirements" refer to requirements stipulated in laws and guidelines, and are necessary to verify that a document meets these standards.

[0677] "Instructions" refer to advice and warnings generated based on normative requirements and guidelines.

[0678] "Appropriate location" refers to the point in the document where instructions or annotations should be added or modified, as suggested by the system.

[0679] "Social sphere" refers to the cultural and legal framework within a country or region, and should be considered when creating multilingual documents.

[0680] "Multilingual support" includes features that assist in creating and translating documents in different languages.

[0681] The information processing system implementing this invention is server-centric and, in cooperation with user terminals, streamlines the document creation and proofreading processes. When a user uploads a document from their terminal to the server, the server analyzes the received document and learns the necessary elements and format. For the analysis, natural language processing technology is applied, using Python's NLTK and SpaCy as specific libraries. This makes it possible to identify important components within a document and understand its format.

[0682] The server generates templates that can be used when creating new documents based on the analysis results and provides them to the terminal. Users can easily create documents using these templates. For example, when creating a contract, it is possible to upload an existing contract and receive a template based on its format.

[0683] Furthermore, if a user uploads multiple files in different formats, the server analyzes the files, extracts text data, and compares their contents. It can clearly detect differences in content even between files of different formats, such as PDF and Word documents. Libraries used for this purpose include PyPDF2 and Tesseract OCR.

[0684] The server also utilizes a generative AI model to generate instructions based on the legal requirements and guidelines of the document, suggesting their appropriate placement within the document. Users receive this information in real time and can verify whether the document meets legal standards. If there are legal issues, the server displays a warning on the terminal, allowing the user to take appropriate action.

[0685] Furthermore, this system features multilingual capabilities, supporting document creation for different social regions. When a user selects a specific language on their terminal, the server provides templates based on corresponding standards, enabling the accurate generation of international documents.

[0686] A concrete example of a prompt message is, "Analyze the uploaded business contract and add any necessary legal annotations." In this way, the present invention aims to provide a system for efficiently creating and reviewing documents of different formats, reducing the user's workload while ensuring legal accuracy.

[0687] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0688] Step 1:

[0689] The user uploads a document from their device to the server. An existing document file (PDF, Word, image, etc.) is used as input. The server verifies the file format in order to analyze the received document.

[0690] Step 2:

[0691] The server analyzes the received document and extracts the necessary elements and formatting. It uses Python's NLTK and SpaCy to tokenize the document and perform syntactic analysis. The input is the entire document file, and the output is data regarding the semantic structure of the items and sentences within the document. This data is used to prepare for the generation of a new template.

[0692] Step 3:

[0693] The server generates a template usable for document creation based on the analysis results. The template includes an arrangement based on the document's structure and format. The input is the item data extracted in step 2, and the output is the template provided to the user. The user receives the template on their terminal and can begin creating a new document.

[0694] Step 4:

[0695] Users upload files in different formats to the server. The server receives them and extracts text data using PyPDF2 or Tesseract OCR. The input consists of multiple files in different formats, and the output is the extracted text data. The server then prepares to compare the contents using this data.

[0696] Step 5:

[0697] The server compares the extracted text data and detects differences between documents. The input is the text data obtained in step 4, and the output is a report on the differences. The differences are displayed on the user's terminal, allowing the user to make appropriate corrections.

[0698] Step 6:

[0699] The server utilizes a generated AI model to produce instructions based on the legal requirements and guidelines of the document. The input is analyzed document data, and the output is annotation information. The annotations suggest the appropriate location within the document and are displayed on the user's terminal.

[0700] Step 7:

[0701] The server verifies whether the created document meets legal standards and provides real-time warnings. The input is the final document data, and the output is warning information regarding legal risks. The user reviews the warnings and adjusts the document as needed.

[0702] Step 8:

[0703] When a user selects a specific language, the server provides a template corresponding to that language. The input is the language information selected by the user, and the output is a multilingual template. This allows users to accurately create documents for different social contexts.

[0704] (Application Example 1)

[0705] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0706] In creating advertising documents, it is essential to efficiently analyze existing documents and easily adhere to the format and legal requirements of newly created documents. However, creating documents in different formats and languages ​​is prone to minor errors and legal non-compliance, resulting in wasted time and costs.

[0707] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0708] In this invention, the server includes means for analyzing existing documents to learn necessary items and formats, means for providing learned templates when creating new documents, and means for analyzing advertising documents to generate advertising templates. This enables efficient analysis and creation of advertising documents.

[0709] "Means for learning necessary items and formats by analyzing existing documents" refers to a technical process that analyzes uploaded documents using natural language processing technology, extracts important items and structures from the documents, and generates templates based on these.

[0710] "A means of providing learned templates when creating a new document" refers to a technology that utilizes information from analyzed documents to generate and provide templates as guidelines for format and structure when a user creates a new document.

[0711] "Methods for comparing multiple documents in different formats and detecting differences" refers to technologies that analyze the text data contained in documents of different formats and automatically identify and display the differences in content.

[0712] "Means for generating annotations based on legal requirements and guidelines and suggesting their appropriate placement within a document" refers to technology that analyzes document content, automatically generates annotations in accordance with relevant legal standards and guidelines, and suggests their placement.

[0713] "A means of verifying the legal requirements of generated documents and providing real-time warnings" refers to a technology that checks the content of created documents in real time and immediately warns of any parts that may violate laws and regulations.

[0714] "Methods for analyzing advertising documents and generating advertising templates" refers to technologies that analyze existing advertising documents and automatically create templates for new advertisements based on their constituent elements.

[0715] This invention is primarily implemented through an information processing system centered on a server, terminal, and user. Upon receiving an existing document from a user, the server analyzes the document's content using a natural language processing library (e.g., spaCy). This analysis includes identifying the document's structure and necessary items, and generating a template. The generated template is provided to the user's terminal and used when creating a new document.

[0716] The server also receives documents in multiple different formats from the user, analyzes them, and compares their contents. This comparison process detects the differences between documents and displays them on the terminal, making it easy for the user to understand what needs to be corrected.

[0717] Furthermore, when creating advertising documents, the server automatically generates annotations regarding the legal requirements that the document content must comply with and suggests appropriate placement on the device. This allows users to proactively prevent legal risks. In addition, through its multilingual support function, the server provides appropriate templates when creating documents for different cultural regions.

[0718] As a concrete example, when an advertising agency agent creates a new ad on a smartphone, they upload an existing ad document, and the server automatically analyzes that document and presents a new ad template. The user can then easily create a new ad based on that template. In this process, a prompt message such as "Please analyze this document and generate an ad template" is used.

[0719] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0720] Step 1:

[0721] The user uploads an existing document to the server. The server receives the uploaded document and parses its format and structure. The input to this process is the document data provided by the user, and the output is the document format and the identification of necessary items. A natural language processing library is used to analyze each element in the document and obtain the basic data for template generation.

[0722] Step 2:

[0723] The server generates a template based on the analysis results obtained in Step 1 and sends it to the terminal. The input is the analysis results, and the output is template data. The server automatically generates an intuitive template that allows the user to easily create documents using the analysis data.

[0724] Step 3:

[0725] When a user creates a new document, they utilize a template provided on the terminal. The user creates a new document based on the submitted template. At this stage, the input is template information, and the output is a completed new document. Using the editing functions on the terminal, the structure of the new document is quickly created according to the generated template.

[0726] Step 4:

[0727] The server receives multiple documents in different formats uploaded by users, analyzes them, and detects differences. The input is multiple documents in different formats, and the output is the differences between the documents. Text data is extracted from the documents, and a comparison algorithm is used to clarify the differences in content.

[0728] Step 5:

[0729] The server generates annotations regarding legal requirements for the created advertising document and suggests appropriate placement on the terminal. The input is the advertising document, and the output is annotations based on legal requirements and their placement suggestions. The server operates an annotation generation engine that reflects legal standards and adds appropriate annotation information to the document.

[0730] Step 6:

[0731] When users create documents for different cultural contexts, the server provides multilingual templates. The input is the user's language selection information, and the output is a template corresponding to the specific language. The server uses a language database to generate templates that apply guidelines based on the selected language.

[0732] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0733] This invention optimizes document creation and editing processes based on the user's emotional state by combining an emotion engine with an existing information processing system, and embodiments thereof will be described below.

[0734] Emotion recognition and interface adjustment

[0735] The server analyzes the user's emotions in real time through the device's camera and voice input device as the user creates or proofreads documents. This analysis is performed using an emotion engine, which identifies the user's emotions from data such as facial expressions, voice tone, and typing speed. Based on the recognized emotions, the device adjusts its interface and suggestion functions to support the user in continuing their work most effectively.

[0736] Using emotional history

[0737] The server stores user sentiment data for a certain period and personalizes document creation templates and annotation suggestions based on past sentiment history. This feature allows the device to provide more appropriate suggestions that take into account the user's past sentiment tendencies.

[0738] Real-time sentiment analysis and adjustment

[0739] When a user's emotions change while they are creating a document, the server immediately analyzes this and makes adjustments accordingly. For example, if it detects that the user is feeling stressed, the device can increase the frequency of suggestions or change to a more relaxing interface. These adjustments aim to improve user efficiency and satisfaction.

[0740] Specific example

[0741] When a user is creating a legal document, if the device detects the user's confused expression, the emotion engine analyzes the emotion and reports the increased stress level to the server. In response, the server begins providing step-by-step guidance based on templates, enhancing automatic completion and advice on legal requirements. On the other hand, if the user is working calmly, the system optimizes the work environment by reducing the number of suggestions or gently changing the interface's color scheme.

[0742] These features allow the present invention to support efficient and accurate document creation while recognizing the user's emotions. As a result, users can reduce work stress and create higher-quality documents.

[0743] The following describes the processing flow.

[0744] Step 1:

[0745] The user begins creating or proofreading a document via the device. At this time, the device starts recording the user's facial expressions and voice data using sensors such as a camera and microphone.

[0746] Step 2:

[0747] The device transmits sensor data it records to the server in real time. The transmitted data includes the user's facial movements and voice tone.

[0748] Step 3:

[0749] The server analyzes the received data using an emotion engine to identify the user's current emotional state. This analysis uses natural language processing and image recognition technologies to quantify the stress, anxiety, concentration, and other emotions the user is likely experiencing.

[0750] Step 4:

[0751] Based on the analysis results, the server generates a plan to design a document environment that is appropriate for the user's emotional state. The plan includes methods for modifying the interface and adjusting the suggestion function.

[0752] Step 5:

[0753] Based on the created plan, the device will modify the interface in response to the user's emotions. These modifications may include adjusting the color scheme and rearranging UI elements.

[0754] Step 6:

[0755] Simultaneously, the device dynamically adjusts its suggestion function based on information obtained from the server. For example, if the user is confused, it provides more detailed guides and frequent suggestions; conversely, if the user is focused, it minimizes suggestions.

[0756] Step 7:

[0757] As the user continues working, emotional data is collected again and sent to the server. This process is repeated in real time, and the server constantly provides feedback based on the most up-to-date emotions.

[0758] Step 8:

[0759] After the task is completed, the server analyzes the accumulated emotional data and provides feedback to the terminal that visualizes the user's emotional fluctuations. This allows the user to reflect on their emotional state during the task and use that feedback to improve their next task.

[0760] (Example 2)

[0761] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0762] In modern information processing systems, there is a growing need to create documents efficiently and accurately, but a lack of consideration for the emotional state of users often leads to stressful situations. Furthermore, when documents are created in diverse cultural contexts and languages, standard templates and guidelines are insufficient. There is a need to address these challenges and provide information processing systems that enable users to create documents more comfortably and efficiently.

[0763] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0764] In this invention, the server includes means for analyzing existing information to learn necessary items and formats, means for comparing multiple different formats of information to detect differences, and means for analyzing emotional states based on emotional data. This enables the adjustment of the interface according to the user's emotional state and multilingual support according to cultural background and language.

[0765] "Information" is a collection of data that is recognized as having value and meaning through organization and analysis.

[0766] "Format" refers to the specific methods and patterns by which information and data are put together, and it describes the structure of a document or its content.

[0767] A "template" is a standardized framework or template used when creating a specific type of information.

[0768] "Difference" refers to the differences or distinct characteristics found between two or more pieces of information or data that are being compared.

[0769] "Emotional data" refers to related information such as facial expressions and voice that is collected to identify the emotional state of the user.

[0770] "Analysis" is the operation or process of breaking down information or data and investigating its properties in detail for better understanding.

[0771] An "interface" refers to the means, such as screens, operating methods, and input devices, that a user uses to interact with a system.

[0772] "Multilingual support" refers to a function that assists in creating and processing information in the appropriate language in situations where different languages ​​are used.

[0773] This invention optimizes document creation in an information processing system according to the user's emotional state. It primarily utilizes a terminal, a server, and an emotion engine.

[0774] Hardware and software configuration

[0775] The device is equipped with a camera and microphone to capture the user's facial expressions and voice in real time. This data is sent to a server, where an emotion engine analyzes it. The emotion engine uses a generative AI model to identify the user's emotions, which are then used to adjust the interface.

[0776] The server is responsible for changing interface colors and adjusting suggestion functions based on emotion data from the emotion engine. Furthermore, the server stores the user's emotion history and customizes document creation templates based on past emotions.

[0777] Specific example

[0778] If a user appears confused while creating a legal document, the device captures their facial expression with its camera and sends it to the server. The server analyzes this data using an emotion engine, and if it determines that the user is experiencing stress, it provides the user with a template guide. This template guide supports the user by displaying step-by-step instructions and presenting the specific requirements of legal documents.

[0779] Example of a prompt

[0780] "How can we instantly enhance template guides if users experience stress while creating legal documents?"

[0781] This system provides advanced support to ensure users can continue working comfortably, while also offering an efficient document creation environment.

[0782] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0783] Step 1:

[0784] The device acquires the user's face and voice data. Specifically, it captures the user's facial expressions with a camera and records their voice tone with a microphone. This input data is sent to the server in real time.

[0785] Step 2:

[0786] The server passes the facial and voice data received from the terminal to the emotion engine. The emotion engine uses a generative AI model to analyze the data and identify the user's emotional state. In this process, it analyzes facial features and phonemes from the input data and outputs an emotion label.

[0787] Step 3:

[0788] The server adjusts the interface based on emotion labels from the emotion engine. Specifically, if the entered emotion label is "stress," the device presents a more relaxing interface to the user and enhances the template guide. The user experience is improved by changing the screen's color scheme to blue and increasing the frequency of suggestion features.

[0789] Step 4:

[0790] The server stores user sentiment data and interface adjustment history in a database. Based on this stored history, it is possible to personalize templates and suggestions that will be useful when creating documents later. Input data includes sentiment labels and their corresponding actions, which are then used to generate optimized output in subsequent sessions.

[0791] (Application Example 2)

[0792] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0793] Modern information processing systems lack sufficient support for users to efficiently create and review documents. In particular, there is a need for interface adjustments and optimization of suggestion functions that reflect the user's emotional state during document creation. Furthermore, features for purchasing support and spending management based on user emotions are also lacking, making it difficult to provide personalized services that meet user needs.

[0794] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0795] In this invention, the server includes means for analyzing existing documents to learn necessary items and formats, means for analyzing the user's emotional state and adjusting the interface and suggestion functions based on that emotion, and means for providing expenditure management functions based on emotion analysis to support the user's purchasing decisions. This enables appropriate interface adjustments and suggestions according to the user's emotional state, resulting in efficient and personalized document creation and purchasing support.

[0796] A "document" is a collection of text and data used to organize and represent information.

[0797] "Emotional state" refers to the state of a user's psychological and mental reactions and mood, which is analyzed through facial expressions, tone of voice, input speed, and other factors.

[0798] An "interface" is a point of contact or layout that allows a user and an information processing system to interact, and is designed to make it easier for users to access the system.

[0799] A "suggestion function" is a feature provided to assist users in their operations by offering relevant information and advice on the next course of action.

[0800] The "spending management function" is a feature that analyzes the user's spending habits and financial situation, and provides advice and warnings to optimize spending.

[0801] This invention relates to an information processing system that analyzes a user's emotional state and adjusts the interface and suggested functions based on this analysis. The server analyzes the user's facial expressions, voice tone, and input speed through terminal cameras such as smartphones and voice input devices, and uses an emotion engine to identify the user's emotional state from this data. The emotional data obtained through the analysis is processed in real time using an appropriate algorithm.

[0802] In terms of hardware, a camera is used to analyze the user's facial expressions, and a microphone is used to analyze the tone of their voice. In terms of software, an image processing library such as OpenCV is used for facial expression analysis, and a specific emotion analysis model such as EmotionEngine is used for emotion analysis. This allows for interface adjustments depending on whether the user is stressed or relaxed.

[0803] As a concrete example, consider a situation where a user is online shopping. If the device detects that the user is relaxed, the server will improve the user's shopping experience by setting the interface's color scheme to a more subdued tone and refraining from providing spending management advice based on purchase history. Conversely, if the server detects that the user is stressed, it will frequently display suggestions to encourage careful spending.

[0804] An example of a prompt for a generative AI model is, "How can we provide a more enjoyable shopping experience when the user is relaxed?" This enables appropriate user support based on their emotions.

[0805] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0806] Step 1:

[0807] The device collects the user's facial expressions and voice. It receives facial image data from the smartphone's camera and audio data from the microphone as input. The device then prepares to send this data to the server.

[0808] Step 2:

[0809] The server receives data from the camera and microphone and performs emotion analysis using EmotionEngine. It uses facial image data and audio data as input, and identifies the user's emotional state (relaxed, stressed, etc.) as output emotion data.

[0810] Step 3:

[0811] The server adjusts the interface based on the user's emotional state. It uses emotional data as input and generates settings to optimize the interface's color scheme and layout as output. The server uses warm colors when the user is relaxed and suggests a simpler design when the user is stressed.

[0812] Step 4:

[0813] The server adjusts its suggestion function based on the user's emotional state. It uses emotional data as input and determines appropriate suggestions (product recommendations, purchase advice, etc.) as output. When the user is emotionally calm, it makes subtle suggestions, while when they are stressed, it emphasizes specific spending advice.

[0814] Step 5:

[0815] The user receives interface settings and suggestions sent from the server. The terminal displays this to the user and prepares to assist the user's actions. This allows the user to experience a personalized interface and make appropriate purchasing decisions.

[0816] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0817] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0818] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0819] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0820] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0821] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0822] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0823] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0824] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0825] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0826] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0827] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0828] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0829] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0830] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0831] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0832] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0833] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0834] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0835] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0836] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0837] The following is further disclosed regarding the embodiments described above.

[0838] (Claim 1)

[0839] A means of learning necessary items and formats by analyzing existing documents,

[0840] A means of providing learned templates when creating a new document,

[0841] A means for comparing documents in multiple different formats and detecting differences,

[0842] A means of generating annotations based on legal requirements and guidelines and suggesting their appropriate location within a document,

[0843] An information processing system that includes means for verifying the legal requirements of generated documents and providing real-time warnings.

[0844] (Claim 2)

[0845] The information processing system according to claim 1, comprising means for providing a suggestion function based on the content of an analyzed document.

[0846] (Claim 3)

[0847] The information processing system according to claim 1, comprising multilingual support means for supporting document creation in multiple languages ​​specialized for different cultural spheres.

[0848] "Example 1"

[0849] (Claim 1)

[0850] A means of learning the necessary elements and formats by analyzing existing information,

[0851] A means of providing a template learned when generating new information,

[0852] A means for comparing multiple different forms of information and detecting differences,

[0853] A means of generating instructions based on normative requirements and guidelines and proposing the appropriate location within the information,

[0854] A means of verifying the normative requirements of the generated information and providing real-time attention,

[0855] A system that includes multilingual support mechanisms to facilitate the generation of information for different social spheres.

[0856] (Claim 2)

[0857] The system according to claim 1, which provides an suggestion function based on the content of the analyzed information.

[0858] (Claim 3)

[0859] The system according to claim 1, comprising means for extracting and comparing content from information files of different formats.

[0860] "Application Example 1"

[0861] (Claim 1)

[0862] A means of learning necessary items and formats by analyzing existing documents,

[0863] A means of providing learned templates when creating a new document,

[0864] A means for comparing documents in multiple different formats and detecting differences,

[0865] A means of generating annotations based on legal requirements and guidelines and suggesting their appropriate location within a document,

[0866] A means to verify the legal requirements of the generated documents and provide real-time warnings,

[0867] A system that includes means for analyzing advertising documents and generating advertising templates.

[0868] (Claim 2)

[0869] The system according to claim 1, comprising means for providing a suggestion function based on the content of an analyzed document.

[0870] (Claim 3)

[0871] The system according to claim 1, including multilingual support means for supporting document creation in multiple languages ​​specific to different cultural spheres.

[0872] "Example 2 of combining an emotion engine"

[0873] (Claim 1)

[0874] A means of learning necessary items and formats by analyzing existing information,

[0875] A means of providing learned templates when creating a new document,

[0876] A means for comparing multiple different forms of information and detecting differences,

[0877] A means of generating annotations based on legal conditions and guidelines and suggesting their appropriate placement within the information,

[0878] A means to verify the legal conditions of the generated information and issue an immediate warning,

[0879] A means of acquiring the user's emotions using the terminal's camera and voice input devices,

[0880] A method for analyzing emotional states based on acquired emotional data,

[0881] A means of adjusting the interface according to the analyzed emotional state,

[0882] A means of saving a user's emotional history and personalizing suggestions based on past emotional tendencies.

[0883] A system that includes this.

[0884] (Claim 2)

[0885] The system according to claim 1, characterized in that it provides a suggestion function based on the content of the analyzed information and the emotional state.

[0886] (Claim 3)

[0887] The system according to claim 1, characterized in that it includes multilingual support means for supporting the creation of multilingual information tailored to different cultural spheres.

[0888] "Application example 2 when combining with an emotional engine"

[0889] (Claim 1)

[0890] A means of learning necessary items and formats by analyzing existing documents,

[0891] A means of providing learned templates when creating a new document,

[0892] A means for comparing documents in multiple different formats and detecting differences,

[0893] A means of generating annotations based on legal requirements and guidelines and suggesting their appropriate location within a document,

[0894] A means to verify the legal requirements of the generated documents and provide real-time warnings,

[0895] A means for analyzing the user's emotional state and adjusting the interface and suggestion functions based on that emotion,

[0896] A system that provides spending management functions based on sentiment analysis and includes means to support users' purchasing decisions.

[0897] (Claim 2)

[0898] The system according to claim 1, comprising means for providing a suggestion function based on the content of the analyzed document and the user's emotional state.

[0899] (Claim 3)

[0900] The system according to claim 1, which is equipped with multilingual support means tailored to different cultural regions and adjusts the interface according to the user's emotions. [Explanation of Symbols]

[0901] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of learning necessary items and formats by analyzing existing documents, A means of providing learned templates when creating a new document, A means for comparing documents in multiple different formats and detecting differences, A means of generating annotations based on legal requirements and guidelines and suggesting their appropriate location within a document, An information processing system that includes means for verifying the legal requirements of generated documents and providing real-time warnings.

2. The information processing system according to claim 1, comprising means for providing a suggestion function based on the content of an analyzed document.

3. The information processing system according to claim 1, comprising multilingual support means for supporting document creation in multiple languages ​​specialized for different cultural spheres.

Images