system

Abstract

We provide the system. [Solution] A means of exchanging information between humans and machines for data input, A means of analyzing input data and selecting a portion of the information, A means for automatically generating the arrangement of information and visual elements based on selected information, A means of monitoring external information sources and updating information in real time, A means to generate presentation scripts and guidelines, and to simulate question-and-answer sessions, A system that includes means for visually previewing and modifying content using an augmented reality environment.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003] [[ID=2l]]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In modern information society, a great deal of time and effort are required for document creation and presentation preparation in business and educational settings. However, these tasks are often monotonous and repetitive, which is a factor hindering the productivity of enterprises and individuals. Also, in order to cope with rapidly changing market environments and information updates, rapid document updates and real-time information reflection are required. There is a need for a new method to effectively solve these problems and achieve efficient document creation and presentation preparation.

Means for Solving the Problems

[0005] This invention solves the above problems by providing a system that automates the entire process from data entry to document generation and real-time updates. This system has a means of inputting data through a user interface, analyzes the input data, and selects the optimal document template. Furthermore, it automatically generates a document layout based on the selected template and adds necessary visual elements. In addition, it monitors external data sources in real time and automatically updates documents, ensuring that the latest information is always reflected. Moreover, it automatically generates presentation scripts and guides and simulates anticipated question-and-answer sessions, comprehensively supporting the user's presentation preparation.

[0006] A "user interface" is an access point for a user to interact with a computer system or application, enabling operation through a screen display that includes input fields, buttons, and other elements.

[0007] "Data" refers to a collection of information and facts processed within a computer system, and can be represented in various forms such as characters, numbers, and symbols.

[0008] A "document template" refers to a standardized layout or stylesheet used when creating documents or presentation slides in a specific format, providing a framework for embedding data.

[0009] "Layout" refers to the arrangement and composition of text, images, and visual elements on a document or screen, and is used to improve visual effect and readability.

[0010] "Visual elements" refer to elements such as graphs, charts, images, and icons used to convey information visually and to improve the appearance of documents.

[0011] "External data sources" refer to data providers located outside the system, including publicly available databases on the internet and data streams from other systems.

[0012] "Real-time updates" refer to a process where information and data are instantly reflected in real-world time, and are a function designed to maintain the latest state at all times.

[0013] A "script" refers to a series of instructions or commands written to achieve a specific purpose, and in a presentation, it is used to indicate the content and flow of what will be said.

[0014] "Question and answer simulation" refers to a process that supports practice and preparation by presenting pre-prepared answers to anticipated questions in presentations and Q&A sessions. [Brief explanation of the drawing]

[0015] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] Shows an emotion map to which a plurality of emotions are mapped. [Figure 10] Shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Embodiment 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Embodiment 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

Mode for Carrying Out the Invention

[0016] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0017] First, the language used in the following description will be explained.

[0018] In the following embodiments, a processor with a reference number (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of a plurality of arithmetic units. Further, the processor may be a single type of arithmetic unit or a combination of a plurality of types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0019] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0020] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0021] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0023] [First Embodiment]

[0024] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0025] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0028] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0031] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0035] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0036] As an embodiment of the present invention, an automated system for document creation and presentation support is provided. This system consists of a user, a terminal, and a server working together. The user can input the data necessary for document creation using the terminal's user interface. Through this interface, the terminal provides the user with guides and hints for data input and transmits the input data to the server.

[0037] The server analyzes the received data and selects an appropriate document template. In this process, the server utilizes natural language processing and data analysis techniques to extract important information and patterns from the input data. Based on the selected template, it automatically generates the document layout and adds visual elements that match the data. For example, if sales data is entered, the server generates an appropriate graph to visualize sales trends.

[0038] Furthermore, the server has the capability to monitor external data sources to regularly update the materials. This ensures that the latest information is automatically reflected in the materials, allowing users to always have access to the most up-to-date content. In addition, to assist with presentation preparation, the server provides features such as speech guides and simulations of anticipated questions. This allows users to efficiently prepare the necessary content for their presentations.

[0039] For example, when a user creates a market analysis report for a product, they first input market data and select the necessary analysis. The server analyzes the input data, selects the appropriate analysis method and document template, and generates a visual report using graphs and charts. Furthermore, the server monitors changes in market trend data and automatically updates the report as needed. Based on the generated document, the user can create a presentation script and practice their presentation using the speech guide suggested by the server.

[0040] Thus, the system of the present invention streamlines document creation and presentation preparation, supporting improved productivity in various business scenarios.

[0041] The following describes the processing flow.

[0042] Step 1:

[0043] Users input the data necessary for document creation through the user interface on the terminal. During input, users can fill in numerical data and text information in the form. The terminal also provides hints and options to assist with input.

[0044] Step 2:

[0045] The terminal receives the input data and converts it into a predetermined format. After the data's integrity is verified, the converted data is sent to the server.

[0046] Step 3:

[0047] The server receives data sent from the terminal. The server uses data analysis algorithms to analyze the data in detail and select the appropriate document template. The analysis includes template recommendation features based on data type and purpose.

[0048] Step 4:

[0049] The server starts automatically generating the document layout based on the selected template. During this process, each element (title, table of contents, graphs, etc.) is positioned to highlight key points. Visual elements are added as needed to enhance the visual effect.

[0050] Step 5:

[0051] The server continuously monitors external data sources and automatically updates the document content if there are any updates to the data related to the document. This real-time update function ensures that users always have access to the latest information.

[0052] Step 6:

[0053] The server generates speech guides and anticipated questions based on the presentation materials to support presentation preparation. This information is provided to the user to assist in presentation preparation.

[0054] Step 7:

[0055] Users can access speech guides and anticipated questions provided by the server via their devices and rehearse their presentations. They can then use these materials to finalize their own presentation scripts.

[0056] (Example 1)

[0057] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0058] In today's business environment, there is a demand for rapidly analyzing large amounts of data and creating documents that effectively visualize the results. However, manual document creation is time-consuming and labor-intensive, and delays in information updates can make it difficult to reflect the latest situation. Furthermore, preparing presentations requires the creation of effective guides and scripts, which also takes a considerable amount of time. This invention aims to efficiently solve these problems.

[0059] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0060] In this invention, the server includes means for providing an input / output device for inputting data, means for analyzing the input information and selecting a document template, and means for monitoring external information sources and automatically updating the materials to the latest state. This allows users to efficiently create materials and always use materials that reflect the latest information. Furthermore, means for creating presentation scripts and lesson plans and simulating question-and-answer sessions facilitate the preparation of presentations.

[0061] An "input / output device" is a device that provides an interface for users to input data and receive information from a system.

[0062] "Information" is a collection of data and materials necessary to create a document.

[0063] "Analysis" is the process of processing input information and extracting useful patterns and features.

[0064] A "document template" is a format of foundational material that structures information in a specific format or style.

[0065] "Visual elements" refer to elements such as graphs, charts, and illustrations used to convey information intuitively and effectively.

[0066] "External information sources" refer to external databases and online information services used to update or supplement materials.

[0067] "Latest status" refers to the state that contains the most up-to-date information available at the present time.

[0068] A "script" is a document that details the flow and content of the oral presentation.

[0069] A "lesson plan" is a document used in preparing a presentation that includes guides and advice to support an effective presentation.

[0070] "Simulation" refers to predicting a real-world situation and reproducing similar conditions as an exercise.

[0071] This invention is an automated system realized through the cooperation of a user, a terminal, and a server. First, the user inputs the information necessary for document creation using input / output devices on the terminal. The terminal verifies the integrity of the input data through the user interface and provides feedback to the user as needed. Software incorporating commonly used validation functions is used for this integrity check.

[0072] The information entered by the terminal is sent from the terminal to the server. The server analyzes the received information and uses natural language processing and data analysis technologies to select the optimal document template. This process utilizes a generative AI model to extract important patterns from the information and find the appropriate template. The server also automatically generates the document layout and visual elements based on the selected template. Visualization libraries and template engines are used in this process.

[0073] Furthermore, the server monitors external information sources and automatically updates the content of the materials by obtaining new information from, for example, online databases and information provision services. This ensures that users always receive materials that reflect the latest information in real time.

[0074] During presentation preparation, the server generates a script and lesson plan, and also simulates anticipated question-and-answer sessions. These processes allow users to efficiently prepare the necessary materials and content for their presentations.

[0075] For example, when a user creates a market analysis report, they input market data into a terminal, the server analyzes it, selects a suitable template, and generates a report including graphs and charts. The server continuously monitors changes in market data and automatically updates the report as needed. The user can then use the generated materials to create a presentation script and practice using a speech guide provided by the server.

[0076] An example of a prompt to input into a generating AI model is: "Describe a system that takes market sales data as input through a user interface and generates visual reports and presentation script templates based on the analyzed results."

[0077] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0078] Step 1:

[0079] The user inputs the information necessary for document creation through the terminal's input / output devices. This input may include, for example, market data or sales data. This data is verified for consistency and validated by the terminal to ensure proper formatting. The output of the input data is consistent and formatted.

[0080] Step 2:

[0081] The terminal sends verified data to the server. During this process, the data is encrypted using the SSL / TLS protocol and securely transferred to the server. The output is the data ready for the server to receive.

[0082] Step 3:

[0083] The server analyzes the received data. Here, the server utilizes a generative AI model and natural language processing techniques to extract important information and patterns from the data. The output of the data analysis consists of summarized key information and selected document templates.

[0084] Step 4:

[0085] The server selects the optimal document template based on the extracted key information and automatically generates the document layout. Visual elements such as graphs and charts appropriate for each data point are also added at this time. The output is a document with a well-organized visual layout.

[0086] Step 5:

[0087] The server monitors external information sources, for example, retrieving the latest information from online databases and reflecting it in the document. A function to automatically update the document as needed is also in operation. The output is a document kept up-to-date.

[0088] Step 6:

[0089] The user reviews the generated materials on their device and prepares for their presentation. The server provides the user with features such as a speech guide and simulated anticipated questions. This allows the user to practice while checking the necessary content to ensure an effective presentation. The output is a ready-to-present-presentation state.

[0090] (Application Example 1)

[0091] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0092] In modern content creation processes, there is a need for a system that centrally manages everything from information input to visual presentation generation, and even real-time data updates and interactive visual verification. Furthermore, there is a lack of means to efficiently verify and revise visual content in an augmented reality environment. Therefore, there is a need for a new system that enables efficient information management and rapid creation of compelling presentations.

[0093] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0094] In this invention, the server includes means for exchanging information between a human and a machine for data input, means for analyzing the input data and selecting a portion of the information, and means for visually previewing and modifying the content using an augmented reality environment. This enables efficient input, analysis, and selection of information, as well as interactive modification in an augmented reality environment.

[0095] "A means of exchanging information between humans and machines for data input" refers to an interface function that allows human users to directly interact with machines to exchange information.

[0096] "Means of analyzing input data and selecting some of the information" refers to algorithms and procedures that analyze data entered by a user and extract highly relevant information from it.

[0097] "Means for automatically generating information arrangement and visual elements based on selected information" refers to a process that automatically arranges information based on extracted information and combines related visual elements to create a visual structure.

[0098] "Means of monitoring external information sources and updating information in real time" refers to a system that continuously checks for new information from external sources and incorporates and updates the latest information.

[0099] "A means of generating presentation scripts and guidelines and simulating Q&A" refers to a function that generates scripts and guides, and simulates anticipated questions and their responses, in order to assist in creating presentation content.

[0100] "Means of visually previewing and modifying content using an augmented reality environment" refers to a system that allows users to visually interact with content using AR technology and directly make necessary corrections.

[0101] The "function of the interface for connecting to a machine to verify the integrity of input information" refers to the function of the interface necessary to check whether the input information is accurate and to ensure a smooth connection with the machine.

[0102] "A means of optimizing the arrangement of information using natural language processing in the automatic information generation process and enabling visualization in an augmented reality environment" refers to a function that uses natural language processing technology to efficiently arrange information and then visualizes that information in an AR environment.

[0103] As an embodiment of the present invention, an automated system is provided that utilizes a smartphone and a head-mounted display to perform information input, analysis, visualization, and presentation preparation in an integrated manner. It consists of communication between a server and a terminal.

[0104] The server receives input data from the user through an interface for data entry. The server analyzes the data, extracts important information using natural language processing techniques, and selects a portion of that information based on that analysis. This process uses natural language processing libraries such as NLTK and Transformers in Python, and Pandas and NumPy for data analysis.

[0105] Next, the system arranges the information based on the selected data and automatically generates visual elements. At this stage, the information is optimized and formatted to be easily understood by the user. Furthermore, the server monitors external information sources and updates the information in real time.

[0106] Users can preview visual content generated through their device using a head-mounted display. This allows them to visually check the content in an AR environment and interactively make any necessary modifications.

[0107] Furthermore, the server generates presentation scripts and guidelines, and also provides a function to simulate anticipated questions and their answers, allowing users to efficiently prepare their presentations.

[0108] As a concrete example, consider a presentation about new product development. The user inputs market data and product characteristics into a terminal, and the system analyzes this data to automatically generate the optimal slide layout and visual effects. The user can then review the content using a head-mounted display and make adjustments in real time.

[0109] An example of a prompt using a generative AI model is, "Based on the entered product data, please suggest templates and visuals to generate a visually interactive content presentation." This further streamlines the content creation process.

[0110] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0111] Step 1:

[0112] The terminal provides a user interface and retrieves input data from the user. This input data includes product characteristics and market information. The terminal's role is to transmit this data to the server.

[0113] Step 2:

[0114] The server analyzes the received data using natural language processing techniques to extract necessary information. Specifically, it uses Python's NLTK and Transformers to extract important keywords and phrases. The input for this process is data provided by the user, and the output is the extracted information.

[0115] Step 3:

[0116] The server automatically generates visual layouts and elements based on the results of the data analysis. This process involves data organization using Pandas and numerical analysis using NumPy. The input is the data set obtained in the previous step, and the output is an optimized document template and visual elements.

[0117] Step 4:

[0118] The generated visual content is provided to the user via a device. The user uses a head-mounted display to visually preview the content in an augmented reality environment and make modifications as needed. The input is the visual content, and the user's actions result in the modified output.

[0119] Step 5:

[0120] The server generates a script and anticipated questions for the presentation. In this step, a generative AI model is used to create a script and Q&A simulation from prompt text. The input is the presentation content, and the output is the script and a list of questions.

[0121] Step 6:

[0122] The server monitors external data sources and automatically updates the documents based on the latest information obtained. Real-time data collection technology is used, and the updated content is shared with the user. The input is external data, and the output is the updated document.

[0123] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0124] This invention provides a document creation and presentation support system that incorporates emotion recognition. This system consists of the interaction of a user, a terminal, a server, and an emotion engine.

[0125] First, the user inputs data for document creation through the terminal's user interface. The terminal assists with data input and sends the verified data to the server. The input data is analyzed on the server, and a process is performed to select an appropriate document template. Based on this analysis, the server automatically generates the document and appropriately places visual elements related to data visualization.

[0126] The emotion engine analyzes the user's emotions in real time using voice and video data captured from the camera. The server receives this emotion data and dynamically adjusts the content of the materials and the presentation approach according to the user's emotional state. For example, if the user is nervous, the system will provide advice to slow down the pace of the presentation or guidance to help them relax. The emotion data is also collected and recorded to help analyze and improve presentation strategies.

[0127] As a concrete example, when a user is creating a proposal for a new product, the emotion engine monitors the user's reactions. If anxiety is detected during a presentation rehearsal, the server displays a concise support message and provides supplementary explanations for difficult parts of the material. This allows the user to prepare a presentation efficiently, taking their own emotional state into consideration.

[0128] Furthermore, the ability to monitor external data sources ensures that the content of the materials always reflects the latest information. The server incorporates market data and relevant industry news, updating the materials in a timely manner, allowing users to access the most up-to-date and relevant data.

[0129] Thus, the system of the present invention goes beyond mere document creation functionality; by recognizing and considering the user's emotions, it enables the preparation of more personalized and effective presentations. Implementation of this system is expected to improve productivity and effectiveness in a variety of business scenarios.

[0130] The following describes the processing flow.

[0131] Step 1:

[0132] Users operate the terminal to input data for document creation. The input screen consists of an intuitive user interface and provides guides and template selection functions to facilitate user input.

[0133] Step 2:

[0134] The terminal checks the integrity of the entered information and verifies that there are no formatting issues. The verified data is then transferred to the server.

[0135] Step 3:

[0136] The server analyzes the received data and selects the most suitable document template. The analysis uses algorithms tailored to the type and purpose of the data.

[0137] Step 4:

[0138] The server starts automatically generating the document based on the selected template. During this process, visual elements such as graphs and charts based on data are added, completing the visual document.

[0139] Step 5:

[0140] The user's device collects emotional data in real time through its camera and microphone. This data, along with the user's facial expressions and tone of voice, is then analyzed by an emotion engine to determine their emotional state.

[0141] Step 6:

[0142] The server receives emotional data from the emotion engine and adjusts the content of the materials and the flow of the presentation accordingly. Based on the user's emotions, it can also provide advice to alleviate tension.

[0143] Step 7:

[0144] The server monitors external data sources and automatically updates the documents to the latest version when it detects updates to relevant information. This information is notified to the user and immediately reflected in the documents.

[0145] Step 8:

[0146] Users can review the completed materials, the speech guide provided by the server, anticipated questions and answers, and prepare their presentations. They can also make final adjustments with the server's assistance to create a more polished presentation.

[0147] (Example 2)

[0148] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0149] Traditional document creation and presentation systems have the problem of not being able to consider the user's emotional state, and because the content of the documents and the flow of the presentations are fixed, they cannot provide support optimized for the user's situation. In addition, because the content of the documents is not quickly updated in response to changes in the external environment, it is difficult to make decisions based on the latest information at all times.

[0150] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0151] In this invention, the server includes means for providing an interactive device for inputting data, means for analyzing the input data and selecting a document template, and means for analyzing the user's emotional state using an emotion analysis engine and dynamically adjusting the presentation content and method. This enables flexible presentations that respond to the user's emotions and the provision of up-to-date materials based on external information.

[0152] A "dialogue device" is a device that provides an interface for users to input data and has functions to verify the integrity of that data.

[0153] A "document template" is a template selected based on the entered data, and it is a type that contains the basic components for creating a document.

[0154] "Layout and composition" refers to the plan of how information and visual elements will be arranged within a generated document.

[0155] "Visual elements" are visual elements such as graphs, charts, and images used to effectively convey information in a document.

[0156] "External information sources" refer to sources of information provided from outside the system, such as market data and industry news.

[0157] An "emotion analysis engine" is a part of software or hardware that analyzes the user's voice and video data to detect and evaluate emotions in real time.

[0158] "Presentation speed" refers to the pace and tempo of a presentation, and is an element that is adjusted according to the emotional state of the user.

[0159] A "generative artificial intelligence model" is a machine learning model trained for use in data analysis and automatic document generation, and is an algorithm that enables sophisticated decision-making.

[0160] This invention is a system that analyzes the user's emotional state and supports document creation and presentations. The system mainly consists of a server, terminals, and an emotion analysis engine.

[0161] The server provides an interactive device for data input and accepts data from the user. The terminal verifies the integrity of the data through this interface and prompts the user to correct any problems. The terminal also has the ability to send data to the server, which analyzes this data and selects a document template.

[0162] The server automatically generates documents based on input data using a generative AI model. This process utilizes a generative artificial intelligence model to optimally arrange information and visual elements within the document. This makes it possible to automatically generate visually effective and easy-to-understand materials.

[0163] The emotion analysis engine analyzes the user's voice and video in real time to determine their emotional state. Based on this emotion data, the server provides advice tailored to the user's emotional state. Specifically, it can adjust the presentation speed or display guidance to alleviate tension.

[0164] Furthermore, the server has the ability to monitor external information sources and incorporate new information such as market data and industry news into the materials. This allows users to always create materials based on the latest information.

[0165] As a concrete example, when creating a new product proposal, the document is automatically generated based on market data entered by the user. If the emotion analysis engine detects the user's anxiety during the presentation, the server displays advice to help them relax. An effective prompt would be, "When creating a new product proposal, please provide advice on how to give a relaxed presentation."

[0166] This system enables users to create flexible documents and presentations that reflect their own emotions. Furthermore, because it allows access to the latest information that adapts to changing circumstances, it is expected to yield effective and efficient results in various business situations.

[0167] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0168] Step 1:

[0169] The user inputs data for document creation via an interactive device. This input data includes, for example, "sales statistics" and "market analysis information." The terminal verifies the integrity of the input data and checks for formatting errors or missing data. If there are any deficiencies, the terminal provides feedback prompting the user to make corrections. The output of this process is consistent data.

[0170] Step 2:

[0171] The terminal sends verified input data to the server. The server analyzes the received data and selects an appropriate document template. This analysis uses a generative AI model to find the optimal document template based on the data content. The output of this process is the selected document template.

[0172] Step 3:

[0173] The server utilizes a regenerative AI model to automatically generate documents based on the selected document template. Specifically, it optimizes the document layout by appropriately arranging visual elements such as graphs and tables to effectively convey information. The input is the selected document template and analysis data, and the output is the completed document.

[0174] Step 4:

[0175] An emotion analysis engine acquires the user's voice and camera footage in real time and analyzes their emotional state. The server receives this analysis result and dynamically adjusts the presentation content and approach according to the user's emotions. For example, if tension is detected, a message will be displayed instructing the user to slow down the pace of the presentation. The input to this process is real-time audio and video data, and the output is the adjusted presentation content.

[0176] Step 5:

[0177] The server constantly monitors external sources such as market data and industry news, and updates the content of the materials in real time. Input is new information from external sources, and output is updated materials reflecting the latest information. This update process ensures that the materials used by users always provide the most up-to-date information.

[0178] (Application Example 2)

[0179] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0180] In today's world, there is a demand for more efficient and effective information transmission in presentations given by individuals and companies. However, preparing for presentations requires a significant amount of time and effort to create materials. Furthermore, the presenter's emotional state during preparation and delivery can greatly impact the outcome, with performance declines due to tension and anxiety being a significant problem. In this context, there is a need to develop a system that can flexibly respond to the presenter's emotional state and provide necessary information in a timely manner.

[0181] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0182] In this invention, the server includes means for providing a user interface for inputting data, means for analyzing the input data and selecting a model form, and means for recognizing the user's emotional state using audio and video data and dynamically adjusting the content of materials and presentations. This enables personalized feedback and material adjustments in response to the user's emotional state during presentation preparation.

[0183] A "user interface" is a means of providing an interface that users use when entering data.

[0184] A "model form" refers to a template or format of a document that is most suitable for the content of the input data, based on its analysis.

[0185] "Structural design" refers to the design procedure for automatically generating the optimal layout and visual elements of a document.

[0186] "Visual elements" refer to elements related to the appearance of a document, such as design and graphics.

[0187] "External information sources" refer to external data sources that are monitored to keep the materials up-to-date.

[0188] "Real-time updates" refers to the process of updating materials in real time based on external information sources.

[0189] A "script" refers to a set of instructions used in a presentation, which guides the speaker on how to proceed.

[0190] A "lesson plan" refers to a guide or plan designed to assist in the progress of a presentation.

[0191] "Simulating a question-and-answer session" means conducting a simulation during a presentation in which you virtually answer questions.

[0192] "Emotional state" refers to the user's psychological and emotional state as recognized using audio and video data.

[0193] "Dynamic adjustment" refers to the process of changing the content of materials and presentations in real time according to the emotional state of the user.

[0194] To implement the present invention, a presentation support system equipped with emotion recognition functionality is required. This system provides a user interface for the user to input data, which is first received by the terminal. The input data is checked for consistency on the terminal before being sent to the server.

[0195] The server analyzes the input data and selects a model form. Based on this selection, the server automatically generates the document's structure and visual elements. The structure design is performed using natural language processing technology to provide an efficient layout for the document. Furthermore, the server monitors data from external sources and processes it to ensure the document is updated in real time.

[0196] The server also includes an emotion recognition engine that analyzes the user's emotional state from audio and video data. This allows the script and lesson plan to dynamically adjust according to the user's emotional state during or after the presentation. For example, if the system detects user anxiety, it may modify the lesson plan to slow down the pace of the presentation and provide advice for relaxation. For instance, the user may receive prompts such as, "What advice have you acted on in recent practice to alleviate anxiety? How have you improved your presentation based on your emotions?", providing new insights for self-improvement.

[0197] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0198] Step 1:

[0199] Users input data through the user interface on their device. The input data undergoes consistency checks on the device, and only consistent data is sent to the server. This process normalizes the input data and converts it into a format suitable for proper transfer to the server.

[0200] Step 2:

[0201] The server analyzes the received data. This analysis utilizes natural language processing techniques to select the most appropriate model form based on the data's content. The input data has its contextual information extracted, and the selected model form is obtained as output. The server's analysis process also includes matching using a similarity database.

[0202] Step 3:

[0203] The server automatically generates the document's structure and visual elements based on the selected template. It visualizes the data content analyzed using natural language processing and designs an effective layout. The generated document is provided in a format that allows the user to visually evaluate it. A template engine is used throughout this process.

[0204] Step 4:

[0205] The server monitors external information sources and updates the data immediately as needed. This process keeps the data up-to-date based on newly acquired data. The retrieval of external data from the database triggers an action to update the data content.

[0206] Step 5:

[0207] The server recognizes the user's emotional state using audio and video data. An emotion recognition engine analyzes the input data in real time to identify the user's emotional patterns. As a result of the processing, the user's current emotional state is output and used to adjust the presentation. In this step, a machine learning model is used for emotion analysis.

[0208] Step 6:

[0209] The server dynamically adjusts the content of materials and presentations based on the user's emotional state. If the user is nervous, advice and specific examples to slow down the script's pace will be displayed on the screen. This adjustment includes sending newly generated prompts to the user to optimize the presentation content.

[0210] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0211] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0212] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0213] [Second Embodiment]

[0214] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0215] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0216] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0217] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0218] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0219] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0220] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0221] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0222] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0223] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0224] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0225] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0226] As an embodiment of the present invention, an automated system for document creation and presentation support is provided. This system consists of a user, a terminal, and a server working together. The user can input the data necessary for document creation using the terminal's user interface. Through this interface, the terminal provides the user with guides and hints for data input and transmits the input data to the server.

[0227] The server analyzes the received data and selects an appropriate document template. In this process, the server utilizes natural language processing and data analysis techniques to extract important information and patterns from the input data. Based on the selected template, it automatically generates the document layout and adds visual elements that match the data. For example, if sales data is entered, the server generates an appropriate graph to visualize sales trends.

[0228] Furthermore, the server has the capability to monitor external data sources to regularly update the materials. This ensures that the latest information is automatically reflected in the materials, allowing users to always have access to the most up-to-date content. In addition, to assist with presentation preparation, the server provides features such as speech guides and simulations of anticipated questions. This allows users to efficiently prepare the necessary content for their presentations.

[0229] For example, when a user creates a market analysis report for a product, they first input market data and select the necessary analysis. The server analyzes the input data, selects the appropriate analysis method and document template, and generates a visual report using graphs and charts. Furthermore, the server monitors changes in market trend data and automatically updates the report as needed. Based on the generated document, the user can create a presentation script and practice their presentation using the speech guide suggested by the server.

[0230] Thus, the system of the present invention streamlines document creation and presentation preparation, supporting improved productivity in various business scenarios.

[0231] The following describes the processing flow.

[0232] Step 1:

[0233] Users input the data necessary for document creation through the user interface on the terminal. During input, users can fill in numerical data and text information in the form. The terminal also provides hints and options to assist with input.

[0234] Step 2:

[0235] The terminal receives the input data and converts it into a predetermined format. After the data's integrity is verified, the converted data is sent to the server.

[0236] Step 3:

[0237] The server receives data sent from the terminal. The server uses data analysis algorithms to analyze the data in detail and select the appropriate document template. The analysis includes template recommendation features based on data type and purpose.

[0238] Step 4:

[0239] The server starts automatically generating the document layout based on the selected template. During this process, each element (title, table of contents, graphs, etc.) is positioned to highlight key points. Visual elements are added as needed to enhance the visual effect.

[0240] Step 5:

[0241] The server continuously monitors external data sources and automatically updates the document content if there are any updates to the data related to the document. This real-time update function ensures that users always have access to the latest information.

[0242] Step 6:

[0243] The server generates speech guides and anticipated questions based on the presentation materials to support presentation preparation. This information is provided to the user to assist in presentation preparation.

[0244] Step 7:

[0245] Users can access speech guides and anticipated questions provided by the server via their devices and rehearse their presentations. They can then use these materials to finalize their own presentation scripts.

[0246] (Example 1)

[0247] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0248] In today's business environment, there is a demand for rapidly analyzing large amounts of data and creating documents that effectively visualize the results. However, manual document creation is time-consuming and labor-intensive, and delays in information updates can make it difficult to reflect the latest situation. Furthermore, preparing presentations requires the creation of effective guides and scripts, which also takes a considerable amount of time. This invention aims to efficiently solve these problems.

[0249] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0250] In this invention, the server includes means for providing an input / output device for inputting data, means for analyzing the input information and selecting a document template, and means for monitoring external information sources and automatically updating the materials to the latest state. This allows users to efficiently create materials and always use materials that reflect the latest information. Furthermore, means for creating presentation scripts and lesson plans and simulating question-and-answer sessions facilitate the preparation of presentations.

[0251] An "input / output device" is a device that provides an interface for users to input data and receive information from a system.

[0252] "Information" is a collection of data and materials necessary to create a document.

[0253] "Analysis" is the process of processing input information and extracting useful patterns and features.

[0254] A "document template" is a format of foundational material that structures information in a specific format or style.

[0255] "Visual elements" refer to elements such as graphs, charts, and illustrations used to convey information intuitively and effectively.

[0256] "External information sources" refer to external databases and online information services used to update or supplement materials.

[0257] "Latest status" refers to the state that contains the most up-to-date information available at the present time.

[0258] A "script" is a document that details the flow and content of the oral presentation.

[0259] A "lesson plan" is a document used in preparing a presentation that includes guides and advice to support an effective presentation.

[0260] "Simulation" refers to predicting a real-world situation and reproducing similar conditions as an exercise.

[0261] This invention is an automated system realized through the cooperation of a user, a terminal, and a server. First, the user inputs the information necessary for document creation using input / output devices on the terminal. The terminal verifies the integrity of the input data through the user interface and provides feedback to the user as needed. Software incorporating commonly used validation functions is used for this integrity check.

[0262] The information entered by the terminal is sent from the terminal to the server. The server analyzes the received information and uses natural language processing and data analysis technologies to select the optimal document template. This process utilizes a generative AI model to extract important patterns from the information and find the appropriate template. The server also automatically generates the document layout and visual elements based on the selected template. Visualization libraries and template engines are used in this process.

[0263] Furthermore, the server monitors external information sources and automatically updates the content of the materials by obtaining new information from, for example, online databases and information provision services. This ensures that users always receive materials that reflect the latest information in real time.

[0264] During presentation preparation, the server generates a script and lesson plan, and also simulates anticipated question-and-answer sessions. These processes allow users to efficiently prepare the necessary materials and content for their presentations.

[0265] For example, when a user creates a market analysis report, they input market data into a terminal, the server analyzes it, selects a suitable template, and generates a report including graphs and charts. The server continuously monitors changes in market data and automatically updates the report as needed. The user can then use the generated materials to create a presentation script and practice using a speech guide provided by the server.

[0266] An example of a prompt to input into a generating AI model is: "Describe a system that takes market sales data as input through a user interface and generates visual reports and presentation script templates based on the analyzed results."

[0267] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0268] Step 1:

[0269] The user inputs the information necessary for document creation through the terminal's input / output devices. This input may include, for example, market data or sales data. This data is verified for consistency and validated by the terminal to ensure proper formatting. The output of the input data is consistent and formatted.

[0270] Step 2:

[0271] The terminal sends verified data to the server. During this process, the data is encrypted using the SSL / TLS protocol and securely transferred to the server. The output is the data ready for the server to receive.

[0272] Step 3:

[0273] The server analyzes the received data. Here, the server utilizes a generative AI model and natural language processing techniques to extract important information and patterns from the data. The output of the data analysis consists of summarized key information and selected document templates.

[0274] Step 4:

[0275] The server selects the optimal document template based on the extracted key information and automatically generates the document layout. Visual elements such as graphs and charts appropriate for each data point are also added at this time. The output is a document with a well-organized visual layout.

[0276] Step 5:

[0277] The server monitors external information sources, for example, retrieving the latest information from online databases and reflecting it in the document. A function to automatically update the document as needed is also in operation. The output is a document kept up-to-date.

[0278] Step 6:

[0279] The user reviews the generated materials on their device and prepares for their presentation. The server provides the user with features such as a speech guide and simulated anticipated questions. This allows the user to practice while checking the necessary content to ensure an effective presentation. The output is a ready-to-present-presentation state.

[0280] (Application Example 1)

[0281] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0282] In modern content creation processes, there is a need for a system that centrally manages everything from information input to visual presentation generation, and even real-time data updates and interactive visual verification. Furthermore, there is a lack of means to efficiently verify and revise visual content in an augmented reality environment. Therefore, there is a need for a new system that enables efficient information management and rapid creation of compelling presentations.

[0283] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0284] In this invention, the server includes means for exchanging information between humans and machines for data input, means for analyzing the input data and selecting a part of the information, and means for visually previewing and modifying the content using an augmented reality environment. This enables efficient input, analysis, and selection of information, as well as interactive modification in the augmented reality environment.

[0285] The "means for exchanging information between humans and machines for data input" is an interface function for direct information exchange between human users and machines.

[0286] The "means for analyzing the input data and selecting a part of the information" is an algorithm or procedure for analyzing the data input by the user and extracting highly relevant information therefrom.

[0287] The "means for automatically generating the arrangement and visual elements of information based on the selected information" is a process of automatically arranging the information and combining related visual elements based on the extracted information.

[0288] The "means for monitoring an external information source and performing real-time update of information" is a mechanism for continuously checking for new information from the outside and incorporating the latest information for update.

[0289] The "means for generating a script or guide for presentation and simulating question and answer" is a function for generating a script or guide and simulating predicted questions and their answers to assist in creating presentation content.

[0290] The "means for visually previewing and modifying the content using an augmented reality environment" is a mechanism for the user to visually interactively check the content using AR technology and directly make necessary modifications.

[0291] The "function of the interface for connecting to a machine to verify the integrity of input information" refers to the function of the interface necessary to check whether the input information is accurate and to ensure a smooth connection with the machine.

[0292] "A means of optimizing the arrangement of information using natural language processing in the automatic information generation process and enabling visualization in an augmented reality environment" refers to a function that uses natural language processing technology to efficiently arrange information and then visualizes that information in an AR environment.

[0293] As an embodiment of the present invention, an automated system is provided that utilizes a smartphone and a head-mounted display to perform information input, analysis, visualization, and presentation preparation in an integrated manner. It consists of communication between a server and a terminal.

[0294] The server receives input data from the user through an interface for data entry. The server analyzes the data, extracts important information using natural language processing techniques, and selects a portion of that information based on that analysis. This process uses natural language processing libraries such as NLTK and Transformers in Python, and Pandas and NumPy for data analysis.

[0295] Next, the system arranges the information based on the selected data and automatically generates visual elements. At this stage, the information is optimized and formatted to be easily understood by the user. Furthermore, the server monitors external information sources and updates the information in real time.

[0296] Users can preview visual content generated through their device using a head-mounted display. This allows them to visually check the content in an AR environment and interactively make any necessary modifications.

[0297] Furthermore, the server generates presentation scripts and guidelines, and also provides a function to simulate anticipated questions and their answers, allowing users to efficiently prepare their presentations.

[0298] As a concrete example, consider a presentation about new product development. The user inputs market data and product characteristics into a terminal, and the system analyzes this data to automatically generate the optimal slide layout and visual effects. The user can then review the content using a head-mounted display and make adjustments in real time.

[0299] An example of a prompt using a generative AI model is, "Based on the entered product data, please suggest templates and visuals to generate a visually interactive content presentation." This further streamlines the content creation process.

[0300] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0301] Step 1:

[0302] The terminal provides a user interface and retrieves input data from the user. This input data includes product characteristics and market information. The terminal's role is to transmit this data to the server.

[0303] Step 2:

[0304] The server analyzes the received data using natural language processing techniques to extract necessary information. Specifically, it uses Python's NLTK and Transformers to extract important keywords and phrases. The input for this process is data provided by the user, and the output is the extracted information.

[0305] Step 3:

[0306] Based on the results of information analysis, the server automatically generates visual layouts and elements. In this process, data cleaning using Pandas and numerical analysis using NumPy are performed. The input is the information group obtained in the previous step, and the output is an optimized document template and visual elements.

[0307] Step 4:

[0308] The generated visual content is provided to the user via a terminal. The user uses a head-mounted display to visually preview the content in an augmented reality environment and make corrections as needed. The input is the visual content, and the operations performed by the user are the correction results of the output.

[0309] Step 5:

[0310] The server generates a script and assumed questions for the presentation. In this step, a generation AI model is used to create a script and a question-and-answer simulation from the prompt text. The input is the presentation content, and the output is the script and the question list.

[0311] Step 6:

[0312] The server monitors external data sources and automatically updates the documents based on the latest information obtained. The real-time data collection technology is used at this time, and the updated content is shared with the user. The input is the external data, and the output is the updated document.

[0313] Furthermore, an emotion engine for estimating the user's emotion may be combined. That is, the specific processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform specific processing using the user's emotion.

[0314] This invention provides a document creation and presentation support system that incorporates emotion recognition. This system consists of the interaction of a user, a terminal, a server, and an emotion engine.

[0315] First, the user inputs data for document creation through the terminal's user interface. The terminal assists with data input and sends the verified data to the server. The input data is analyzed on the server, and a process is performed to select an appropriate document template. Based on this analysis, the server automatically generates the document and appropriately places visual elements related to data visualization.

[0316] The emotion engine analyzes the user's emotions in real time using voice and video data captured from the camera. The server receives this emotion data and dynamically adjusts the content of the materials and the presentation approach according to the user's emotional state. For example, if the user is nervous, the system will provide advice to slow down the pace of the presentation or guidance to help them relax. The emotion data is also collected and recorded to help analyze and improve presentation strategies.

[0317] As a concrete example, when a user is creating a proposal for a new product, the emotion engine monitors the user's reactions. If anxiety is detected during a presentation rehearsal, the server displays a concise support message and provides supplementary explanations for difficult parts of the material. This allows the user to prepare a presentation efficiently, taking their own emotional state into consideration.

[0318] Furthermore, the ability to monitor external data sources ensures that the content of the materials always reflects the latest information. The server incorporates market data and relevant industry news, updating the materials in a timely manner, allowing users to access the most up-to-date and relevant data.

[0319] Thus, the system of the present invention goes beyond mere document creation functionality; by recognizing and considering the user's emotions, it enables the preparation of more personalized and effective presentations. Implementation of this system is expected to improve productivity and effectiveness in a variety of business scenarios.

[0320] The following describes the processing flow.

[0321] Step 1:

[0322] Users operate the terminal to input data for document creation. The input screen consists of an intuitive user interface and provides guides and template selection functions to facilitate user input.

[0323] Step 2:

[0324] The terminal checks the integrity of the entered information and verifies that there are no formatting issues. The verified data is then transferred to the server.

[0325] Step 3:

[0326] The server analyzes the received data and selects the most suitable document template. The analysis uses algorithms tailored to the type and purpose of the data.

[0327] Step 4:

[0328] The server starts automatically generating the document based on the selected template. During this process, visual elements such as graphs and charts based on data are added, completing the visual document.

[0329] Step 5:

[0330] The user's device collects emotional data in real time through its camera and microphone. This data, along with the user's facial expressions and tone of voice, is then analyzed by an emotion engine to determine their emotional state.

[0331] Step 6:

[0332] The server receives emotional data from the emotion engine and adjusts the content of the materials and the flow of the presentation accordingly. Based on the user's emotions, it can also provide advice to alleviate tension.

[0333] Step 7:

[0334] The server monitors external data sources and automatically updates the documents to the latest version when it detects updates to relevant information. This information is notified to the user and immediately reflected in the documents.

[0335] Step 8:

[0336] Users can review the completed materials, the speech guide provided by the server, anticipated questions and answers, and prepare their presentations. They can also make final adjustments with the server's assistance to create a more polished presentation.

[0337] (Example 2)

[0338] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0339] Traditional document creation and presentation systems have the problem of not being able to consider the user's emotional state, and because the content of the documents and the flow of the presentations are fixed, they cannot provide support optimized for the user's situation. In addition, because the content of the documents is not quickly updated in response to changes in the external environment, it is difficult to make decisions based on the latest information at all times.

[0340] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0341] In this invention, the server includes means for providing an interactive device for inputting data, means for analyzing the input data and selecting a document template, and means for analyzing the user's emotional state using an emotion analysis engine and dynamically adjusting the presentation content and method. This enables flexible presentations that respond to the user's emotions and the provision of up-to-date materials based on external information.

[0342] A "dialogue device" is a device that provides an interface for users to input data and has functions to verify the integrity of that data.

[0343] A "document template" is a template selected based on the entered data, and it is a type that contains the basic components for creating a document.

[0344] "Layout and composition" refers to the plan of how information and visual elements will be arranged within a generated document.

[0345] "Visual elements" are visual elements such as graphs, charts, and images used to effectively convey information in a document.

[0346] "External information sources" refer to sources of information provided from outside the system, such as market data and industry news.

[0347] An "emotion analysis engine" is a part of software or hardware that analyzes the user's voice and video data to detect and evaluate emotions in real time.

[0348] "Presentation speed" refers to the pace and tempo of a presentation, and is an element that is adjusted according to the emotional state of the user.

[0349] A "generative artificial intelligence model" is a machine learning model trained for use in data analysis and automatic document generation, and is an algorithm that enables sophisticated decision-making.

[0350] This invention is a system that analyzes the user's emotional state and supports document creation and presentations. The system mainly consists of a server, terminals, and an emotion analysis engine.

[0351] The server provides an interactive device for data input and accepts data from the user. The terminal verifies the integrity of the data through this interface and prompts the user to correct any problems. The terminal also has the ability to send data to the server, which analyzes this data and selects a document template.

[0352] The server automatically generates documents based on input data using a generative AI model. This process utilizes a generative artificial intelligence model to optimally arrange information and visual elements within the document. This makes it possible to automatically generate visually effective and easy-to-understand materials.

[0353] The emotion analysis engine analyzes the user's voice and video in real time to determine their emotional state. Based on this emotion data, the server provides advice tailored to the user's emotional state. Specifically, it can adjust the presentation speed or display guidance to alleviate tension.

[0354] Furthermore, the server has the ability to monitor external information sources and incorporate new information such as market data and industry news into the materials. This allows users to always create materials based on the latest information.

[0355] As a concrete example, when creating a new product proposal, the document is automatically generated based on market data entered by the user. If the emotion analysis engine detects the user's anxiety during the presentation, the server displays advice to help them relax. An effective prompt would be, "When creating a new product proposal, please provide advice on how to give a relaxed presentation."

[0356] This system enables users to create flexible documents and presentations that reflect their own emotions. Furthermore, because it allows access to the latest information that adapts to changing circumstances, it is expected to yield effective and efficient results in various business situations.

[0357] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0358] Step 1:

[0359] The user inputs data for document creation via an interactive device. This input data includes, for example, "sales statistics" and "market analysis information." The terminal verifies the integrity of the input data and checks for formatting errors or missing data. If there are any deficiencies, the terminal provides feedback prompting the user to make corrections. The output of this process is consistent data.

[0360] Step 2:

[0361] The terminal sends verified input data to the server. The server analyzes the received data and selects an appropriate document template. This analysis uses a generative AI model to find the optimal document template based on the data content. The output of this process is the selected document template.

[0362] Step 3:

[0363] The server utilizes a regenerative AI model to automatically generate documents based on the selected document template. Specifically, it optimizes the document layout by appropriately arranging visual elements such as graphs and tables to effectively convey information. The input is the selected document template and analysis data, and the output is the completed document.

[0364] Step 4:

[0365] An emotion analysis engine acquires the user's voice and camera footage in real time and analyzes their emotional state. The server receives this analysis result and dynamically adjusts the presentation content and approach according to the user's emotions. For example, if tension is detected, a message will be displayed instructing the user to slow down the pace of the presentation. The input to this process is real-time audio and video data, and the output is the adjusted presentation content.

[0366] Step 5:

[0367] The server constantly monitors external sources such as market data and industry news, and updates the content of the materials in real time. Input is new information from external sources, and output is updated materials reflecting the latest information. This update process ensures that the materials used by users always provide the most up-to-date information.

[0368] (Application Example 2)

[0369] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0370] In today's world, there is a demand for more efficient and effective information transmission in presentations given by individuals and companies. However, preparing for presentations requires a significant amount of time and effort to create materials. Furthermore, the presenter's emotional state during preparation and delivery can greatly impact the outcome, with performance declines due to tension and anxiety being a significant problem. In this context, there is a need to develop a system that can flexibly respond to the presenter's emotional state and provide necessary information in a timely manner.

[0371] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0372] In this invention, the server includes means for providing a user interface for inputting data, means for analyzing the input data and selecting a model form, and means for recognizing the user's emotional state using audio and video data and dynamically adjusting the content of materials and presentations. This enables personalized feedback and material adjustments in response to the user's emotional state during presentation preparation.

[0373] A "user interface" is a means of providing an interface that users use when entering data.

[0374] A "model form" refers to a template or format of a document that is most suitable for the content of the input data, based on its analysis.

[0375] "Structural design" refers to the design procedure for automatically generating the optimal layout and visual elements of a document.

[0376] "Visual elements" refer to elements related to the appearance of a document, such as design and graphics.

[0377] "External information sources" refer to external data sources that are monitored to keep the materials up-to-date.

[0378] "Real-time updates" refers to the process of updating materials in real time based on external information sources.

[0379] A "script" refers to a set of instructions used in a presentation, which guides the speaker on how to proceed.

[0380] A "lesson plan" refers to a guide or plan designed to assist in the progress of a presentation.

[0381] "Simulating a question-and-answer session" means conducting a simulation during a presentation in which you virtually answer questions.

[0382] "Emotional state" refers to the user's psychological and emotional state as recognized using audio and video data.

[0383] "Dynamic adjustment" refers to the process of changing the content of materials and presentations in real time according to the emotional state of the user.

[0384] To implement the present invention, a presentation support system equipped with emotion recognition functionality is required. This system provides a user interface for the user to input data, which is first received by the terminal. The input data is checked for consistency on the terminal before being sent to the server.

[0385] The server analyzes the input data and selects a model form. Based on this selection, the server automatically generates the document's structure and visual elements. The structure design is performed using natural language processing technology to provide an efficient layout for the document. Furthermore, the server monitors data from external sources and processes it to ensure the document is updated in real time.

[0386] The server also includes an emotion recognition engine that analyzes the user's emotional state from audio and video data. This allows the script and lesson plan to dynamically adjust according to the user's emotional state during or after the presentation. For example, if the system detects user anxiety, it may modify the lesson plan to slow down the pace of the presentation and provide advice for relaxation. For instance, the user may receive prompts such as, "What advice have you acted on in recent practice to alleviate anxiety? How have you improved your presentation based on your emotions?", providing new insights for self-improvement.

[0387] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0388] Step 1:

[0389] Users input data through the user interface on their device. The input data undergoes consistency checks on the device, and only consistent data is sent to the server. This process normalizes the input data and converts it into a format suitable for proper transfer to the server.

[0390] Step 2:

[0391] The server analyzes the received data. This analysis utilizes natural language processing techniques to select the most appropriate model form based on the data's content. The input data has its contextual information extracted, and the selected model form is obtained as output. The server's analysis process also includes matching using a similarity database.

[0392] Step 3:

[0393] The server automatically generates the document's structure and visual elements based on the selected template. It visualizes the data content analyzed using natural language processing and designs an effective layout. The generated document is provided in a format that allows the user to visually evaluate it. A template engine is used throughout this process.

[0394] Step 4:

[0395] The server monitors external information sources and updates the data immediately as needed. This process keeps the data up-to-date based on newly acquired data. The retrieval of external data from the database triggers an action to update the data content.

[0396] Step 5:

[0397] The server recognizes the user's emotional state using audio and video data. An emotion recognition engine analyzes the input data in real time to identify the user's emotional patterns. As a result of the processing, the user's current emotional state is output and used to adjust the presentation. In this step, a machine learning model is used for emotion analysis.

[0398] Step 6:

[0399] The server dynamically adjusts the content of materials and presentations based on the user's emotional state. If the user is nervous, advice and specific examples to slow down the script's pace will be displayed on the screen. This adjustment includes sending newly generated prompts to the user to optimize the presentation content.

[0400] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0401] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0402] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0403] [Third Embodiment]

[0404] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0405] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0406] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0407] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0408] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0409] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0410] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0411] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0412] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0413] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0414] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0415] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0416] As an embodiment of the present invention, an automated system for document creation and presentation support is provided. This system consists of a user, a terminal, and a server working together. The user can input the data necessary for document creation using the terminal's user interface. Through this interface, the terminal provides the user with guides and hints for data input and transmits the input data to the server.

[0417] The server analyzes the received data and selects an appropriate document template. In this process, the server utilizes natural language processing and data analysis techniques to extract important information and patterns from the input data. Based on the selected template, it automatically generates the document layout and adds visual elements that match the data. For example, if sales data is entered, the server generates an appropriate graph to visualize sales trends.

[0418] Furthermore, the server has the capability to monitor external data sources to regularly update the materials. This ensures that the latest information is automatically reflected in the materials, allowing users to always have access to the most up-to-date content. In addition, to assist with presentation preparation, the server provides features such as speech guides and simulations of anticipated questions. This allows users to efficiently prepare the necessary content for their presentations.

[0419] For example, when a user creates a market analysis report for a product, they first input market data and select the necessary analysis. The server analyzes the input data, selects the appropriate analysis method and document template, and generates a visual report using graphs and charts. Furthermore, the server monitors changes in market trend data and automatically updates the report as needed. Based on the generated document, the user can create a presentation script and practice their presentation using the speech guide suggested by the server.

[0420] Thus, the system of the present invention streamlines document creation and presentation preparation, supporting improved productivity in various business scenarios.

[0421] The following describes the processing flow.

[0422] Step 1:

[0423] Users input the data necessary for document creation through the user interface on the terminal. During input, users can fill in numerical data and text information in the form. The terminal also provides hints and options to assist with input.

[0424] Step 2:

[0425] The terminal receives the input data and converts it into a predetermined format. After the data's integrity is verified, the converted data is sent to the server.

[0426] Step 3:

[0427] The server receives data sent from the terminal. The server uses data analysis algorithms to analyze the data in detail and select the appropriate document template. The analysis includes template recommendation features based on data type and purpose.

[0428] Step 4:

[0429] The server starts automatically generating the document layout based on the selected template. During this process, each element (title, table of contents, graphs, etc.) is positioned to highlight key points. Visual elements are added as needed to enhance the visual effect.

[0430] Step 5:

[0431] The server continuously monitors external data sources and automatically updates the document content if there are any updates to the data related to the document. This real-time update function ensures that users always have access to the latest information.

[0432] Step 6:

[0433] The server generates speech guides and anticipated questions based on the presentation materials to support presentation preparation. This information is provided to the user to assist in presentation preparation.

[0434] Step 7:

[0435] Users can access speech guides and anticipated questions provided by the server via their devices and rehearse their presentations. They can then use these materials to finalize their own presentation scripts.

[0436] (Example 1)

[0437] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0438] In today's business environment, there is a demand for rapidly analyzing large amounts of data and creating documents that effectively visualize the results. However, manual document creation is time-consuming and labor-intensive, and delays in information updates can make it difficult to reflect the latest situation. Furthermore, preparing presentations requires the creation of effective guides and scripts, which also takes a considerable amount of time. This invention aims to efficiently solve these problems.

[0439] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0440] In this invention, the server includes means for providing an input / output device for inputting data, means for analyzing the input information and selecting a document template, and means for monitoring external information sources and automatically updating the materials to the latest state. This allows users to efficiently create materials and always use materials that reflect the latest information. Furthermore, means for creating presentation scripts and lesson plans and simulating question-and-answer sessions facilitate the preparation of presentations.

[0441] An "input / output device" is a device that provides an interface for users to input data and receive information from a system.

[0442] "Information" is a collection of data and materials necessary to create a document.

[0443] "Analysis" is the process of processing input information and extracting useful patterns and features.

[0444] A "document template" is a format of foundational material that structures information in a specific format or style.

[0445] "Visual elements" refer to elements such as graphs, charts, and illustrations used to convey information intuitively and effectively.

[0446] "External information sources" refer to external databases and online information services used to update or supplement materials.

[0447] "Latest status" refers to the state that contains the most up-to-date information available at the present time.

[0448] A "script" is a document that details the flow and content of the oral presentation.

[0449] A "lesson plan" is a document used in preparing a presentation that includes guides and advice to support an effective presentation.

[0450] "Simulation" refers to predicting a real-world situation and reproducing similar conditions as an exercise.

[0451] This invention is an automated system realized through the cooperation of a user, a terminal, and a server. First, the user inputs the information necessary for document creation using input / output devices on the terminal. The terminal verifies the integrity of the input data through the user interface and provides feedback to the user as needed. Software incorporating commonly used validation functions is used for this integrity check.

[0452] The information entered by the terminal is sent from the terminal to the server. The server analyzes the received information and uses natural language processing and data analysis technologies to select the optimal document template. This process utilizes a generative AI model to extract important patterns from the information and find the appropriate template. The server also automatically generates the document layout and visual elements based on the selected template. Visualization libraries and template engines are used in this process.

[0453] Furthermore, the server monitors external information sources and automatically updates the content of the materials by obtaining new information from, for example, online databases and information provision services. This ensures that users always receive materials that reflect the latest information in real time.

[0454] During presentation preparation, the server generates a script and lesson plan, and also simulates anticipated question-and-answer sessions. These processes allow users to efficiently prepare the necessary materials and content for their presentations.

[0455] For example, when a user creates a market analysis report, they input market data into a terminal, the server analyzes it, selects a suitable template, and generates a report including graphs and charts. The server continuously monitors changes in market data and automatically updates the report as needed. The user can then use the generated materials to create a presentation script and practice using a speech guide provided by the server.

[0456] An example of a prompt to input into a generating AI model is: "Describe a system that takes market sales data as input through a user interface and generates visual reports and presentation script templates based on the analyzed results."

[0457] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0458] Step 1:

[0459] The user inputs the information necessary for document creation through the terminal's input / output devices. This input may include, for example, market data or sales data. This data is verified for consistency and validated by the terminal to ensure proper formatting. The output of the input data is consistent and formatted.

[0460] Step 2:

[0461] The terminal sends verified data to the server. During this process, the data is encrypted using the SSL / TLS protocol and securely transferred to the server. The output is the data ready for the server to receive.

[0462] Step 3:

[0463] The server analyzes the received data. Here, the server utilizes a generative AI model and natural language processing techniques to extract important information and patterns from the data. The output of the data analysis consists of summarized key information and selected document templates.

[0464] Step 4:

[0465] The server selects the optimal document template based on the extracted key information and automatically generates the document layout. Visual elements such as graphs and charts appropriate for each data point are also added at this time. The output is a document with a well-organized visual layout.

[0466] Step 5:

[0467] The server monitors external information sources, for example, retrieving the latest information from online databases and reflecting it in the document. A function to automatically update the document as needed is also in operation. The output is a document kept up-to-date.

[0468] Step 6:

[0469] The user reviews the generated materials on their device and prepares for their presentation. The server provides the user with features such as a speech guide and simulated anticipated questions. This allows the user to practice while checking the necessary content to ensure an effective presentation. The output is a ready-to-present-presentation state.

[0470] (Application Example 1)

[0471] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0472] In modern content creation processes, there is a need for a system that centrally manages everything from information input to visual presentation generation, and even real-time data updates and interactive visual verification. Furthermore, there is a lack of means to efficiently verify and revise visual content in an augmented reality environment. Therefore, there is a need for a new system that enables efficient information management and rapid creation of compelling presentations.

[0473] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0474] In this invention, the server includes means for exchanging information between a human and a machine for data input, means for analyzing the input data and selecting a portion of the information, and means for visually previewing and modifying the content using an augmented reality environment. This enables efficient input, analysis, and selection of information, as well as interactive modification in an augmented reality environment.

[0475] "A means of exchanging information between humans and machines for data input" refers to an interface function that allows human users to directly interact with machines to exchange information.

[0476] "Means of analyzing input data and selecting some of the information" refers to algorithms and procedures that analyze data entered by a user and extract highly relevant information from it.

[0477] "Means for automatically generating information arrangement and visual elements based on selected information" refers to a process that automatically arranges information based on extracted information and combines related visual elements to create a visual structure.

[0478] "Means of monitoring external information sources and updating information in real time" refers to a system that continuously checks for new information from external sources and incorporates and updates the latest information.

[0479] "A means of generating presentation scripts and guidelines and simulating Q&A" refers to a function that generates scripts and guides, and simulates anticipated questions and their responses, in order to assist in creating presentation content.

[0480] "Means of visually previewing and modifying content using an augmented reality environment" refers to a system that allows users to visually interact with content using AR technology and directly make necessary corrections.

[0481] The "function of the interface for connecting to a machine to verify the integrity of input information" refers to the function of the interface necessary to check whether the input information is accurate and to ensure a smooth connection with the machine.

[0482] "A means of optimizing the arrangement of information using natural language processing in the automatic information generation process and enabling visualization in an augmented reality environment" refers to a function that uses natural language processing technology to efficiently arrange information and then visualizes that information in an AR environment.

[0483] As an embodiment of the present invention, an automated system is provided that utilizes a smartphone and a head-mounted display to perform information input, analysis, visualization, and presentation preparation in an integrated manner. It consists of communication between a server and a terminal.

[0484] The server receives input data from the user through an interface for data entry. The server analyzes the data, extracts important information using natural language processing techniques, and selects a portion of that information based on that analysis. This process uses natural language processing libraries such as NLTK and Transformers in Python, and Pandas and NumPy for data analysis.

[0485] Next, the system arranges the information based on the selected data and automatically generates visual elements. At this stage, the information is optimized and formatted to be easily understood by the user. Furthermore, the server monitors external information sources and updates the information in real time.

[0486] Users can preview visual content generated through their device using a head-mounted display. This allows them to visually check the content in an AR environment and interactively make any necessary modifications.

[0487] Furthermore, the server generates presentation scripts and guidelines, and also provides a function to simulate anticipated questions and their answers, allowing users to efficiently prepare their presentations.

[0488] As a concrete example, consider a presentation about new product development. The user inputs market data and product characteristics into a terminal, and the system analyzes this data to automatically generate the optimal slide layout and visual effects. The user can then review the content using a head-mounted display and make adjustments in real time.

[0489] An example of a prompt using a generative AI model is, "Based on the entered product data, please suggest templates and visuals to generate a visually interactive content presentation." This further streamlines the content creation process.

[0490] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0491] Step 1:

[0492] The terminal provides a user interface and retrieves input data from the user. This input data includes product characteristics and market information. The terminal's role is to transmit this data to the server.

[0493] Step 2:

[0494] The server analyzes the received data using natural language processing techniques to extract necessary information. Specifically, it uses Python's NLTK and Transformers to extract important keywords and phrases. The input for this process is data provided by the user, and the output is the extracted information.

[0495] Step 3:

[0496] The server automatically generates visual layouts and elements based on the results of the data analysis. This process involves data organization using Pandas and numerical analysis using NumPy. The input is the data set obtained in the previous step, and the output is an optimized document template and visual elements.

[0497] Step 4:

[0498] The generated visual content is provided to the user via a device. The user uses a head-mounted display to visually preview the content in an augmented reality environment and make modifications as needed. The input is the visual content, and the user's actions result in the modified output.

[0499] Step 5:

[0500] The server generates a script and anticipated questions for the presentation. In this step, a generative AI model is used to create a script and Q&A simulation from prompt text. The input is the presentation content, and the output is the script and a list of questions.

[0501] Step 6:

[0502] The server monitors external data sources and automatically updates the documents based on the latest information obtained. Real-time data collection technology is used, and the updated content is shared with the user. The input is external data, and the output is the updated document.

[0503] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0504] This invention provides a document creation and presentation support system that incorporates emotion recognition. This system consists of the interaction of a user, a terminal, a server, and an emotion engine.

[0505] First, the user inputs data for document creation through the terminal's user interface. The terminal assists with data input and sends the verified data to the server. The input data is analyzed on the server, and a process is performed to select an appropriate document template. Based on this analysis, the server automatically generates the document and appropriately places visual elements related to data visualization.

[0506] The emotion engine analyzes the user's emotions in real time using voice and video data captured from the camera. The server receives this emotion data and dynamically adjusts the content of the materials and the presentation approach according to the user's emotional state. For example, if the user is nervous, the system will provide advice to slow down the pace of the presentation or guidance to help them relax. The emotion data is also collected and recorded to help analyze and improve presentation strategies.

[0507] As a concrete example, when a user is creating a proposal for a new product, the emotion engine monitors the user's reactions. If anxiety is detected during a presentation rehearsal, the server displays a concise support message and provides supplementary explanations for difficult parts of the material. This allows the user to prepare a presentation efficiently, taking their own emotional state into consideration.

[0508] Furthermore, the ability to monitor external data sources ensures that the content of the materials always reflects the latest information. The server incorporates market data and relevant industry news, updating the materials in a timely manner, allowing users to access the most up-to-date and relevant data.

[0509] Thus, the system of the present invention goes beyond mere document creation functionality; by recognizing and considering the user's emotions, it enables the preparation of more personalized and effective presentations. Implementation of this system is expected to improve productivity and effectiveness in a variety of business scenarios.

[0510] The following describes the processing flow.

[0511] Step 1:

[0512] Users operate the terminal to input data for document creation. The input screen consists of an intuitive user interface and provides guides and template selection functions to facilitate user input.

[0513] Step 2:

[0514] The terminal checks the integrity of the entered information and verifies that there are no formatting issues. The verified data is then transferred to the server.

[0515] Step 3:

[0516] The server analyzes the received data and selects the most suitable document template. The analysis uses algorithms tailored to the type and purpose of the data.

[0517] Step 4:

[0518] The server starts automatically generating the document based on the selected template. During this process, visual elements such as graphs and charts based on data are added, completing the visual document.

[0519] Step 5:

[0520] The user's device collects emotional data in real time through its camera and microphone. This data, along with the user's facial expressions and tone of voice, is then analyzed by an emotion engine to determine their emotional state.

[0521] Step 6:

[0522] The server receives emotional data from the emotion engine and adjusts the content of the materials and the flow of the presentation accordingly. Based on the user's emotions, it can also provide advice to alleviate tension.

[0523] Step 7:

[0524] The server monitors external data sources and automatically updates the documents to the latest version when it detects updates to relevant information. This information is notified to the user and immediately reflected in the documents.

[0525] Step 8:

[0526] Users can review the completed materials, the speech guide provided by the server, anticipated questions and answers, and prepare their presentations. They can also make final adjustments with the server's assistance to create a more polished presentation.

[0527] (Example 2)

[0528] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0529] Traditional document creation and presentation systems have the problem of not being able to consider the user's emotional state, and because the content of the documents and the flow of the presentations are fixed, they cannot provide support optimized for the user's situation. In addition, because the content of the documents is not quickly updated in response to changes in the external environment, it is difficult to make decisions based on the latest information at all times.

[0530] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0531] In this invention, the server includes means for providing an interactive device for inputting data, means for analyzing the input data and selecting a document template, and means for analyzing the user's emotional state using an emotion analysis engine and dynamically adjusting the presentation content and method. This enables flexible presentations that respond to the user's emotions and the provision of up-to-date materials based on external information.

[0532] A "dialogue device" is a device that provides an interface for users to input data and has functions to verify the integrity of that data.

[0533] A "document template" is a template selected based on the entered data, and it is a type that contains the basic components for creating a document.

[0534] "Layout and composition" refers to the plan of how information and visual elements will be arranged within a generated document.

[0535] "Visual elements" are visual elements such as graphs, charts, and images used to effectively convey information in a document.

[0536] "External information sources" refer to sources of information provided from outside the system, such as market data and industry news.

[0537] An "emotion analysis engine" is a part of software or hardware that analyzes the user's voice and video data to detect and evaluate emotions in real time.

[0538] "Presentation speed" refers to the pace and tempo of a presentation, and is an element that is adjusted according to the emotional state of the user.

[0539] A "generative artificial intelligence model" is a machine learning model trained for use in data analysis and automatic document generation, and is an algorithm that enables sophisticated decision-making.

[0540] This invention is a system that analyzes the user's emotional state and supports document creation and presentations. The system mainly consists of a server, terminals, and an emotion analysis engine.

[0541] The server provides an interactive device for data input and accepts data from the user. The terminal verifies the integrity of the data through this interface and prompts the user to correct any problems. The terminal also has the ability to send data to the server, which analyzes this data and selects a document template.

[0542] The server automatically generates documents based on input data using a generative AI model. This process utilizes a generative artificial intelligence model to optimally arrange information and visual elements within the document. This makes it possible to automatically generate visually effective and easy-to-understand materials.

[0543] The emotion analysis engine analyzes the user's voice and video in real time to determine their emotional state. Based on this emotion data, the server provides advice tailored to the user's emotional state. Specifically, it can adjust the presentation speed or display guidance to alleviate tension.

[0544] Furthermore, the server has the ability to monitor external information sources and incorporate new information such as market data and industry news into the materials. This allows users to always create materials based on the latest information.

[0545] As a concrete example, when creating a new product proposal, the document is automatically generated based on market data entered by the user. If the emotion analysis engine detects the user's anxiety during the presentation, the server displays advice to help them relax. An effective prompt would be, "When creating a new product proposal, please provide advice on how to give a relaxed presentation."

[0546] This system enables users to create flexible documents and presentations that reflect their own emotions. Furthermore, because it allows access to the latest information that adapts to changing circumstances, it is expected to yield effective and efficient results in various business situations.

[0547] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0548] Step 1:

[0549] The user inputs data for document creation via an interactive device. This input data includes, for example, "sales statistics" and "market analysis information." The terminal verifies the integrity of the input data and checks for formatting errors or missing data. If there are any deficiencies, the terminal provides feedback prompting the user to make corrections. The output of this process is consistent data.

[0550] Step 2:

[0551] The terminal sends verified input data to the server. The server analyzes the received data and selects an appropriate document template. This analysis uses a generative AI model to find the optimal document template based on the data content. The output of this process is the selected document template.

[0552] Step 3:

[0553] The server utilizes a regenerative AI model to automatically generate documents based on the selected document template. Specifically, it optimizes the document layout by appropriately arranging visual elements such as graphs and tables to effectively convey information. The input is the selected document template and analysis data, and the output is the completed document.

[0554] Step 4:

[0555] An emotion analysis engine acquires the user's voice and camera footage in real time and analyzes their emotional state. The server receives this analysis result and dynamically adjusts the presentation content and approach according to the user's emotions. For example, if tension is detected, a message will be displayed instructing the user to slow down the pace of the presentation. The input to this process is real-time audio and video data, and the output is the adjusted presentation content.

[0556] Step 5:

[0557] The server constantly monitors external sources such as market data and industry news, and updates the content of the materials in real time. Input is new information from external sources, and output is updated materials reflecting the latest information. This update process ensures that the materials used by users always provide the most up-to-date information.

[0558] (Application Example 2)

[0559] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0560] In today's world, there is a demand for more efficient and effective information transmission in presentations given by individuals and companies. However, preparing for presentations requires a significant amount of time and effort to create materials. Furthermore, the presenter's emotional state during preparation and delivery can greatly impact the outcome, with performance declines due to tension and anxiety being a significant problem. In this context, there is a need to develop a system that can flexibly respond to the presenter's emotional state and provide necessary information in a timely manner.

[0561] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0562] In this invention, the server includes means for providing a user interface for inputting data, means for analyzing the input data and selecting a model form, and means for recognizing the user's emotional state using audio and video data and dynamically adjusting the content of materials and presentations. This enables personalized feedback and material adjustments in response to the user's emotional state during presentation preparation.

[0563] A "user interface" is a means of providing an interface that users use when entering data.

[0564] A "model form" refers to a template or format of a document that is most suitable for the content of the input data, based on its analysis.

[0565] "Structural design" refers to the design procedure for automatically generating the optimal layout and visual elements of a document.

[0566] "Visual elements" refer to elements related to the appearance of a document, such as design and graphics.

[0567] "External information sources" refer to external data sources that are monitored to keep the materials up-to-date.

[0568] "Real-time updates" refers to the process of updating materials in real time based on external information sources.

[0569] A "script" refers to a set of instructions used in a presentation, which guides the speaker on how to proceed.

[0570] A "lesson plan" refers to a guide or plan designed to assist in the progress of a presentation.

[0571] "Simulating a question-and-answer session" means conducting a simulation during a presentation in which you virtually answer questions.

[0572] "Emotional state" refers to the user's psychological and emotional state as recognized using audio and video data.

[0573] "Dynamic adjustment" refers to the process of changing the content of materials and presentations in real time according to the emotional state of the user.

[0574] To implement the present invention, a presentation support system equipped with emotion recognition functionality is required. This system provides a user interface for the user to input data, which is first received by the terminal. The input data is checked for consistency on the terminal before being sent to the server.

[0575] The server analyzes the input data and selects a model form. Based on this selection, the server automatically generates the document's structure and visual elements. The structure design is performed using natural language processing technology to provide an efficient layout for the document. Furthermore, the server monitors data from external sources and processes it to ensure the document is updated in real time.

[0576] The server also includes an emotion recognition engine that analyzes the user's emotional state from audio and video data. This allows the script and lesson plan to dynamically adjust according to the user's emotional state during or after the presentation. For example, if the system detects user anxiety, it may modify the lesson plan to slow down the pace of the presentation and provide advice for relaxation. For instance, the user may receive prompts such as, "What advice have you acted on in recent practice to alleviate anxiety? How have you improved your presentation based on your emotions?", providing new insights for self-improvement.

[0577] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0578] Step 1:

[0579] Users input data through the user interface on their device. The input data undergoes consistency checks on the device, and only consistent data is sent to the server. This process normalizes the input data and converts it into a format suitable for proper transfer to the server.

[0580] Step 2:

[0581] The server analyzes the received data. This analysis utilizes natural language processing techniques to select the most appropriate model form based on the data's content. The input data has its contextual information extracted, and the selected model form is obtained as output. The server's analysis process also includes matching using a similarity database.

[0582] Step 3:

[0583] The server automatically generates the document's structure and visual elements based on the selected template. It visualizes the data content analyzed using natural language processing and designs an effective layout. The generated document is provided in a format that allows the user to visually evaluate it. A template engine is used throughout this process.

[0584] Step 4:

[0585] The server monitors external information sources and updates the data immediately as needed. This process keeps the data up-to-date based on newly acquired data. The retrieval of external data from the database triggers an action to update the data content.

[0586] Step 5:

[0587] The server recognizes the user's emotional state using audio and video data. An emotion recognition engine analyzes the input data in real time to identify the user's emotional patterns. As a result of the processing, the user's current emotional state is output and used to adjust the presentation. In this step, a machine learning model is used for emotion analysis.

[0588] Step 6:

[0589] The server dynamically adjusts the content of materials and presentations based on the user's emotional state. If the user is nervous, advice and specific examples to slow down the script's pace will be displayed on the screen. This adjustment includes sending newly generated prompts to the user to optimize the presentation content.

[0590] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0591] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0592] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0593] [Fourth Embodiment]

[0594] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0595] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0596] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0597] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0598] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0599] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0600] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0601] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0602] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0603] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0604] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0605] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0606] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0607] As an embodiment of the present invention, an automated system for document creation and presentation support is provided. This system consists of a user, a terminal, and a server working together. The user can input the data necessary for document creation using the terminal's user interface. Through this interface, the terminal provides the user with guides and hints for data input and transmits the input data to the server.

[0608] The server analyzes the received data and selects an appropriate document template. In this process, the server utilizes natural language processing and data analysis techniques to extract important information and patterns from the input data. Based on the selected template, it automatically generates the document layout and adds visual elements that match the data. For example, if sales data is entered, the server generates an appropriate graph to visualize sales trends.

[0609] Furthermore, the server has the capability to monitor external data sources to regularly update the materials. This ensures that the latest information is automatically reflected in the materials, allowing users to always have access to the most up-to-date content. In addition, to assist with presentation preparation, the server provides features such as speech guides and simulations of anticipated questions. This allows users to efficiently prepare the necessary content for their presentations.

[0610] For example, when a user creates a market analysis report for a product, they first input market data and select the necessary analysis. The server analyzes the input data, selects the appropriate analysis method and document template, and generates a visual report using graphs and charts. Furthermore, the server monitors changes in market trend data and automatically updates the report as needed. Based on the generated document, the user can create a presentation script and practice their presentation using the speech guide suggested by the server.

[0611] Thus, the system of the present invention streamlines document creation and presentation preparation, supporting improved productivity in various business scenarios.

[0612] The following describes the processing flow.

[0613] Step 1:

[0614] Users input the data necessary for document creation through the user interface on the terminal. During input, users can fill in numerical data and text information in the form. The terminal also provides hints and options to assist with input.

[0615] Step 2:

[0616] The terminal receives the input data and converts it into a predetermined format. After the data's integrity is verified, the converted data is sent to the server.

[0617] Step 3:

[0618] The server receives data sent from the terminal. The server uses data analysis algorithms to analyze the data in detail and select the appropriate document template. The analysis includes template recommendation features based on data type and purpose.

[0619] Step 4:

[0620] The server starts automatically generating the document layout based on the selected template. During this process, each element (title, table of contents, graphs, etc.) is positioned to highlight key points. Visual elements are added as needed to enhance the visual effect.

[0621] Step 5:

[0622] The server continuously monitors external data sources and automatically updates the document content if there are any updates to the data related to the document. This real-time update function ensures that users always have access to the latest information.

[0623] Step 6:

[0624] The server generates speech guides and anticipated questions based on the presentation materials to support presentation preparation. This information is provided to the user to assist in presentation preparation.

[0625] Step 7:

[0626] Users can access speech guides and anticipated questions provided by the server via their devices and rehearse their presentations. They can then use these materials to finalize their own presentation scripts.

[0627] (Example 1)

[0628] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0629] In today's business environment, there is a demand for rapidly analyzing large amounts of data and creating documents that effectively visualize the results. However, manual document creation is time-consuming and labor-intensive, and delays in information updates can make it difficult to reflect the latest situation. Furthermore, preparing presentations requires the creation of effective guides and scripts, which also takes a considerable amount of time. This invention aims to efficiently solve these problems.

[0630] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0631] In this invention, the server includes means for providing an input / output device for inputting data, means for analyzing the input information and selecting a document template, and means for monitoring external information sources and automatically updating the materials to the latest state. This allows users to efficiently create materials and always use materials that reflect the latest information. Furthermore, means for creating presentation scripts and lesson plans and simulating question-and-answer sessions facilitate the preparation of presentations.

[0632] An "input / output device" is a device that provides an interface for users to input data and receive information from a system.

[0633] "Information" is a collection of data and materials necessary to create a document.

[0634] "Analysis" is the process of processing input information and extracting useful patterns and features.

[0635] A "document template" is a format of foundational material that structures information in a specific format or style.

[0636] "Visual elements" refer to elements such as graphs, charts, and illustrations used to convey information intuitively and effectively.

[0637] "External information sources" refer to external databases and online information services used to update or supplement materials.

[0638] "Latest status" refers to the state that contains the most up-to-date information available at the present time.

[0639] A "script" is a document that details the flow and content of the oral presentation.

[0640] A "lesson plan" is a document used in preparing a presentation that includes guides and advice to support an effective presentation.

[0641] "Simulation" refers to predicting a real-world situation and reproducing similar conditions as an exercise.

[0642] This invention is an automated system realized through the cooperation of a user, a terminal, and a server. First, the user inputs the information necessary for document creation using input / output devices on the terminal. The terminal verifies the integrity of the input data through the user interface and provides feedback to the user as needed. Software incorporating commonly used validation functions is used for this integrity check.

[0643] The information entered by the terminal is sent from the terminal to the server. The server analyzes the received information and uses natural language processing and data analysis technologies to select the optimal document template. This process utilizes a generative AI model to extract important patterns from the information and find the appropriate template. The server also automatically generates the document layout and visual elements based on the selected template. Visualization libraries and template engines are used in this process.

[0644] Furthermore, the server monitors external information sources and automatically updates the content of the materials by obtaining new information from, for example, online databases and information provision services. This ensures that users always receive materials that reflect the latest information in real time.

[0645] During presentation preparation, the server generates a script and lesson plan, and also simulates anticipated question-and-answer sessions. These processes allow users to efficiently prepare the necessary materials and content for their presentations.

[0646] For example, when a user creates a market analysis report, they input market data into a terminal, the server analyzes it, selects a suitable template, and generates a report including graphs and charts. The server continuously monitors changes in market data and automatically updates the report as needed. The user can then use the generated materials to create a presentation script and practice using a speech guide provided by the server.

[0647] An example of a prompt to input into a generating AI model is: "Describe a system that takes market sales data as input through a user interface and generates visual reports and presentation script templates based on the analyzed results."

[0648] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0649] Step 1:

[0650] The user inputs the information necessary for document creation through the terminal's input / output devices. This input may include, for example, market data or sales data. This data is verified for consistency and validated by the terminal to ensure proper formatting. The output of the input data is consistent and formatted.

[0651] Step 2:

[0652] The terminal sends verified data to the server. During this process, the data is encrypted using the SSL / TLS protocol and securely transferred to the server. The output is the data ready for the server to receive.

[0653] Step 3:

[0654] The server analyzes the received data. Here, the server utilizes a generative AI model and natural language processing techniques to extract important information and patterns from the data. The output of the data analysis consists of summarized key information and selected document templates.

[0655] Step 4:

[0656] The server selects the optimal document template based on the extracted key information and automatically generates the document layout. Visual elements such as graphs and charts appropriate for each data point are also added at this time. The output is a document with a well-organized visual layout.

[0657] Step 5:

[0658] The server monitors external information sources, for example, retrieving the latest information from online databases and reflecting it in the document. A function to automatically update the document as needed is also in operation. The output is a document kept up-to-date.

[0659] Step 6:

[0660] The user reviews the generated materials on their device and prepares for their presentation. The server provides the user with features such as a speech guide and simulated anticipated questions. This allows the user to practice while checking the necessary content to ensure an effective presentation. The output is a ready-to-present-presentation state.

[0661] (Application Example 1)

[0662] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0663] In modern content creation processes, there is a need for a system that centrally manages everything from information input to visual presentation generation, and even real-time data updates and interactive visual verification. Furthermore, there is a lack of means to efficiently verify and revise visual content in an augmented reality environment. Therefore, there is a need for a new system that enables efficient information management and rapid creation of compelling presentations.

[0664] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0665] In this invention, the server includes means for exchanging information between a human and a machine for data input, means for analyzing the input data and selecting a portion of the information, and means for visually previewing and modifying the content using an augmented reality environment. This enables efficient input, analysis, and selection of information, as well as interactive modification in an augmented reality environment.

[0666] "A means of exchanging information between humans and machines for data input" refers to an interface function that allows human users to directly interact with machines to exchange information.

[0667] "Means of analyzing input data and selecting some of the information" refers to algorithms and procedures that analyze data entered by a user and extract highly relevant information from it.

[0668] "Means for automatically generating information arrangement and visual elements based on selected information" refers to a process that automatically arranges information based on extracted information and combines related visual elements to create a visual structure.

[0669] "Means of monitoring external information sources and updating information in real time" refers to a system that continuously checks for new information from external sources and incorporates and updates the latest information.

[0670] "A means of generating presentation scripts and guidelines and simulating Q&A" refers to a function that generates scripts and guides, and simulates anticipated questions and their responses, in order to assist in creating presentation content.

[0671] "Means of visually previewing and modifying content using an augmented reality environment" refers to a system that allows users to visually interact with content using AR technology and directly make necessary corrections.

[0672] The "function of the interface for connecting to a machine to verify the integrity of input information" refers to the function of the interface necessary to check whether the input information is accurate and to ensure a smooth connection with the machine.

[0673] "A means of optimizing the arrangement of information using natural language processing in the automatic information generation process and enabling visualization in an augmented reality environment" refers to a function that uses natural language processing technology to efficiently arrange information and then visualizes that information in an AR environment.

[0674] As an embodiment of the present invention, an automated system is provided that utilizes a smartphone and a head-mounted display to perform information input, analysis, visualization, and presentation preparation in an integrated manner. It consists of communication between a server and a terminal.

[0675] The server receives input data from the user through an interface for data entry. The server analyzes the data, extracts important information using natural language processing techniques, and selects a portion of that information based on that analysis. This process uses natural language processing libraries such as NLTK and Transformers in Python, and Pandas and NumPy for data analysis.

[0676] Next, the system arranges the information based on the selected data and automatically generates visual elements. At this stage, the information is optimized and formatted to be easily understood by the user. Furthermore, the server monitors external information sources and updates the information in real time.

[0677] Users can preview visual content generated through their device using a head-mounted display. This allows them to visually check the content in an AR environment and interactively make any necessary modifications.

[0678] Furthermore, the server generates presentation scripts and guidelines, and also provides a function to simulate anticipated questions and their answers, allowing users to efficiently prepare their presentations.

[0679] As a concrete example, consider a presentation about new product development. The user inputs market data and product characteristics into a terminal, and the system analyzes this data to automatically generate the optimal slide layout and visual effects. The user can then review the content using a head-mounted display and make adjustments in real time.

[0680] An example of a prompt using a generative AI model is, "Based on the entered product data, please suggest templates and visuals to generate a visually interactive content presentation." This further streamlines the content creation process.

[0681] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0682] Step 1:

[0683] The terminal provides a user interface and retrieves input data from the user. This input data includes product characteristics and market information. The terminal's role is to transmit this data to the server.

[0684] Step 2:

[0685] The server analyzes the received data using natural language processing techniques to extract necessary information. Specifically, it uses Python's NLTK and Transformers to extract important keywords and phrases. The input for this process is data provided by the user, and the output is the extracted information.

[0686] Step 3:

[0687] The server automatically generates visual layouts and elements based on the results of the data analysis. This process involves data organization using Pandas and numerical analysis using NumPy. The input is the data set obtained in the previous step, and the output is an optimized document template and visual elements.

[0688] Step 4:

[0689] The generated visual content is provided to the user via a device. The user uses a head-mounted display to visually preview the content in an augmented reality environment and make modifications as needed. The input is the visual content, and the user's actions result in the modified output.

[0690] Step 5:

[0691] The server generates a script and anticipated questions for the presentation. In this step, a generative AI model is used to create a script and Q&A simulation from prompt text. The input is the presentation content, and the output is the script and a list of questions.

[0692] Step 6:

[0693] The server monitors external data sources and automatically updates the documents based on the latest information obtained. Real-time data collection technology is used, and the updated content is shared with the user. The input is external data, and the output is the updated document.

[0694] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0695] This invention provides a document creation and presentation support system that incorporates emotion recognition. This system consists of the interaction of a user, a terminal, a server, and an emotion engine.

[0696] First, the user inputs data for document creation through the terminal's user interface. The terminal assists with data input and sends the verified data to the server. The input data is analyzed on the server, and a process is performed to select an appropriate document template. Based on this analysis, the server automatically generates the document and appropriately places visual elements related to data visualization.

[0697] The emotion engine analyzes the user's emotions in real time using voice and video data captured from the camera. The server receives this emotion data and dynamically adjusts the content of the materials and the presentation approach according to the user's emotional state. For example, if the user is nervous, the system will provide advice to slow down the pace of the presentation or guidance to help them relax. The emotion data is also collected and recorded to help analyze and improve presentation strategies.

[0698] As a concrete example, when a user is creating a proposal for a new product, the emotion engine monitors the user's reactions. If anxiety is detected during a presentation rehearsal, the server displays a concise support message and provides supplementary explanations for difficult parts of the material. This allows the user to prepare a presentation efficiently, taking their own emotional state into consideration.

[0699] Furthermore, the ability to monitor external data sources ensures that the content of the materials always reflects the latest information. The server incorporates market data and relevant industry news, updating the materials in a timely manner, allowing users to access the most up-to-date and relevant data.

[0700] Thus, the system of the present invention goes beyond mere document creation functionality; by recognizing and considering the user's emotions, it enables the preparation of more personalized and effective presentations. Implementation of this system is expected to improve productivity and effectiveness in a variety of business scenarios.

[0701] The following describes the processing flow.

[0702] Step 1:

[0703] Users operate the terminal to input data for document creation. The input screen consists of an intuitive user interface and provides guides and template selection functions to facilitate user input.

[0704] Step 2:

[0705] The terminal checks the integrity of the entered information and verifies that there are no formatting issues. The verified data is then transferred to the server.

[0706] Step 3:

[0707] The server analyzes the received data and selects the most suitable document template. The analysis uses algorithms tailored to the type and purpose of the data.

[0708] Step 4:

[0709] The server starts automatically generating the document based on the selected template. During this process, visual elements such as graphs and charts based on data are added, completing the visual document.

[0710] Step 5:

[0711] The user's device collects emotional data in real time through its camera and microphone. This data, along with the user's facial expressions and tone of voice, is then analyzed by an emotion engine to determine their emotional state.

[0712] Step 6:

[0713] The server receives emotional data from the emotion engine and adjusts the content of the materials and the flow of the presentation accordingly. Based on the user's emotions, it can also provide advice to alleviate tension.

[0714] Step 7:

[0715] The server monitors external data sources and automatically updates the documents to the latest version when it detects updates to relevant information. This information is notified to the user and immediately reflected in the documents.

[0716] Step 8:

[0717] Users can review the completed materials, the speech guide provided by the server, anticipated questions and answers, and prepare their presentations. They can also make final adjustments with the server's assistance to create a more polished presentation.

[0718] (Example 2)

[0719] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0720] Traditional document creation and presentation systems have the problem of not being able to consider the user's emotional state, and because the content of the documents and the flow of the presentations are fixed, they cannot provide support optimized for the user's situation. In addition, because the content of the documents is not quickly updated in response to changes in the external environment, it is difficult to make decisions based on the latest information at all times.

[0721] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0722] In this invention, the server includes means for providing an interactive device for inputting data, means for analyzing the input data and selecting a document template, and means for analyzing the user's emotional state using an emotion analysis engine and dynamically adjusting the presentation content and method. This enables flexible presentations that respond to the user's emotions and the provision of up-to-date materials based on external information.

[0723] A "dialogue device" is a device that provides an interface for users to input data and has functions to verify the integrity of that data.

[0724] A "document template" is a template selected based on the entered data, and it is a type that contains the basic components for creating a document.

[0725] "Layout and composition" refers to the plan of how information and visual elements will be arranged within a generated document.

[0726] "Visual elements" are visual elements such as graphs, charts, and images used to effectively convey information in a document.

[0727] "External information sources" refer to sources of information provided from outside the system, such as market data and industry news.

[0728] An "emotion analysis engine" is a part of software or hardware that analyzes the user's voice and video data to detect and evaluate emotions in real time.

[0729] "Presentation speed" refers to the pace and tempo of a presentation, and is an element that is adjusted according to the emotional state of the user.

[0730] A "generative artificial intelligence model" is a machine learning model trained for use in data analysis and automatic document generation, and is an algorithm that enables sophisticated decision-making.

[0731] This invention is a system that analyzes the user's emotional state and supports document creation and presentations. The system mainly consists of a server, terminals, and an emotion analysis engine.

[0732] The server provides an interactive device for data input and accepts data from the user. The terminal verifies the integrity of the data through this interface and prompts the user to correct any problems. The terminal also has the ability to send data to the server, which analyzes this data and selects a document template.

[0733] The server automatically generates documents based on input data using a generative AI model. This process utilizes a generative artificial intelligence model to optimally arrange information and visual elements within the document. This makes it possible to automatically generate visually effective and easy-to-understand materials.

[0734] The emotion analysis engine analyzes the user's voice and video in real time to determine their emotional state. Based on this emotion data, the server provides advice tailored to the user's emotional state. Specifically, it can adjust the presentation speed or display guidance to alleviate tension.

[0735] Furthermore, the server has the ability to monitor external information sources and incorporate new information such as market data and industry news into the materials. This allows users to always create materials based on the latest information.

[0736] As a concrete example, when creating a new product proposal, the document is automatically generated based on market data entered by the user. If the emotion analysis engine detects the user's anxiety during the presentation, the server displays advice to help them relax. An effective prompt would be, "When creating a new product proposal, please provide advice on how to give a relaxed presentation."

[0737] This system enables users to create flexible documents and presentations that reflect their own emotions. Furthermore, because it allows access to the latest information that adapts to changing circumstances, it is expected to yield effective and efficient results in various business situations.

[0738] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0739] Step 1:

[0740] The user inputs data for document creation via an interactive device. This input data includes, for example, "sales statistics" and "market analysis information." The terminal verifies the integrity of the input data and checks for formatting errors or missing data. If there are any deficiencies, the terminal provides feedback prompting the user to make corrections. The output of this process is consistent data.

[0741] Step 2:

[0742] The terminal sends verified input data to the server. The server analyzes the received data and selects an appropriate document template. This analysis uses a generative AI model to find the optimal document template based on the data content. The output of this process is the selected document template.

[0743] Step 3:

[0744] The server utilizes a regenerative AI model to automatically generate documents based on the selected document template. Specifically, it optimizes the document layout by appropriately arranging visual elements such as graphs and tables to effectively convey information. The input is the selected document template and analysis data, and the output is the completed document.

[0745] Step 4:

[0746] An emotion analysis engine acquires the user's voice and camera footage in real time and analyzes their emotional state. The server receives this analysis result and dynamically adjusts the presentation content and approach according to the user's emotions. For example, if tension is detected, a message will be displayed instructing the user to slow down the pace of the presentation. The input to this process is real-time audio and video data, and the output is the adjusted presentation content.

[0747] Step 5:

[0748] The server constantly monitors external sources such as market data and industry news, and updates the content of the materials in real time. Input is new information from external sources, and output is updated materials reflecting the latest information. This update process ensures that the materials used by users always provide the most up-to-date information.

[0749] (Application Example 2)

[0750] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0751] In today's world, there is a demand for more efficient and effective information transmission in presentations given by individuals and companies. However, preparing for presentations requires a significant amount of time and effort to create materials. Furthermore, the presenter's emotional state during preparation and delivery can greatly impact the outcome, with performance declines due to tension and anxiety being a significant problem. In this context, there is a need to develop a system that can flexibly respond to the presenter's emotional state and provide necessary information in a timely manner.

[0752] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0753] In this invention, the server includes means for providing a user interface for inputting data, means for analyzing the input data and selecting a model form, and means for recognizing the user's emotional state using audio and video data and dynamically adjusting the content of materials and presentations. This enables personalized feedback and material adjustments in response to the user's emotional state during presentation preparation.

[0754] A "user interface" is a means of providing an interface that users use when entering data.

[0755] A "model form" refers to a template or format of a document that is most suitable for the content of the input data, based on its analysis.

[0756] "Structural design" refers to the design procedure for automatically generating the optimal layout and visual elements of a document.

[0757] "Visual elements" refer to elements related to the appearance of a document, such as design and graphics.

[0758] "External information sources" refer to external data sources that are monitored to keep the materials up-to-date.

[0759] "Real-time updates" refers to the process of updating materials in real time based on external information sources.

[0760] A "script" refers to a set of instructions used in a presentation, which guides the speaker on how to proceed.

[0761] A "lesson plan" refers to a guide or plan designed to assist in the progress of a presentation.

[0762] "Simulating a question-and-answer session" means conducting a simulation during a presentation in which you virtually answer questions.

[0763] "Emotional state" refers to the user's psychological and emotional state as recognized using audio and video data.

[0764] "Dynamic adjustment" refers to the process of changing the content of materials and presentations in real time according to the emotional state of the user.

[0765] To implement the present invention, a presentation support system equipped with emotion recognition functionality is required. This system provides a user interface for the user to input data, which is first received by the terminal. The input data is checked for consistency on the terminal before being sent to the server.

[0766] The server analyzes the input data and selects a model form. Based on this selection, the server automatically generates the document's structure and visual elements. The structure design is performed using natural language processing technology to provide an efficient layout for the document. Furthermore, the server monitors data from external sources and processes it to ensure the document is updated in real time.

[0767] The server also includes an emotion recognition engine that analyzes the user's emotional state from audio and video data. This allows the script and lesson plan to dynamically adjust according to the user's emotional state during or after the presentation. For example, if the system detects user anxiety, it may modify the lesson plan to slow down the pace of the presentation and provide advice for relaxation. For instance, the user may receive prompts such as, "What advice have you acted on in recent practice to alleviate anxiety? How have you improved your presentation based on your emotions?", providing new insights for self-improvement.

[0768] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0769] Step 1:

[0770] Users input data through the user interface on their device. The input data undergoes consistency checks on the device, and only consistent data is sent to the server. This process normalizes the input data and converts it into a format suitable for proper transfer to the server.

[0771] Step 2:

[0772] The server analyzes the received data. This analysis utilizes natural language processing techniques to select the most appropriate model form based on the data's content. The input data has its contextual information extracted, and the selected model form is obtained as output. The server's analysis process also includes matching using a similarity database.

[0773] Step 3:

[0774] The server automatically generates the document's structure and visual elements based on the selected template. It visualizes the data content analyzed using natural language processing and designs an effective layout. The generated document is provided in a format that allows the user to visually evaluate it. A template engine is used throughout this process.

[0775] Step 4:

[0776] The server monitors external information sources and updates the data immediately as needed. This process keeps the data up-to-date based on newly acquired data. The retrieval of external data from the database triggers an action to update the data content.

[0777] Step 5:

[0778] The server recognizes the user's emotional state using audio and video data. An emotion recognition engine analyzes the input data in real time to identify the user's emotional patterns. As a result of the processing, the user's current emotional state is output and used to adjust the presentation. In this step, a machine learning model is used for emotion analysis.

[0779] Step 6:

[0780] The server dynamically adjusts the content of materials and presentations based on the user's emotional state. If the user is nervous, advice and specific examples to slow down the script's pace will be displayed on the screen. This adjustment includes sending newly generated prompts to the user to optimize the presentation content.

[0781] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0782] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0783] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0784] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0785] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0786] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0787] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0788] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0789] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0790] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0791] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0792] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0793] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0794] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0795] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0796] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0797] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0798] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0799] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0800] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0801] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0802] The following is further disclosed regarding the embodiments described above.

[0803] (Claim 1)

[0804] A means of providing a user interface for inputting data,

[0805] A means of analyzing the input data and selecting a document template,

[0806] A means for automatically generating the layout and visual elements of a document based on a selected template,

[0807] A means of monitoring external data sources and updating documents in real time,

[0808] A system that includes means for generating presentation scripts and guides, and for simulating question-and-answer sessions.

[0809] (Claim 2)

[0810] The system according to claim 1, wherein the user interface has a function to check the integrity of the input data.

[0811] (Claim 3)

[0812] The system according to claim 1, comprising means for optimizing the layout of materials using natural language processing in the automated material generation process.

[0813] "Example 1"

[0814] (Claim 1)

[0815] Means for providing an input / output device for inputting data,

[0816] A means of analyzing the input information and selecting a document template,

[0817] A means for automatically generating the arrangement of materials and visual elements based on the selected template,

[0818] A means of monitoring external information sources and automatically updating materials to the latest state,

[0819] Methods for creating presentation scripts and lesson plans, and for simulating question-and-answer sessions,

[0820] A system that includes means for selecting the optimal document template by analyzing input information using natural language processing and data analysis technologies.

[0821] (Claim 2)

[0822] The system according to claim 1, wherein the input / output device has a function to verify the integrity of the input information.

[0823] (Claim 3)

[0824] The system according to claim 1, comprising means for optimizing the arrangement of materials using a generation AI model in the automatic document generation process.

[0825] "Application Example 1"

[0826] (Claim 1)

[0827] A means of exchanging information between humans and machines for data input,

[0828] A means of analyzing input data and selecting a portion of the information,

[0829] A means for automatically generating the arrangement of information and visual elements based on selected information,

[0830] A means of monitoring external information sources and updating information in real time,

[0831] A means to generate presentation scripts and guidelines, and to simulate question-and-answer sessions,

[0832] A system that includes means for visually previewing and modifying content using an augmented reality environment.

[0833] (Claim 2)

[0834] The system according to claim 1, wherein the interface for connecting to a machine has a function for verifying the integrity of input information.

[0835] (Claim 3)

[0836] The system according to claim 1, comprising means for optimizing the arrangement of information using natural language processing during the automatic information generation process, thereby enabling visualization in an augmented reality environment.

[0837] "Example 2 of combining an emotion engine"

[0838] (Claim 1)

[0839] Means for providing an interactive device for inputting data,

[0840] A means of analyzing the input data and selecting a document template,

[0841] A means for automatically generating the layout and visual elements of a document based on a selected template,

[0842] A means of monitoring external information sources and continuously updating documents,

[0843] A means of dynamically adjusting the content and presentation method by analyzing the user's emotional state using an emotion analysis engine,

[0844] A system that includes means to adjust the presentation speed according to the user's emotional state and provide guidance to improve stability.

[0845] (Claim 2)

[0846] The system according to claim 1, wherein the dialogue device has a function to verify the integrity of the input data.

[0847] (Claim 3)

[0848] The system according to claim 1, comprising means for optimizing the arrangement of documents using a generation artificial intelligence model in the automatic document generation process.

[0849] "Application example 2 when combining with an emotional engine"

[0850] (Claim 1)

[0851] A means of providing a user interface for entering data,

[0852] A means for analyzing input data and selecting a model form,

[0853] A means for automatically generating the structure and visual elements of a document based on a selected model,

[0854] A means of monitoring external sources and updating materials in real time,

[0855] A means of generating presentation scripts and lesson plans, and simulating question-and-answer sessions,

[0856] A system that uses audio and video data to recognize the emotional state of a user and includes means for dynamically adjusting the content of materials and presentations.

[0857] (Claim 2)

[0858] The system according to claim 1, wherein the user operation screen is equipped with a function to verify the consistency of input data.

[0859] (Claim 3)

[0860] The system according to claim 1, comprising means for optimizing the structure design of a document using non-routine language processing in an automated document generation procedure. [Explanation of Symbols]

[0861] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of exchanging information between humans and machines for data input, A means of analyzing input data and selecting a portion of the information, A means for automatically generating the arrangement of information and visual elements based on selected information, A means of monitoring external information sources and updating information in real time, A means to generate presentation scripts and guidelines, and to simulate question-and-answer sessions, A system that includes means for visually previewing and modifying content using an augmented reality environment.

2. The system according to claim 1, wherein the interface for connecting to a machine has a function for verifying the integrity of input information.

3. The system according to claim 1, comprising means for optimizing the arrangement of information using natural language processing during the automatic information generation process, thereby enabling visualization in an augmented reality environment.

Images