system

Abstract

We provide the system. [Solution] A means of receiving user input and searching past information resources, A means for generating new information resources by combining data from past information resources extracted by the search with new information resources, A means of visualizing newly generated information resources, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, and includes steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance that responds to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Efficient generation and update of information resources are essential in a variety of business environments. However, in conventional methods, past information resources are not effectively utilized, and a great deal of time is often spent. There is a need to solve this problem and reduce the overall time and labor required for information resource generation by effectively reusing the recorded information resources.

Means for Solving the Problems

[0005] This system provides a means to search for past information resources based on user input and combine them with new information to generate new information resources. Furthermore, by visualizing the generated information resources and making them easily accessible to users, it aims to improve the efficiency of conventional information resource generation and updating processes. In this way, it realizes a system that maximizes the use of past information and efficiently provides new information resources.

[0006] "User input" refers to data and instructions provided by users to the system, and is used when generating or updating information resources.

[0007] "Information resources" include documents, presentation slides, and data files stored in electronic format, and provide information necessary for business operations.

[0008] "Past information resources" refer to information resources that were previously created and stored, and which can be reused to generate or update new information resources.

[0009] "Searching" is the process of identifying relevant information resources based on user input, and it uses methods such as keywords and similarity analysis.

[0010] "Generation" is the process of creating new information resources by combining past information resources with new information, and it is carried out according to the user's instructions.

[0011] "Visualization" is the process of representing data and information as charts and graphs, making the information easier to understand intuitively. [Brief explanation of the drawing]

[0012] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which multiple emotions are mapped. [Figure 10] It shows an emotion map to which multiple emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when the emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when the emotion engine is combined.

MODE FOR CARRYING OUT THE INVENTION

[0013] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0014] First, the language used in the following description will be explained.

[0015] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0016] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0017] In the following embodiments, the numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0018] In the following embodiments, the numbered communication I / F (Interface) is an interface including a communication processor and an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark), etc.

[0019] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0020] [First Embodiment]

[0021] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0022] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0023] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0024] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0025] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0026] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0027] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0028] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0029] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0030] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0031] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0032] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0033] This system consists of user, terminal, and server elements, and is designed to support the efficient generation and updating of information resources. The user selects a theme for document creation via the terminal and enters prompts to provide specific details. This input information is then transmitted from the terminal to the server.

[0034] The server has the ability to quickly search relevant historical information resources based on the user's input. This identifies useful historical data and extracts the necessary content. In this process, the server uses methods such as keyword analysis and similarity analysis to efficiently find appropriate materials from historical information resources.

[0035] Subsequently, the server utilizes generation AI to combine the extracted historical information resources with new information to generate new information resources. This process aims to effectively create new documents and presentation materials based on direct user instructions. The server also visualizes the generated information resources as graphs and charts as needed to aid in document comprehension.

[0036] The completed information resources are sent from the server to the terminal and provided to the user. The user can review the completed document and make any necessary adjustments on the terminal to finalize the document. For example, when a user creates an "Annual Business Report," the server automatically searches for report data from the past several years and quickly generates and provides a new report combined with the latest economic indicators. This makes it possible to significantly reduce the time and effort required for document creation compared to traditional manual methods.

[0037] The following describes the processing flow.

[0038] Step 1:

[0039] The user uses a terminal to select a topic for the document and enters specific instructions (prompts). For example, they might select "Quarterly Performance Report" as the topic and enter the instruction "Include sales analysis and market forecast."

[0040] Step 2:

[0041] The terminal sends user input information to the server. This information includes the theme and prompts, and the data is received appropriately on the server side.

[0042] Step 3:

[0043] Based on the theme and prompt received by the server, it searches historical information resources stored in cloud storage and on-premises servers. This search uses keywords and similarity analysis to identify highly relevant materials.

[0044] Step 4:

[0045] The server extracts highly relevant data from past documents based on the search results. For example, this could include historical sales data or information on market trends.

[0046] Step 5:

[0047] The server uses AI generation to combine extracted historical data with new information to create new information resources. Here, content is automatically formed according to the user's instructions.

[0048] Step 6:

[0049] To visualize the information resources generated by the server, graphs and charts are created and incorporated into the document as needed. This visualization process is carried out to facilitate understanding of the document.

[0050] Step 7:

[0051] The server transmits the completed information resources to the terminal, making them accessible to the user. This material is prepared in a format that the user can review.

[0052] Step 8:

[0053] Users can review completed documents on their devices and make necessary edits and adjustments. For example, they can highlight specific sections or perform visual revisions.

[0054] (Example 1)

[0055] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0056] When users create documents or analyze information, manually searching for necessary information from a vast amount of historical data and combining it with new information is a time-consuming and laborious process. Furthermore, visualizing and presenting that information in an easily understandable way requires advanced skills. There is a need for a system that streamlines these tasks, reduces the burden on users, and enables the rapid generation of high-quality information.

[0057] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0058] In this invention, the server includes means for receiving operation instructions from a user and searching for past information, means for generating new information by combining the data of the past information extracted by the search with new information, and means for visually displaying the generated new information. This enables the user to efficiently search, combine, and visualize information and quickly generate data and analysis results.

[0059] "Means for receiving operational instructions from the user" refers to a system that has the function of receiving specific instructions or requests via a device operated by the user and transferring that information to a processing unit.

[0060] A "means of searching for past information" refers to a function that allows for the rapid extraction of relevant data from already accumulated databases and makes it appropriately available for use.

[0061] "Means of generating new information by combining historical data with new information" refers to a function that combines extracted historical data with the latest data and content to create information resources with new value.

[0062] "Means for visually displaying newly generated information" refers to functions that display the created information in formats such as charts and graphs so that users can intuitively understand it.

[0063] "Means of sending to a terminal and enabling the user to review and adjust" refers to a function that sends generated information to a device operated by the user, allowing the user to review the content and make corrections as needed.

[0064] This invention consists of a system that efficiently generates and updates information, comprising user, terminal, and server components.

[0065] First, the user begins the operation using the terminal. The user selects a theme for document creation and enters prompt messages into the terminal that provide specific instructions. These prompt messages describe the outline and requirements of the information the user wants to create. For example, if the theme is "Creation of Year-End Management Report," the user might enter a prompt message such as, "Please generate a new report by combining report data from the past few years with the latest economic indicators."

[0066] The entered prompt text is sent from the terminal to the server. Upon receiving this prompt, the server searches for historical information in its database. This search utilizes advanced natural language processing techniques and data analysis methods, efficiently extracting relevant data through keyword and similarity analysis. Hardware requirements include a server with a high-performance processor and ample storage. Software requirements include generative AI models and text analysis libraries.

[0067] Next, the server uses the extracted historical data to generate new information resources. At this stage, a generative AI model is utilized, and new information is created by combining historical and current data. The generated information is then visualized using advanced graphics processing, and represented as charts and graphs to aid understanding. Software libraries such as Matplotlib and D3.js are used as visualization tools for this process.

[0068] Finally, the completed information resources are sent from the server to the terminal and provided to the user. The user can review these materials on the terminal, make adjustments as needed using word processors and presentation software, and finalize the document. This system allows users to create documents quickly and efficiently, resulting in a significant reduction in time and effort compared to traditional manual methods.

[0069] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0070] Step 1:

[0071] The user selects a theme for document creation on their terminal and enters a prompt message containing specific details. This prompt message might be something like, "Generate a new report by combining report data from the past few years with the latest economic indicators," representing a specific request to the system. This prompt message is then sent to the server, using the user's needs as the starting point for data processing.

[0072] Step 2:

[0073] The terminal sends the prompt text entered by the user to the server using a secure communication protocol. At this stage, the server receives the prompt as a trigger for information retrieval. This input data triggers the server to start the next processing step.

[0074] Step 3:

[0075] Based on the received prompt, the server queries the database to retrieve historical information. This search process uses natural language processing (NLP) techniques to extract keywords from the prompt and score similarity to extract relevant data from the database. The output data is a collection of historical information to address the needs identified in the prompt.

[0076] Step 4:

[0077] The server inputs the extracted historical information, along with new data, into a generative AI model. This input allows the generative AI model to combine the historical and new information to generate new information resources. Data processing utilizes statistical analysis and pattern recognition to extract valuable insights and generate new documents and content as output.

[0078] Step 5:

[0079] The newly generated information is visually processed by the server. Specifically, visualization tools and graphics libraries are used to create graphs and charts to visualize the data. The output at this stage is a document that includes an intuitively understandable visual representation.

[0080] Step 6:

[0081] The completed information resources are sent from the server to the terminal. The terminal displays this document to the user, who then reviews its contents. The user uses word processing or presentation software to make any necessary adjustments to the document, finalizing it as a report or presentation. The output is the final version of the document that the user has reviewed.

[0082] (Application Example 1)

[0083] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0084] Businesses and individuals are required to efficiently and quickly create promotional materials and informational documents for real-world use. In particular, there is a lack of means to create documents that utilize past information while incorporating new market trends, and therefore, technology is needed to facilitate this process.

[0085] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0086] In this invention, the server includes means for receiving user input and searching for past information resources, means for generating new information resources by combining data from past information resources extracted by the search with new information resources, and means for visualizing the generated new information resources. This enables the efficient creation of presentation materials in the real world and the rapid provision of materials that reflect past data and new market trends.

[0087] "User input" refers to instructions or information provided by system users through their terminals.

[0088] "Past information resources" refer to a collection of information that was previously collected and stored in databases or similar systems.

[0089] "Search methods" refer to functions and technologies used to find information resources based on entered keywords and conditions.

[0090] "New information resources" refer to newly generated information and data, which are constructed by combining them with past information resources.

[0091] "Means of generation" refer to technologies and functions for creating new data and materials based on existing information.

[0092] "Means of visualization" refers to the technical process of organizing and displaying generated information in an easily understandable format.

[0093] "Real-world presentation materials" refer to informational materials created for use in a physical space, such as flyers and posters.

[0094] A "portable device" refers to a communication device that can be carried around, such as a smartphone or tablet.

[0095] A "prompt" is a sentence or keyword that a user enters to give instructions to the system.

[0096] "Market trends" refer to commercial or economic movements, specifically trends within a particular market or industry.

[0097] This invention assumes that the user uses a mobile device such as a smartphone to input prompts. The user inputs the theme and requirements for document creation as prompt text. This user input is then transmitted to a server by the device.

[0098] The server quickly searches historical information resources based on the received prompt. This search utilizes techniques such as keyword analysis and similarity analysis to identify appropriate historical information resources. Past sales data and market trends are analyzed, and the necessary data is extracted.

[0099] A generative AI model operates on the server, automatically generating new information resources by combining extracted historical information resources with new market trend data. This process creates documents and presentations. Furthermore, the generated information is visualized and organized in the form of graphs and charts.

[0100] As a concrete example, suppose a user enters the prompt "Create promotional materials for the new fall fashion collection. Include past best-selling items and this year's trends." Based on this prompt, the server extracts data on past best-selling products and quickly generates promotional materials with this year's fashion trend data added.

[0101] The completed information resources are sent from the server to the terminal and made available for use by the user. This entire process utilizes generative AI models, prompt messages, similarity analysis techniques, and historical information resource management techniques.

[0102] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0103] Step 1:

[0104] The user enters a prompt message on their smartphone. The input includes a specific request, such as, "Create promotional materials for the new fall fashion collection. Include past best-selling items and this year's trends." The entered prompt message describes the request to the system.

[0105] Step 2:

[0106] The terminal sends the entered prompt message to the server. This transmission allows the server to receive the user's request. In this case, the prompt message is the input to the server.

[0107] Step 3:

[0108] The server analyzes the received prompt message and extracts keywords for searching past information resources. Using a generative AI model, the server identifies important keywords from the prompt message and prepares for efficient retrieval of relevant past information. This step involves data processing to define the necessary search conditions from the prompt message.

[0109] Step 4:

[0110] The server searches for historical information resources using the extracted keywords. It identifies past sales data and related information from its database and extracts the necessary information. This search result becomes the server's output.

[0111] Step 5:

[0112] The server uses a generative AI model to combine extracted historical information resources with the latest market trend data to generate new information. This process generates materials that serve as specific responses to prompt sentences. Using the extracted data as material, the AI ​​model performs data calculations to create new text and visual materials.

[0113] Step 6:

[0114] The newly generated information resources are visualized by the server. The data is formatted as graphs and charts, transforming it into an easily understandable form. Here, data processing is performed to visually construct the data.

[0115] Step 7:

[0116] The server sends the generated document to the terminal. The user can review the sent document and make corrections as needed. At this stage, the output from the server is sent to the terminal.

[0117] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0118] This system considers user input and emotions when generating and updating information resources, and provides new information resources accordingly. Users select a theme for document creation and enter prompts through a terminal. During this input process, an emotion engine built into the terminal recognizes the user's emotions from their voice and visual feedback.

[0119] The server searches for relevant historical information resources based on the received user input and recognized sentiment data. Furthermore, it adjusts the tone and style to match the user's emotions as determined by the sentiment engine, so that the user can obtain materials that better align with their intentions. For this reason, the retrieved data is structured in a format and presentation style that matches the sentiment information.

[0120] When the server creates new information resources using generative AI, it also incorporates recognized emotions to adjust the content. For example, if a user is creating a presentation and the emotion engine recognizes a calm tone, the presentation will be automatically adjusted to a formal style suitable for business. On the other hand, if the user is excited, the presentation will include more stimulating and visually impactful elements.

[0121] The generated information resources are visualized and organized into an easy-to-understand format. The server sends the completed document to the terminal, where the user can review it and make final adjustments based on their emotions. For example, when a user is creating a performance report, if emotions of joy are recognized, the report will highlight success stories and positive elements.

[0122] This system allows for more precise and efficient document creation, accurately reflecting the user's expected output.

[0123] The following describes the processing flow.

[0124] Step 1:

[0125] The user uses the device to select a theme for the document and enters instructions (prompts) regarding the content they want to generate or update. During this process, the device's built-in emotion engine analyzes the user's facial expressions and tone of voice in real time to recognize their emotions.

[0126] Step 2:

[0127] The device sends user input information and recognized sentiment data to the server. The data sent includes the topic of the material, user prompts, and sentiment state.

[0128] Step 3:

[0129] Based on the theme, prompt data, and sentiment information received by the server, it searches historical information resources stored in the cloud and within the company. Appropriate keywords and similarity analysis are used in the search.

[0130] Step 4:

[0131] The server extracts highly relevant data from past information resources based on the search results. At this stage, it takes emotional information into account and prioritizes selecting data with an appropriate tone and style.

[0132] Step 5:

[0133] The server uses AI generation to combine extracted data with new information to create new information resources. The tone and style of the materials are automatically adjusted to reflect the user's emotions.

[0134] Step 6:

[0135] The server visualizes the generated information resources and creates and incorporates graphs and charts as needed. Emotion-based visual emphasis is also provided.

[0136] Step 7:

[0137] The server transmits the completed information resources to the terminal and provides them to the user. The materials are prepared in a format that the user can easily review.

[0138] Step 8:

[0139] Users can review the completed document on their device and make necessary edits or fine-tune it to suit their mood. For example, they can highlight parts of the document or change the tone.

[0140] (Example 2)

[0141] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0142] In the process of generating information resources, there is a current lack of content provision that takes user emotions into consideration. Conventional systems have difficulty generating materials that match users' emotions and intentions, and flexible adjustments are needed to increase user satisfaction.

[0143] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0144] In this invention, the server includes means for receiving user input and recognizing emotional information based on said input, means for searching past information resources and adjusting the tone and style based on the emotional information, and means for creating new information resources generated by said search and adjustment using a generation AI model. This makes it possible to generate information resources that are in line with the user's emotions and intentions.

[0145] "User input" refers to information provided by users through a terminal, which is used to generate and adjust information resources.

[0146] "Emotional information" refers to data about the user's emotional state, obtained from the user's voice and visual feedback.

[0147] "Past information resources" refer to previously generated information and content stored in databases and storage.

[0148] "Tone and style" refer to elements that shape the appearance and atmosphere of information resources, including formality and visual impression.

[0149] A "generative AI model" refers to an algorithm or program that uses artificial intelligence to generate new content.

[0150] "Visualization" refers to the process of displaying newly generated information resources in a visually easy-to-understand format, making them accessible for viewing.

[0151] In this system, the user selects a theme for document creation using a terminal and enters prompt text. The terminal has a built-in emotion engine that analyzes the user's voice and visual feedback to recognize emotional information. The emotion engine uses voice and image analysis technologies to identify the user's emotions in real time.

[0152] The server receives user input and recognized sentiment information, and searches its database for historical information resources. The search uses SQL queries and search algorithms to identify highly relevant information. Based on the retrieved information, the server generates new information resources by adjusting the tone and style to take sentiment information into account.

[0153] Furthermore, the server uses a generative AI model to create new information resources based on user input prompts and emotional information. The generative AI model utilizes natural language processing technology to generate content that the user desires. For example, if a user requests presentation materials in a calm tone, materials reflecting formal and accurate language will be generated.

[0154] The generated information resources are visualized using visualization tools and organized into an easy-to-understand format. The server sends the final document to the terminal, where the user can review it and make final adjustments to suit their own needs and preferences.

[0155] For example, if a user enters the prompt, "Please create the latest quarterly performance report. The tone should be joyful," the server will generate a report that highlights success stories and emphasizes positive elements. In this way, the system enables the creation of flexible information resources tailored to the user's emotions and intentions.

[0156] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0157] Step 1:

[0158] The user uses a terminal to select a topic for document creation and enter a prompt. This prompt is in text format, such as "Create the latest quarterly earnings report. The tone is joyful." The terminal then prepares to send this input to the server in the appropriate format.

[0159] Step 2:

[0160] An emotion engine built into the device analyzes the user's voice and visual feedback in real time to recognize emotional information. Specifically, voice analysis software analyzes the tone of voice, and image analysis technology identifies emotions from facial expressions. Through this process, emotional information is generated and stored on the device.

[0161] Step 3:

[0162] The terminal sends the user's prompt text and recognized sentiment information to the server. The server receives this data and begins processing it. The received input data includes the prompt text and sentiment information.

[0163] Step 4:

[0164] The server searches the database for historical information resources. During this process, it performs keyword searches and similarity analyses according to the received prompts, extracting relevant information. The output of this process is a list of relevant information as search results.

[0165] Step 5:

[0166] The server uses a generative AI model based on the retrieved information to generate new information resources. At this stage, it takes recognized emotional information into account and adjusts the tone and style. Specifically, the generative AI model combines natural language processing and machine learning techniques to generate content and adjusts its output.

[0167] Step 6:

[0168] The generated information resources are visualized on the server using visualization tools. During visualization, charts and layouts are adjusted and converted into a user-friendly format. This output is the final visualized document.

[0169] Step 7:

[0170] The server sends the completed document to the user's device. The user receives and reviews the document on their device and can make final adjustments to suit their emotional needs as needed. This allows for customization that aligns with the user's intentions.

[0171] (Application Example 2)

[0172] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0173] In today's information society, there is a demand for the rapid and appropriate provision of information that aligns with the emotions and intentions of individual users. In particular, content delivery services tailored to viewers' psychological states and preferences have limitations in terms of diversity and suitability using traditional methods. Therefore, providing appropriate information and content that reflects users' emotions has become a new challenge.

[0174] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0175] In this invention, the server includes means for receiving user input and emotional information and searching a past information set based thereon; means for combining the data from the past information set extracted by the search with a new information set and generating a new information set considering the emotional information; and means for visualizing the generated new information set and presenting it in a format that matches the user's emotions. This makes it possible to provide diverse and appropriate information and content that is in line with the user's emotions.

[0176] "User input" refers to information and instructions entered by the user through a terminal. This information serves as the basic data for the system to generate or retrieve information.

[0177] "Emotional information" refers to data that indicates a user's psychological state and emotions, and is acquired through emotion recognition technology. This information is used to adjust the tone and style of the generated information resources.

[0178] An "information set" refers to a collection of data organized based on a specific category or theme.

[0179] "Past information set" refers to a collection of information accumulated in the past from among existing data sets.

[0180] A "new information set" refers to a newly generated group of information, created by combining existing data and emotional information.

[0181] "Searching methods" refer to the methods and techniques used to extract information from a database.

[0182] "Means of generation" refers to the technology or process for assembling new information resources.

[0183] "Means of visualization" refers to methods and tools for displaying generated information in a way that is easy for users to understand.

[0184] A "system" refers to a platform that integrates these means to generate, search, and display information.

[0185] The embodiment of the invention provides a system that analyzes information and its emotional information entered by a user using a terminal and generates appropriate content. This system uses the following main hardware and software:

[0186] The server receives user input and creates information resources using a generative AI model. The generative AI model has the ability to search for appropriate data from existing information sets and generate new information sets. Furthermore, it adjusts the tone and style of the generated information based on the user's emotional information. Emotional information is acquired through the camera and microphone installed in the device and processed by Google Cloud's emotion recognition API.

[0187] The device captures the user's voice and facial expressions in real time and sends them to the server. This process utilizes the smartphone's camera and microphone. Once emotional information is recognized, the server generates a new set of information accordingly and visualizes it. This visualized information includes rich visuals and interactive elements and is provided to the user as content.

[0188] For example, if a user prompts the system with "I want to watch some relaxing videos," the system recognizes this desire from the user's voice and facial expressions. In response, a playlist containing nature videos and calming music is generated and displayed on the device. Thus, the prompt itself conveys the user's wish in the form of "I want to watch some relaxing videos."

[0189] This system enables the collection and display of information tailored to the user's emotions, realizing personalized content delivery.

[0190] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0191] Step 1:

[0192] The user uses a terminal to input a request for relaxing content, along with a prompt. The input data is text information containing a summary and intent of the content the user is seeking. The terminal receives this input and sends the data to the server.

[0193] Step 2:

[0194] The server analyzes user input data received from the terminal and searches past information sets using a generative AI model. This search involves extracting relevant information from existing databases using keywords associated with the prompt text. The output is the relevant existing information set.

[0195] Step 3:

[0196] The device captures the user's voice and facial expressions using a camera and microphone to acquire emotional information. This is done in real time, and the device analyzes the emotions corresponding to the prompt text entered by the user from the visual and audio data. The input is raw audio and video, and the output is the analyzed emotional information.

[0197] Step 4:

[0198] The server combines acquired emotional information with the retrieved information set to generate a new information set that matches the user's emotions. Specifically, it interprets emotional data using an emotion recognition API, integrates the information using a generative AI model, and adjusts the tone and style. The final output is a customized information set that aligns with the user's emotions.

[0199] Step 5:

[0200] To visualize the newly generated information set, the server sends the results to the terminal. The terminal displays the information, including rich visuals and visually interactive elements, providing content tailored to the user's requests. This output is the content itself, based on the user's prompts.

[0201] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0202] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0203] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0204] [Second Embodiment]

[0205] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0206] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0207] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0208] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0209] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0210] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0211] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0212] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0213] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0214] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0215] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0216] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0217] This system consists of user, terminal, and server elements, and is designed to support the efficient generation and updating of information resources. The user selects a theme for document creation via the terminal and enters prompts to provide specific details. This input information is then transmitted from the terminal to the server.

[0218] The server has the ability to quickly search relevant historical information resources based on the user's input. This identifies useful historical data and extracts the necessary content. In this process, the server uses methods such as keyword analysis and similarity analysis to efficiently find appropriate materials from historical information resources.

[0219] Subsequently, the server utilizes generation AI to combine the extracted historical information resources with new information to generate new information resources. This process aims to effectively create new documents and presentation materials based on direct user instructions. The server also visualizes the generated information resources as graphs and charts as needed to aid in document comprehension.

[0220] The completed information resources are sent from the server to the terminal and provided to the user. The user can review the completed document and make any necessary adjustments on the terminal to finalize the document. For example, when a user creates an "Annual Business Report," the server automatically searches for report data from the past several years and quickly generates and provides a new report combined with the latest economic indicators. This makes it possible to significantly reduce the time and effort required for document creation compared to traditional manual methods.

[0221] The following describes the processing flow.

[0222] Step 1:

[0223] The user uses a terminal to select a topic for the document and enters specific instructions (prompts). For example, they might select "Quarterly Performance Report" as the topic and enter the instruction "Include sales analysis and market forecast."

[0224] Step 2:

[0225] The terminal sends user input information to the server. This information includes the theme and prompts, and the data is received appropriately on the server side.

[0226] Step 3:

[0227] Based on the theme and prompt received by the server, it searches historical information resources stored in cloud storage and on-premises servers. This search uses keywords and similarity analysis to identify highly relevant materials.

[0228] Step 4:

[0229] The server extracts highly relevant data from past documents based on the search results. For example, this could include historical sales data or information on market trends.

[0230] Step 5:

[0231] The server uses AI generation to combine extracted historical data with new information to create new information resources. Here, content is automatically formed according to the user's instructions.

[0232] Step 6:

[0233] To visualize the information resources generated by the server, graphs and charts are created and incorporated into the document as needed. This visualization process is carried out to facilitate understanding of the document.

[0234] Step 7:

[0235] The server transmits the completed information resources to the terminal, making them accessible to the user. This material is prepared in a format that the user can review.

[0236] Step 8:

[0237] Users can review completed documents on their devices and make necessary edits and adjustments. For example, they can highlight specific sections or perform visual revisions.

[0238] (Example 1)

[0239] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0240] When users create documents or analyze information, manually searching for necessary information from a vast amount of historical data and combining it with new information is a time-consuming and laborious process. Furthermore, visualizing and presenting that information in an easily understandable way requires advanced skills. There is a need for a system that streamlines these tasks, reduces the burden on users, and enables the rapid generation of high-quality information.

[0241] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0242] In this invention, the server includes means for receiving operation instructions from a user and searching for past information, means for generating new information by combining the data of the past information extracted by the search with new information, and means for visually displaying the generated new information. This enables the user to efficiently search, combine, and visualize information and quickly generate data and analysis results.

[0243] "Means for receiving operational instructions from the user" refers to a system that has the function of receiving specific instructions or requests via a device operated by the user and transferring that information to a processing unit.

[0244] A "means of searching for past information" refers to a function that allows for the rapid extraction of relevant data from already accumulated databases and makes it appropriately available for use.

[0245] "Means of generating new information by combining historical data with new information" refers to a function that combines extracted historical data with the latest data and content to create information resources with new value.

[0246] "Means for visually displaying newly generated information" refers to functions that display the created information in formats such as charts and graphs so that users can intuitively understand it.

[0247] "Means of sending to a terminal and enabling the user to review and adjust" refers to a function that sends generated information to a device operated by the user, allowing the user to review the content and make corrections as needed.

[0248] This invention consists of a system that efficiently generates and updates information, comprising user, terminal, and server components.

[0249] First, the user begins the operation using the terminal. The user selects a theme for document creation and enters prompt messages into the terminal that provide specific instructions. These prompt messages describe the outline and requirements of the information the user wants to create. For example, if the theme is "Creation of Year-End Management Report," the user might enter a prompt message such as, "Please generate a new report by combining report data from the past few years with the latest economic indicators."

[0250] The entered prompt text is sent from the terminal to the server. Upon receiving this prompt, the server searches for historical information in its database. This search utilizes advanced natural language processing techniques and data analysis methods, efficiently extracting relevant data through keyword and similarity analysis. Hardware requirements include a server with a high-performance processor and ample storage. Software requirements include generative AI models and text analysis libraries.

[0251] Next, the server uses the extracted historical data to generate new information resources. At this stage, a generative AI model is utilized, and new information is created by combining historical and current data. The generated information is then visualized using advanced graphics processing, and represented as charts and graphs to aid understanding. Software libraries such as Matplotlib and D3.js are used as visualization tools for this process.

[0252] Finally, the completed information resources are sent from the server to the terminal and provided to the user. The user can review these materials on the terminal, make adjustments as needed using word processors and presentation software, and finalize the document. This system allows users to create documents quickly and efficiently, resulting in a significant reduction in time and effort compared to traditional manual methods.

[0253] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0254] Step 1:

[0255] The user selects a theme for document creation on their terminal and enters a prompt message containing specific details. This prompt message might be something like, "Generate a new report by combining report data from the past few years with the latest economic indicators," representing a specific request to the system. This prompt message is then sent to the server, using the user's needs as the starting point for data processing.

[0256] Step 2:

[0257] The terminal sends the prompt text entered by the user to the server using a secure communication protocol. At this stage, the server receives the prompt as a trigger for information retrieval. This input data triggers the server to start the next processing step.

[0258] Step 3:

[0259] Based on the received prompt, the server queries the database to retrieve historical information. This search process uses natural language processing (NLP) techniques to extract keywords from the prompt and score similarity to extract relevant data from the database. The output data is a collection of historical information to address the needs identified in the prompt.

[0260] Step 4:

[0261] The server inputs the extracted historical information, along with new data, into a generative AI model. This input allows the generative AI model to combine the historical and new information to generate new information resources. Data processing utilizes statistical analysis and pattern recognition to extract valuable insights and generate new documents and content as output.

[0262] Step 5:

[0263] The newly generated information is visually processed by the server. Specifically, visualization tools and graphics libraries are used to create graphs and charts to visualize the data. The output at this stage is a document that includes an intuitively understandable visual representation.

[0264] Step 6:

[0265] The completed information resources are sent from the server to the terminal. The terminal displays this document to the user, who then reviews its contents. The user uses word processing or presentation software to make any necessary adjustments to the document, finalizing it as a report or presentation. The output is the final version of the document that the user has reviewed.

[0266] (Application Example 1)

[0267] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0268] Businesses and individuals are required to efficiently and quickly create promotional materials and informational documents for real-world use. In particular, there is a lack of means to create documents that utilize past information while incorporating new market trends, and therefore, technology is needed to facilitate this process.

[0269] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0270] In this invention, the server includes means for receiving user input and searching for past information resources, means for generating new information resources by combining data from past information resources extracted by the search with new information resources, and means for visualizing the generated new information resources. This enables the efficient creation of presentation materials in the real world and the rapid provision of materials that reflect past data and new market trends.

[0271] "User input" refers to instructions or information provided by system users through their terminals.

[0272] "Past information resources" refer to a collection of information that was previously collected and stored in databases or similar systems.

[0273] "Search methods" refer to functions and technologies used to find information resources based on entered keywords and conditions.

[0274] "New information resources" refer to newly generated information and data, which are constructed by combining them with past information resources.

[0275] "Means of generation" refer to technologies and functions for creating new data and materials based on existing information.

[0276] "Means of visualization" refers to the technical process of organizing and displaying generated information in an easily understandable format.

[0277] "Real-world presentation materials" refer to informational materials created for use in a physical space, such as flyers and posters.

[0278] A "portable device" refers to a communication device that can be carried around, such as a smartphone or tablet.

[0279] A "prompt" refers to the text or keywords that a user inputs to give instructions to a system.

[0280] "Market trends" refer to commercial or economic movements and indicate trends in a specific market or industry.

[0281] In this invention, it is assumed that a user uses a mobile terminal such as a smartphone to input a prompt. The user inputs the theme and requirements for document creation as a prompt sentence. This user input is transmitted to the server by the terminal.

[0282] The server quickly searches for past information resources based on the received prompt. Technologies such as keywords and similarity analysis are used in this search and are utilized to identify appropriate past information resources. Past sales data and market trends are analyzed, and the required data is extracted.

[0283] A generative AI model operates within the server and automatically generates new information resources by combining the extracted past information resources and new market trend data. In this process, materials such as documents and presentations are created. Furthermore, the generated information is visualized and organized in the form of graphs and charts.

[0284] As a specific example, assume that a user inputs a prompt to the terminal such as "Create a promotion document for the new autumn fashion collection. Include past best-selling items and this year's trends." The server quickly generates a promotion document by extracting past best-selling product data based on this prompt and adding this year's fashion trend data.

[0285] The completed information resources are transmitted from the server to the terminal and provided for the user to use. This series of processes utilizes a generative AI model, prompt sentences, similarity analysis technology, and past information resource management technology.

[0286] The flow of the specific process in Application Example 1 will be described using FIG. 12.

[0287] Step 1:

[0288] The user inputs a prompt sentence on the terminal of the smartphone. The input includes specific requirements such as "Create promotion materials for the new autumn fashion collection. Include best-selling items from the past and this year's trends." The input prompt sentence describes the requirements for the system.

[0289] Step 2:

[0290] The terminal sends the input prompt sentence to the server. Through this transmission, the server can receive the requirements from the user. Here, the prompt sentence serves as the input to the server.

[0291] Step 3:

[0292] The server analyzes the received prompt sentence and extracts keywords for searching past information resources. The server uses the generative AI model to identify important keywords from the prompt sentence and prepares to efficiently search for appropriate past information. In this step, data processing is performed to determine the necessary search conditions from the prompt sentence.

[0293] Step 4:

[0294] The server searches for past information resources using the extracted keywords. It identifies past sales data and related information from the database within the server and extracts the necessary information. This search result becomes the output by the server.

[0295] Step 5:

[0296] The server uses a generative AI model to combine extracted historical information resources with the latest market trend data to generate new information. This process generates materials that serve as specific responses to prompt sentences. Using the extracted data as material, the AI ​​model performs data calculations to create new text and visual materials.

[0297] Step 6:

[0298] The newly generated information resources are visualized by the server. The data is formatted as graphs and charts, transforming it into an easily understandable form. Here, data processing is performed to visually construct the data.

[0299] Step 7:

[0300] The server sends the generated document to the terminal. The user can review the sent document and make corrections as needed. At this stage, the output from the server is sent to the terminal.

[0301] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0302] This system considers user input and emotions when generating and updating information resources, and provides new information resources accordingly. Users select a theme for document creation and enter prompts through a terminal. During this input process, an emotion engine built into the terminal recognizes the user's emotions from their voice and visual feedback.

[0303] The server searches for relevant historical information resources based on the received user input and recognized sentiment data. Furthermore, it adjusts the tone and style to match the user's emotions as determined by the sentiment engine, so that the user can obtain materials that better align with their intentions. For this reason, the retrieved data is structured in a format and presentation style that matches the sentiment information.

[0304] When the server creates a new information resource using generative AI, it also incorporates the recognized emotions to adjust the content. For example, if the user is creating a presentation material and the emotion engine recognizes a calm tone, the material will be automatically adjusted to a formal style suitable for business. On the other hand, if the user is in an excited state, more stimulating and visually impactful elements will be included in the material.

[0305] The generated information resource is organized through a visualization process into an easy-to-understand format. The server sends the completed material to the terminal, allowing the user to review it and make final adjustments according to their emotions. As a specific example, when the user creates a performance report and a happy emotion is recognized, highlights of successful cases and positive elements will be emphasized in the report.

[0306] This system enables the creation of materials to more precisely reflect the output expected by the user and be achieved efficiently compared to normal.

[0307] The processing flow will be described below.

[0308] Step 1:

[0309] The user uses the terminal to select the theme of the material and enter an instruction text (prompt) regarding the content to be generated or updated. At this time, the emotion engine built into the terminal analyzes the user's expression and voice tone in real time to recognize the emotion.

[0310] Step 2:

[0311] The terminal sends the user's input information and the recognized emotion data to the server. The data to be sent includes the theme of the material, the user prompt, and the emotional state.

[0312] Step 3:

[0313] Based on the theme, prompt data, and sentiment information received by the server, it searches historical information resources stored in the cloud and within the company. Appropriate keywords and similarity analysis are used in the search.

[0314] Step 4:

[0315] The server extracts highly relevant data from past information resources based on the search results. At this stage, it takes emotional information into account and prioritizes selecting data with an appropriate tone and style.

[0316] Step 5:

[0317] The server uses AI generation to combine extracted data with new information to create new information resources. The tone and style of the materials are automatically adjusted to reflect the user's emotions.

[0318] Step 6:

[0319] The server visualizes the generated information resources and creates and incorporates graphs and charts as needed. Emotion-based visual emphasis is also provided.

[0320] Step 7:

[0321] The server transmits the completed information resources to the terminal and provides them to the user. The materials are prepared in a format that the user can easily review.

[0322] Step 8:

[0323] Users can review the completed document on their device and make necessary edits or fine-tune it to suit their mood. For example, they can highlight parts of the document or change the tone.

[0324] (Example 2)

[0325] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0326] In the process of generating information resources, there is a current lack of content provision that takes user emotions into consideration. Conventional systems have difficulty generating materials that match users' emotions and intentions, and flexible adjustments are needed to increase user satisfaction.

[0327] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0328] In this invention, the server includes means for receiving user input and recognizing emotional information based on said input, means for searching past information resources and adjusting the tone and style based on the emotional information, and means for creating new information resources generated by said search and adjustment using a generation AI model. This makes it possible to generate information resources that are in line with the user's emotions and intentions.

[0329] "User input" refers to information provided by users through a terminal, which is used to generate and adjust information resources.

[0330] "Emotional information" refers to data about the user's emotional state, obtained from the user's voice and visual feedback.

[0331] "Past information resources" refer to previously generated information and content stored in databases and storage.

[0332] "Tone and style" refer to elements that shape the appearance and atmosphere of information resources, including formality and visual impression.

[0333] A "generative AI model" refers to an algorithm or program that uses artificial intelligence to generate new content.

[0334] "Visualization" refers to the process of displaying newly generated information resources in a visually easy-to-understand format, making them accessible for viewing.

[0335] In this system, the user selects a theme for document creation using a terminal and enters prompt text. The terminal has a built-in emotion engine that analyzes the user's voice and visual feedback to recognize emotional information. The emotion engine uses voice and image analysis technologies to identify the user's emotions in real time.

[0336] The server receives user input and recognized sentiment information, and searches its database for historical information resources. The search uses SQL queries and search algorithms to identify highly relevant information. Based on the retrieved information, the server generates new information resources by adjusting the tone and style to take sentiment information into account.

[0337] Furthermore, the server uses a generative AI model to create new information resources based on user input prompts and emotional information. The generative AI model utilizes natural language processing technology to generate content that the user desires. For example, if a user requests presentation materials in a calm tone, materials reflecting formal and accurate language will be generated.

[0338] The generated information resources are visualized using visualization tools and organized into an easy-to-understand format. The server sends the final document to the terminal, where the user can review it and make final adjustments to suit their own needs and preferences.

[0339] For example, if a user enters the prompt, "Please create the latest quarterly performance report. The tone should be joyful," the server will generate a report that highlights success stories and emphasizes positive elements. In this way, the system enables the creation of flexible information resources tailored to the user's emotions and intentions.

[0340] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0341] Step 1:

[0342] The user uses a terminal to select a topic for document creation and enter a prompt. This prompt is in text format, such as "Create the latest quarterly earnings report. The tone is joyful." The terminal then prepares to send this input to the server in the appropriate format.

[0343] Step 2:

[0344] An emotion engine built into the device analyzes the user's voice and visual feedback in real time to recognize emotional information. Specifically, voice analysis software analyzes the tone of voice, and image analysis technology identifies emotions from facial expressions. Through this process, emotional information is generated and stored on the device.

[0345] Step 3:

[0346] The terminal sends the user's prompt text and recognized sentiment information to the server. The server receives this data and begins processing it. The received input data includes the prompt text and sentiment information.

[0347] Step 4:

[0348] The server searches the database for historical information resources. During this process, it performs keyword searches and similarity analyses according to the received prompts, extracting relevant information. The output of this process is a list of relevant information as search results.

[0349] Step 5:

[0350] The server uses a generative AI model based on the retrieved information to generate new information resources. At this stage, it takes recognized emotional information into account and adjusts the tone and style. Specifically, the generative AI model combines natural language processing and machine learning techniques to generate content and adjusts its output.

[0351] Step 6:

[0352] The generated information resources are visualized on the server using visualization tools. During visualization, charts and layouts are adjusted and converted into a user-friendly format. This output is the final visualized document.

[0353] Step 7:

[0354] The server sends the completed document to the user's device. The user receives and reviews the document on their device and can make final adjustments to suit their emotional needs as needed. This allows for customization that aligns with the user's intentions.

[0355] (Application Example 2)

[0356] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0357] In today's information society, there is a demand for the rapid and appropriate provision of information that aligns with the emotions and intentions of individual users. In particular, content delivery services tailored to viewers' psychological states and preferences have limitations in terms of diversity and suitability using traditional methods. Therefore, providing appropriate information and content that reflects users' emotions has become a new challenge.

[0358] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0359] In this invention, the server includes means for receiving user input and emotional information and searching a past information set based thereon; means for combining the data from the past information set extracted by the search with a new information set and generating a new information set considering the emotional information; and means for visualizing the generated new information set and presenting it in a format that matches the user's emotions. This makes it possible to provide diverse and appropriate information and content that is in line with the user's emotions.

[0360] "User input" refers to information and instructions entered by the user through a terminal. This information serves as the basic data for the system to generate or retrieve information.

[0361] "Emotional information" refers to data that indicates a user's psychological state and emotions, and is acquired through emotion recognition technology. This information is used to adjust the tone and style of the generated information resources.

[0362] An "information set" refers to a collection of data organized based on a specific category or theme.

[0363] "Past information set" refers to a collection of information accumulated in the past from among existing data sets.

[0364] A "new information set" refers to a newly generated group of information, created by combining existing data and emotional information.

[0365] "Searching methods" refer to the methods and techniques used to extract information from a database.

[0366] "Means of generation" refers to the technology or process for assembling new information resources.

[0367] "Means of visualization" refers to methods and tools for displaying generated information in a way that is easy for users to understand.

[0368] A "system" refers to a platform that integrates these means to generate, search, and display information.

[0369] The embodiment of the invention provides a system that analyzes information and its emotional information entered by a user using a terminal and generates appropriate content. This system uses the following main hardware and software:

[0370] The server receives user input and creates information resources using a generative AI model. The generative AI model has the ability to search for appropriate data from existing information sets and generate new information sets. Furthermore, it adjusts the tone and style of the generated information based on the user's emotional information. Emotional information is acquired through the camera and microphone installed in the device and processed by Google Cloud's emotion recognition API.

[0371] The device captures the user's voice and facial expressions in real time and sends them to the server. This process utilizes the smartphone's camera and microphone. Once emotional information is recognized, the server generates a new set of information accordingly and visualizes it. This visualized information includes rich visuals and interactive elements and is provided to the user as content.

[0372] For example, if a user prompts the system with "I want to watch some relaxing videos," the system recognizes this desire from the user's voice and facial expressions. In response, a playlist containing nature videos and calming music is generated and displayed on the device. Thus, the prompt itself conveys the user's wish in the form of "I want to watch some relaxing videos."

[0373] This system enables the collection and display of information tailored to the user's emotions, realizing personalized content delivery.

[0374] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0375] Step 1:

[0376] The user uses a terminal to input a request for relaxing content, along with a prompt. The input data is text information containing a summary and intent of the content the user is seeking. The terminal receives this input and sends the data to the server.

[0377] Step 2:

[0378] The server analyzes user input data received from the terminal and searches past information sets using a generative AI model. This search involves extracting relevant information from existing databases using keywords associated with the prompt text. The output is the relevant existing information set.

[0379] Step 3:

[0380] The device captures the user's voice and facial expressions using a camera and microphone to acquire emotional information. This is done in real time, and the device analyzes the emotions corresponding to the prompt text entered by the user from the visual and audio data. The input is raw audio and video, and the output is the analyzed emotional information.

[0381] Step 4:

[0382] The server combines acquired emotional information with the retrieved information set to generate a new information set that matches the user's emotions. Specifically, it interprets emotional data using an emotion recognition API, integrates the information using a generative AI model, and adjusts the tone and style. The final output is a customized information set that aligns with the user's emotions.

[0383] Step 5:

[0384] To visualize the newly generated information set, the server sends the results to the terminal. The terminal displays the information, including rich visuals and visually interactive elements, providing content tailored to the user's requests. This output is the content itself, based on the user's prompts.

[0385] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0386] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0387] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0388] [Third Embodiment]

[0389] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0390] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0391] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0392] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0393] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0394] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0395] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0396] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0397] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0398] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0399] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0400] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0401] This system consists of user, terminal, and server elements, and is designed to support the efficient generation and updating of information resources. The user selects a theme for document creation via the terminal and enters prompts to provide specific details. This input information is then transmitted from the terminal to the server.

[0402] The server has the ability to quickly search relevant historical information resources based on the user's input. This identifies useful historical data and extracts the necessary content. In this process, the server uses methods such as keyword analysis and similarity analysis to efficiently find appropriate materials from historical information resources.

[0403] Subsequently, the server utilizes generation AI to combine the extracted historical information resources with new information to generate new information resources. This process aims to effectively create new documents and presentation materials based on direct user instructions. The server also visualizes the generated information resources as graphs and charts as needed to aid in document comprehension.

[0404] The completed information resources are sent from the server to the terminal and provided to the user. The user can review the completed document and make any necessary adjustments on the terminal to finalize the document. For example, when a user creates an "Annual Business Report," the server automatically searches for report data from the past several years and quickly generates and provides a new report combined with the latest economic indicators. This makes it possible to significantly reduce the time and effort required for document creation compared to traditional manual methods.

[0405] The following describes the processing flow.

[0406] Step 1:

[0407] The user uses a terminal to select a topic for the document and enters specific instructions (prompts). For example, they might select "Quarterly Performance Report" as the topic and enter the instruction "Include sales analysis and market forecast."

[0408] Step 2:

[0409] The terminal sends user input information to the server. This information includes the theme and prompts, and the data is received appropriately on the server side.

[0410] Step 3:

[0411] Based on the theme and prompt received by the server, it searches historical information resources stored in cloud storage and on-premises servers. This search uses keywords and similarity analysis to identify highly relevant materials.

[0412] Step 4:

[0413] The server extracts highly relevant data from past documents based on the search results. For example, this could include historical sales data or information on market trends.

[0414] Step 5:

[0415] The server uses AI generation to combine extracted historical data with new information to create new information resources. Here, content is automatically formed according to the user's instructions.

[0416] Step 6:

[0417] To visualize the information resources generated by the server, graphs and charts are created and incorporated into the document as needed. This visualization process is carried out to facilitate understanding of the document.

[0418] Step 7:

[0419] The server transmits the completed information resources to the terminal, making them accessible to the user. This material is prepared in a format that the user can review.

[0420] Step 8:

[0421] Users can review completed documents on their devices and make necessary edits and adjustments. For example, they can highlight specific sections or perform visual revisions.

[0422] (Example 1)

[0423] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0424] When users create documents or analyze information, manually searching for necessary information from a vast amount of historical data and combining it with new information is a time-consuming and laborious process. Furthermore, visualizing and presenting that information in an easily understandable way requires advanced skills. There is a need for a system that streamlines these tasks, reduces the burden on users, and enables the rapid generation of high-quality information.

[0425] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0426] In this invention, the server includes means for receiving operation instructions from a user and searching for past information, means for generating new information by combining the data of the past information extracted by the search with new information, and means for visually displaying the generated new information. This enables the user to efficiently search, combine, and visualize information and quickly generate data and analysis results.

[0427] "Means for receiving operational instructions from the user" refers to a system that has the function of receiving specific instructions or requests via a device operated by the user and transferring that information to a processing unit.

[0428] A "means of searching for past information" refers to a function that allows for the rapid extraction of relevant data from already accumulated databases and makes it appropriately available for use.

[0429] "Means of generating new information by combining historical data with new information" refers to a function that combines extracted historical data with the latest data and content to create information resources with new value.

[0430] "Means for visually displaying newly generated information" refers to functions that display the created information in formats such as charts and graphs so that users can intuitively understand it.

[0431] "Means of sending to a terminal and enabling the user to review and adjust" refers to a function that sends generated information to a device operated by the user, allowing the user to review the content and make corrections as needed.

[0432] This invention consists of a system that efficiently generates and updates information, comprising user, terminal, and server components.

[0433] First, the user begins the operation using the terminal. The user selects a theme for document creation and enters prompt messages into the terminal that provide specific instructions. These prompt messages describe the outline and requirements of the information the user wants to create. For example, if the theme is "Creation of Year-End Management Report," the user might enter a prompt message such as, "Please generate a new report by combining report data from the past few years with the latest economic indicators."

[0434] The entered prompt text is sent from the terminal to the server. Upon receiving this prompt, the server searches for historical information in its database. This search utilizes advanced natural language processing techniques and data analysis methods, efficiently extracting relevant data through keyword and similarity analysis. Hardware requirements include a server with a high-performance processor and ample storage. Software requirements include generative AI models and text analysis libraries.

[0435] Next, the server uses the extracted historical data to generate new information resources. At this stage, a generative AI model is utilized, and new information is created by combining historical and current data. The generated information is then visualized using advanced graphics processing, and represented as charts and graphs to aid understanding. Software libraries such as Matplotlib and D3.js are used as visualization tools for this process.

[0436] Finally, the completed information resources are sent from the server to the terminal and provided to the user. The user can review these materials on the terminal, make adjustments as needed using word processors and presentation software, and finalize the document. This system allows users to create documents quickly and efficiently, resulting in a significant reduction in time and effort compared to traditional manual methods.

[0437] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0438] Step 1:

[0439] The user selects a theme for document creation on their terminal and enters a prompt message containing specific details. This prompt message might be something like, "Generate a new report by combining report data from the past few years with the latest economic indicators," representing a specific request to the system. This prompt message is then sent to the server, using the user's needs as the starting point for data processing.

[0440] Step 2:

[0441] The terminal sends the prompt text entered by the user to the server using a secure communication protocol. At this stage, the server receives the prompt as a trigger for information retrieval. This input data triggers the server to start the next processing step.

[0442] Step 3:

[0443] Based on the received prompt, the server queries the database to retrieve historical information. This search process uses natural language processing (NLP) techniques to extract keywords from the prompt and score similarity to extract relevant data from the database. The output data is a collection of historical information to address the needs identified in the prompt.

[0444] Step 4:

[0445] The server inputs the extracted historical information, along with new data, into a generative AI model. This input allows the generative AI model to combine the historical and new information to generate new information resources. Data processing utilizes statistical analysis and pattern recognition to extract valuable insights and generate new documents and content as output.

[0446] Step 5:

[0447] The newly generated information is visually processed by the server. Specifically, visualization tools and graphics libraries are used to create graphs and charts to visualize the data. The output at this stage is a document that includes an intuitively understandable visual representation.

[0448] Step 6:

[0449] The completed information resources are sent from the server to the terminal. The terminal displays this document to the user, who then reviews its contents. The user uses word processing or presentation software to make any necessary adjustments to the document, finalizing it as a report or presentation. The output is the final version of the document that the user has reviewed.

[0450] (Application Example 1)

[0451] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0452] Businesses and individuals are required to efficiently and quickly create promotional materials and informational documents for real-world use. In particular, there is a lack of means to create documents that utilize past information while incorporating new market trends, and therefore, technology is needed to facilitate this process.

[0453] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0454] In this invention, the server includes means for receiving user input and searching for past information resources, means for generating new information resources by combining data from past information resources extracted by the search with new information resources, and means for visualizing the generated new information resources. This enables the efficient creation of presentation materials in the real world and the rapid provision of materials that reflect past data and new market trends.

[0455] "User input" refers to instructions or information provided by system users through their terminals.

[0456] "Past information resources" refer to a collection of information that was previously collected and stored in databases or similar systems.

[0457] "Search methods" refer to functions and technologies used to find information resources based on entered keywords and conditions.

[0458] "New information resources" refer to newly generated information and data, which are constructed by combining them with past information resources.

[0459] "Means of generation" refer to technologies and functions for creating new data and materials based on existing information.

[0460] "Means of visualization" refers to the technical process of organizing and displaying generated information in an easily understandable format.

[0461] "Real-world presentation materials" refer to informational materials created for use in a physical space, such as flyers and posters.

[0462] A "portable device" refers to a communication device that can be carried around, such as a smartphone or tablet.

[0463] A "prompt" is a sentence or keyword that a user enters to give instructions to the system.

[0464] "Market trends" refer to commercial or economic movements, specifically trends within a particular market or industry.

[0465] This invention assumes that the user uses a mobile device such as a smartphone to input prompts. The user inputs the theme and requirements for document creation as prompt text. This user input is then transmitted to a server by the device.

[0466] The server quickly searches historical information resources based on the received prompt. This search utilizes techniques such as keyword analysis and similarity analysis to identify appropriate historical information resources. Past sales data and market trends are analyzed, and the necessary data is extracted.

[0467] A generative AI model operates on the server, automatically generating new information resources by combining extracted historical information resources with new market trend data. This process creates documents and presentations. Furthermore, the generated information is visualized and organized in the form of graphs and charts.

[0468] As a concrete example, suppose a user enters the prompt "Create promotional materials for the new fall fashion collection. Include past best-selling items and this year's trends." Based on this prompt, the server extracts data on past best-selling products and quickly generates promotional materials with this year's fashion trend data added.

[0469] The completed information resources are sent from the server to the terminal and made available for use by the user. This entire process utilizes generative AI models, prompt messages, similarity analysis techniques, and historical information resource management techniques.

[0470] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0471] Step 1:

[0472] The user enters a prompt message on their smartphone. The input includes a specific request, such as, "Create promotional materials for the new fall fashion collection. Include past best-selling items and this year's trends." The entered prompt message describes the request to the system.

[0473] Step 2:

[0474] The terminal sends the entered prompt message to the server. This transmission allows the server to receive the user's request. In this case, the prompt message is the input to the server.

[0475] Step 3:

[0476] The server analyzes the received prompt message and extracts keywords for searching past information resources. Using a generative AI model, the server identifies important keywords from the prompt message and prepares for efficient retrieval of relevant past information. This step involves data processing to define the necessary search conditions from the prompt message.

[0477] Step 4:

[0478] The server searches for historical information resources using the extracted keywords. It identifies past sales data and related information from its database and extracts the necessary information. This search result becomes the server's output.

[0479] Step 5:

[0480] The server uses a generative AI model to combine extracted historical information resources with the latest market trend data to generate new information. This process generates materials that serve as specific responses to prompt sentences. Using the extracted data as material, the AI ​​model performs data calculations to create new text and visual materials.

[0481] Step 6:

[0482] The newly generated information resources are visualized by the server. The data is formatted as graphs and charts, transforming it into an easily understandable form. Here, data processing is performed to visually construct the data.

[0483] Step 7:

[0484] The server sends the generated document to the terminal. The user can review the sent document and make corrections as needed. At this stage, the output from the server is sent to the terminal.

[0485] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0486] This system considers user input and emotions when generating and updating information resources, and provides new information resources accordingly. Users select a theme for document creation and enter prompts through a terminal. During this input process, an emotion engine built into the terminal recognizes the user's emotions from their voice and visual feedback.

[0487] The server searches for relevant historical information resources based on the received user input and recognized sentiment data. Furthermore, it adjusts the tone and style to match the user's emotions as determined by the sentiment engine, so that the user can obtain materials that better align with their intentions. For this reason, the retrieved data is structured in a format and presentation style that matches the sentiment information.

[0488] When the server creates new information resources using generative AI, it also incorporates recognized emotions to adjust the content. For example, if a user is creating a presentation and the emotion engine recognizes a calm tone, the presentation will be automatically adjusted to a formal style suitable for business. On the other hand, if the user is excited, the presentation will include more stimulating and visually impactful elements.

[0489] The generated information resources are visualized and organized into an easy-to-understand format. The server sends the completed document to the terminal, where the user can review it and make final adjustments based on their emotions. For example, when a user is creating a performance report, if emotions of joy are recognized, the report will highlight success stories and positive elements.

[0490] This system allows for more precise and efficient document creation, accurately reflecting the user's expected output.

[0491] The following describes the processing flow.

[0492] Step 1:

[0493] The user uses the device to select a theme for the document and enters instructions (prompts) regarding the content they want to generate or update. During this process, the device's built-in emotion engine analyzes the user's facial expressions and tone of voice in real time to recognize their emotions.

[0494] Step 2:

[0495] The device sends user input information and recognized sentiment data to the server. The data sent includes the topic of the material, user prompts, and sentiment state.

[0496] Step 3:

[0497] Based on the theme, prompt data, and sentiment information received by the server, it searches historical information resources stored in the cloud and within the company. Appropriate keywords and similarity analysis are used in the search.

[0498] Step 4:

[0499] The server extracts highly relevant data from past information resources based on the search results. At this stage, it takes emotional information into account and prioritizes selecting data with an appropriate tone and style.

[0500] Step 5:

[0501] The server uses AI generation to combine extracted data with new information to create new information resources. The tone and style of the materials are automatically adjusted to reflect the user's emotions.

[0502] Step 6:

[0503] The server visualizes the generated information resources and creates and incorporates graphs and charts as needed. Emotion-based visual emphasis is also provided.

[0504] Step 7:

[0505] The server transmits the completed information resources to the terminal and provides them to the user. The materials are prepared in a format that the user can easily review.

[0506] Step 8:

[0507] Users can review the completed document on their device and make necessary edits or fine-tune it to suit their mood. For example, they can highlight parts of the document or change the tone.

[0508] (Example 2)

[0509] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0510] In the process of generating information resources, there is a current lack of content provision that takes user emotions into consideration. Conventional systems have difficulty generating materials that match users' emotions and intentions, and flexible adjustments are needed to increase user satisfaction.

[0511] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0512] In this invention, the server includes means for receiving user input and recognizing emotional information based on said input, means for searching past information resources and adjusting the tone and style based on the emotional information, and means for creating new information resources generated by said search and adjustment using a generation AI model. This makes it possible to generate information resources that are in line with the user's emotions and intentions.

[0513] "User input" refers to information provided by users through a terminal, which is used to generate and adjust information resources.

[0514] "Emotional information" refers to data about the user's emotional state, obtained from the user's voice and visual feedback.

[0515] "Past information resources" refer to previously generated information and content stored in databases and storage.

[0516] "Tone and style" refer to elements that shape the appearance and atmosphere of information resources, including formality and visual impression.

[0517] A "generative AI model" refers to an algorithm or program that uses artificial intelligence to generate new content.

[0518] "Visualization" refers to the process of displaying newly generated information resources in a visually easy-to-understand format, making them accessible for viewing.

[0519] In this system, the user selects a theme for document creation using a terminal and enters prompt text. The terminal has a built-in emotion engine that analyzes the user's voice and visual feedback to recognize emotional information. The emotion engine uses voice and image analysis technologies to identify the user's emotions in real time.

[0520] The server receives user input and recognized sentiment information, and searches its database for historical information resources. The search uses SQL queries and search algorithms to identify highly relevant information. Based on the retrieved information, the server generates new information resources by adjusting the tone and style to take sentiment information into account.

[0521] Furthermore, the server uses a generative AI model to create new information resources based on user input prompts and emotional information. The generative AI model utilizes natural language processing technology to generate content that the user desires. For example, if a user requests presentation materials in a calm tone, materials reflecting formal and accurate language will be generated.

[0522] The generated information resources are visualized using visualization tools and organized into an easy-to-understand format. The server sends the final document to the terminal, where the user can review it and make final adjustments to suit their own needs and preferences.

[0523] For example, if a user enters the prompt, "Please create the latest quarterly performance report. The tone should be joyful," the server will generate a report that highlights success stories and emphasizes positive elements. In this way, the system enables the creation of flexible information resources tailored to the user's emotions and intentions.

[0524] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0525] Step 1:

[0526] The user uses a terminal to select a topic for document creation and enter a prompt. This prompt is in text format, such as "Create the latest quarterly earnings report. The tone is joyful." The terminal then prepares to send this input to the server in the appropriate format.

[0527] Step 2:

[0528] An emotion engine built into the device analyzes the user's voice and visual feedback in real time to recognize emotional information. Specifically, voice analysis software analyzes the tone of voice, and image analysis technology identifies emotions from facial expressions. Through this process, emotional information is generated and stored on the device.

[0529] Step 3:

[0530] The terminal sends the user's prompt text and recognized sentiment information to the server. The server receives this data and begins processing it. The received input data includes the prompt text and sentiment information.

[0531] Step 4:

[0532] The server searches the database for historical information resources. During this process, it performs keyword searches and similarity analyses according to the received prompts, extracting relevant information. The output of this process is a list of relevant information as search results.

[0533] Step 5:

[0534] The server uses a generative AI model based on the retrieved information to generate new information resources. At this stage, it takes recognized emotional information into account and adjusts the tone and style. Specifically, the generative AI model combines natural language processing and machine learning techniques to generate content and adjusts its output.

[0535] Step 6:

[0536] The generated information resources are visualized on the server using visualization tools. During visualization, charts and layouts are adjusted and converted into a user-friendly format. This output is the final visualized document.

[0537] Step 7:

[0538] The server sends the completed document to the user's device. The user receives and reviews the document on their device and can make final adjustments to suit their emotional needs as needed. This allows for customization that aligns with the user's intentions.

[0539] (Application Example 2)

[0540] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0541] In today's information society, there is a demand for the rapid and appropriate provision of information that aligns with the emotions and intentions of individual users. In particular, content delivery services tailored to viewers' psychological states and preferences have limitations in terms of diversity and suitability using traditional methods. Therefore, providing appropriate information and content that reflects users' emotions has become a new challenge.

[0542] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0543] In this invention, the server includes means for receiving user input and emotional information and searching a past information set based thereon; means for combining the data from the past information set extracted by the search with a new information set and generating a new information set considering the emotional information; and means for visualizing the generated new information set and presenting it in a format that matches the user's emotions. This makes it possible to provide diverse and appropriate information and content that is in line with the user's emotions.

[0544] "User input" refers to information and instructions entered by the user through a terminal. This information serves as the basic data for the system to generate or retrieve information.

[0545] "Emotional information" refers to data that indicates a user's psychological state and emotions, and is acquired through emotion recognition technology. This information is used to adjust the tone and style of the generated information resources.

[0546] An "information set" refers to a collection of data organized based on a specific category or theme.

[0547] "Past information set" refers to a collection of information accumulated in the past from among existing data sets.

[0548] A "new information set" refers to a newly generated group of information, created by combining existing data and emotional information.

[0549] "Searching methods" refer to the methods and techniques used to extract information from a database.

[0550] "Means of generation" refers to the technology or process for assembling new information resources.

[0551] "Means of visualization" refers to methods and tools for displaying generated information in a way that is easy for users to understand.

[0552] A "system" refers to a platform that integrates these means to generate, search, and display information.

[0553] The embodiment of the invention provides a system that analyzes information and its emotional information entered by a user using a terminal and generates appropriate content. This system uses the following main hardware and software:

[0554] The server receives user input and creates information resources using a generative AI model. The generative AI model has the ability to search for appropriate data from existing information sets and generate new information sets. Furthermore, it adjusts the tone and style of the generated information based on the user's emotional information. Emotional information is acquired through the camera and microphone installed in the device and processed by Google Cloud's emotion recognition API.

[0555] The device captures the user's voice and facial expressions in real time and sends them to the server. This process utilizes the smartphone's camera and microphone. Once emotional information is recognized, the server generates a new set of information accordingly and visualizes it. This visualized information includes rich visuals and interactive elements and is provided to the user as content.

[0556] For example, if a user prompts the system with "I want to watch some relaxing videos," the system recognizes this desire from the user's voice and facial expressions. In response, a playlist containing nature videos and calming music is generated and displayed on the device. Thus, the prompt itself conveys the user's wish in the form of "I want to watch some relaxing videos."

[0557] This system enables the collection and display of information tailored to the user's emotions, realizing personalized content delivery.

[0558] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0559] Step 1:

[0560] The user uses a terminal to input a request for relaxing content, along with a prompt. The input data is text information containing a summary and intent of the content the user is seeking. The terminal receives this input and sends the data to the server.

[0561] Step 2:

[0562] The server analyzes user input data received from the terminal and searches past information sets using a generative AI model. This search involves extracting relevant information from existing databases using keywords associated with the prompt text. The output is the relevant existing information set.

[0563] Step 3:

[0564] The device captures the user's voice and facial expressions using a camera and microphone to acquire emotional information. This is done in real time, and the device analyzes the emotions corresponding to the prompt text entered by the user from the visual and audio data. The input is raw audio and video, and the output is the analyzed emotional information.

[0565] Step 4:

[0566] The server combines acquired emotional information with the retrieved information set to generate a new information set that matches the user's emotions. Specifically, it interprets emotional data using an emotion recognition API, integrates the information using a generative AI model, and adjusts the tone and style. The final output is a customized information set that aligns with the user's emotions.

[0567] Step 5:

[0568] To visualize the newly generated information set, the server sends the results to the terminal. The terminal displays the information, including rich visuals and visually interactive elements, providing content tailored to the user's requests. This output is the content itself, based on the user's prompts.

[0569] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0570] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0571] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0572] [Fourth Embodiment]

[0573] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0574] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0575] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0576] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0577] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0578] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0579] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0580] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0581] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0582] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0583] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0584] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0585] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0586] This system consists of user, terminal, and server elements, and is designed to support the efficient generation and updating of information resources. The user selects a theme for document creation via the terminal and enters prompts to provide specific details. This input information is then transmitted from the terminal to the server.

[0587] The server has the ability to quickly search relevant historical information resources based on the user's input. This identifies useful historical data and extracts the necessary content. In this process, the server uses methods such as keyword analysis and similarity analysis to efficiently find appropriate materials from historical information resources.

[0588] Subsequently, the server utilizes generation AI to combine the extracted historical information resources with new information to generate new information resources. This process aims to effectively create new documents and presentation materials based on direct user instructions. The server also visualizes the generated information resources as graphs and charts as needed to aid in document comprehension.

[0589] The completed information resources are sent from the server to the terminal and provided to the user. The user can review the completed document and make any necessary adjustments on the terminal to finalize the document. For example, when a user creates an "Annual Business Report," the server automatically searches for report data from the past several years and quickly generates and provides a new report combined with the latest economic indicators. This makes it possible to significantly reduce the time and effort required for document creation compared to traditional manual methods.

[0590] The following describes the processing flow.

[0591] Step 1:

[0592] The user uses a terminal to select a topic for the document and enters specific instructions (prompts). For example, they might select "Quarterly Performance Report" as the topic and enter the instruction "Include sales analysis and market forecast."

[0593] Step 2:

[0594] The terminal sends user input information to the server. This information includes the theme and prompts, and the data is received appropriately on the server side.

[0595] Step 3:

[0596] Based on the theme and prompt received by the server, it searches historical information resources stored in cloud storage and on-premises servers. This search uses keywords and similarity analysis to identify highly relevant materials.

[0597] Step 4:

[0598] The server extracts highly relevant data from past documents based on the search results. For example, this could include historical sales data or information on market trends.

[0599] Step 5:

[0600] The server uses AI generation to combine extracted historical data with new information to create new information resources. Here, content is automatically formed according to the user's instructions.

[0601] Step 6:

[0602] To visualize the information resources generated by the server, graphs and charts are created and incorporated into the document as needed. This visualization process is carried out to facilitate understanding of the document.

[0603] Step 7:

[0604] The server transmits the completed information resources to the terminal, making them accessible to the user. This material is prepared in a format that the user can review.

[0605] Step 8:

[0606] Users can review completed documents on their devices and make necessary edits and adjustments. For example, they can highlight specific sections or perform visual revisions.

[0607] (Example 1)

[0608] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0609] When users create documents or analyze information, manually searching for necessary information from a vast amount of historical data and combining it with new information is a time-consuming and laborious process. Furthermore, visualizing and presenting that information in an easily understandable way requires advanced skills. There is a need for a system that streamlines these tasks, reduces the burden on users, and enables the rapid generation of high-quality information.

[0610] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0611] In this invention, the server includes means for receiving operation instructions from a user and searching for past information, means for generating new information by combining the data of the past information extracted by the search with new information, and means for visually displaying the generated new information. This enables the user to efficiently search, combine, and visualize information and quickly generate data and analysis results.

[0612] "Means for receiving operational instructions from the user" refers to a system that has the function of receiving specific instructions or requests via a device operated by the user and transferring that information to a processing unit.

[0613] A "means of searching for past information" refers to a function that allows for the rapid extraction of relevant data from already accumulated databases and makes it appropriately available for use.

[0614] "Means of generating new information by combining historical data with new information" refers to a function that combines extracted historical data with the latest data and content to create information resources with new value.

[0615] "Means for visually displaying newly generated information" refers to functions that display the created information in formats such as charts and graphs so that users can intuitively understand it.

[0616] "Means of sending to a terminal and enabling the user to review and adjust" refers to a function that sends generated information to a device operated by the user, allowing the user to review the content and make corrections as needed.

[0617] This invention consists of a system that efficiently generates and updates information, comprising user, terminal, and server components.

[0618] First, the user begins the operation using the terminal. The user selects a theme for document creation and enters prompt messages into the terminal that provide specific instructions. These prompt messages describe the outline and requirements of the information the user wants to create. For example, if the theme is "Creation of Year-End Management Report," the user might enter a prompt message such as, "Please generate a new report by combining report data from the past few years with the latest economic indicators."

[0619] The entered prompt text is sent from the terminal to the server. Upon receiving this prompt, the server searches for historical information in its database. This search utilizes advanced natural language processing techniques and data analysis methods, efficiently extracting relevant data through keyword and similarity analysis. Hardware requirements include a server with a high-performance processor and ample storage. Software requirements include generative AI models and text analysis libraries.

[0620] Next, the server uses the extracted historical data to generate new information resources. At this stage, a generative AI model is utilized, and new information is created by combining historical and current data. The generated information is then visualized using advanced graphics processing, and represented as charts and graphs to aid understanding. Software libraries such as Matplotlib and D3.js are used as visualization tools for this process.

[0621] Finally, the completed information resources are sent from the server to the terminal and provided to the user. The user can review these materials on the terminal, make adjustments as needed using word processors and presentation software, and finalize the document. This system allows users to create documents quickly and efficiently, resulting in a significant reduction in time and effort compared to traditional manual methods.

[0622] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0623] Step 1:

[0624] The user selects a theme for document creation on their terminal and enters a prompt message containing specific details. This prompt message might be something like, "Generate a new report by combining report data from the past few years with the latest economic indicators," representing a specific request to the system. This prompt message is then sent to the server, using the user's needs as the starting point for data processing.

[0625] Step 2:

[0626] The terminal sends the prompt text entered by the user to the server using a secure communication protocol. At this stage, the server receives the prompt as a trigger for information retrieval. This input data triggers the server to start the next processing step.

[0627] Step 3:

[0628] Based on the received prompt, the server queries the database to retrieve historical information. This search process uses natural language processing (NLP) techniques to extract keywords from the prompt and score similarity to extract relevant data from the database. The output data is a collection of historical information to address the needs identified in the prompt.

[0629] Step 4:

[0630] The server inputs the extracted historical information, along with new data, into a generative AI model. This input allows the generative AI model to combine the historical and new information to generate new information resources. Data processing utilizes statistical analysis and pattern recognition to extract valuable insights and generate new documents and content as output.

[0631] Step 5:

[0632] The newly generated information is visually processed by the server. Specifically, visualization tools and graphics libraries are used to create graphs and charts to visualize the data. The output at this stage is a document that includes an intuitively understandable visual representation.

[0633] Step 6:

[0634] The completed information resources are sent from the server to the terminal. The terminal displays this document to the user, who then reviews its contents. The user uses word processing or presentation software to make any necessary adjustments to the document, finalizing it as a report or presentation. The output is the final version of the document that the user has reviewed.

[0635] (Application Example 1)

[0636] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0637] Businesses and individuals are required to efficiently and quickly create promotional materials and informational documents for real-world use. In particular, there is a lack of means to create documents that utilize past information while incorporating new market trends, and therefore, technology is needed to facilitate this process.

[0638] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0639] In this invention, the server includes means for receiving user input and searching for past information resources, means for generating new information resources by combining data from past information resources extracted by the search with new information resources, and means for visualizing the generated new information resources. This enables the efficient creation of presentation materials in the real world and the rapid provision of materials that reflect past data and new market trends.

[0640] "User input" refers to instructions or information provided by system users through their terminals.

[0641] "Past information resources" refer to a collection of information that was previously collected and stored in databases or similar systems.

[0642] "Search methods" refer to functions and technologies used to find information resources based on entered keywords and conditions.

[0643] "New information resources" refer to newly generated information and data, which are constructed by combining them with past information resources.

[0644] "Means of generation" refer to technologies and functions for creating new data and materials based on existing information.

[0645] "Means of visualization" refers to the technical process of organizing and displaying generated information in an easily understandable format.

[0646] "Real-world presentation materials" refer to informational materials created for use in a physical space, such as flyers and posters.

[0647] A "portable device" refers to a communication device that can be carried around, such as a smartphone or tablet.

[0648] A "prompt" is a sentence or keyword that a user enters to give instructions to the system.

[0649] "Market trends" refer to commercial or economic movements, specifically trends within a particular market or industry.

[0650] This invention assumes that the user uses a mobile device such as a smartphone to input prompts. The user inputs the theme and requirements for document creation as prompt text. This user input is then transmitted to a server by the device.

[0651] The server quickly searches historical information resources based on the received prompt. This search utilizes techniques such as keyword analysis and similarity analysis to identify appropriate historical information resources. Past sales data and market trends are analyzed, and the necessary data is extracted.

[0652] A generative AI model operates on the server, automatically generating new information resources by combining extracted historical information resources with new market trend data. This process creates documents and presentations. Furthermore, the generated information is visualized and organized in the form of graphs and charts.

[0653] As a concrete example, suppose a user enters the prompt "Create promotional materials for the new fall fashion collection. Include past best-selling items and this year's trends." Based on this prompt, the server extracts data on past best-selling products and quickly generates promotional materials with this year's fashion trend data added.

[0654] The completed information resources are sent from the server to the terminal and made available for use by the user. This entire process utilizes generative AI models, prompt messages, similarity analysis techniques, and historical information resource management techniques.

[0655] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0656] Step 1:

[0657] The user enters a prompt message on their smartphone. The input includes a specific request, such as, "Create promotional materials for the new fall fashion collection. Include past best-selling items and this year's trends." The entered prompt message describes the request to the system.

[0658] Step 2:

[0659] The terminal sends the entered prompt message to the server. This transmission allows the server to receive the user's request. In this case, the prompt message is the input to the server.

[0660] Step 3:

[0661] The server analyzes the received prompt message and extracts keywords for searching past information resources. Using a generative AI model, the server identifies important keywords from the prompt message and prepares for efficient retrieval of relevant past information. This step involves data processing to define the necessary search conditions from the prompt message.

[0662] Step 4:

[0663] The server searches for historical information resources using the extracted keywords. It identifies past sales data and related information from its database and extracts the necessary information. This search result becomes the server's output.

[0664] Step 5:

[0665] The server uses a generative AI model to combine extracted historical information resources with the latest market trend data to generate new information. This process generates materials that serve as specific responses to prompt sentences. Using the extracted data as material, the AI ​​model performs data calculations to create new text and visual materials.

[0666] Step 6:

[0667] The newly generated information resources are visualized by the server. The data is formatted as graphs and charts, transforming it into an easily understandable form. Here, data processing is performed to visually construct the data.

[0668] Step 7:

[0669] The server sends the generated document to the terminal. The user can review the sent document and make corrections as needed. At this stage, the output from the server is sent to the terminal.

[0670] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0671] This system considers user input and emotions when generating and updating information resources, and provides new information resources accordingly. Users select a theme for document creation and enter prompts through a terminal. During this input process, an emotion engine built into the terminal recognizes the user's emotions from their voice and visual feedback.

[0672] The server searches for relevant historical information resources based on the received user input and recognized sentiment data. Furthermore, it adjusts the tone and style to match the user's emotions as determined by the sentiment engine, so that the user can obtain materials that better align with their intentions. For this reason, the retrieved data is structured in a format and presentation style that matches the sentiment information.

[0673] When the server creates new information resources using generative AI, it also incorporates recognized emotions to adjust the content. For example, if a user is creating a presentation and the emotion engine recognizes a calm tone, the presentation will be automatically adjusted to a formal style suitable for business. On the other hand, if the user is excited, the presentation will include more stimulating and visually impactful elements.

[0674] The generated information resources are visualized and organized into an easy-to-understand format. The server sends the completed document to the terminal, where the user can review it and make final adjustments based on their emotions. For example, when a user is creating a performance report, if emotions of joy are recognized, the report will highlight success stories and positive elements.

[0675] This system allows for more precise and efficient document creation, accurately reflecting the user's expected output.

[0676] The following describes the processing flow.

[0677] Step 1:

[0678] The user uses the device to select a theme for the document and enters instructions (prompts) regarding the content they want to generate or update. During this process, the device's built-in emotion engine analyzes the user's facial expressions and tone of voice in real time to recognize their emotions.

[0679] Step 2:

[0680] The device sends user input information and recognized sentiment data to the server. The data sent includes the topic of the material, user prompts, and sentiment state.

[0681] Step 3:

[0682] Based on the theme, prompt data, and sentiment information received by the server, it searches historical information resources stored in the cloud and within the company. Appropriate keywords and similarity analysis are used in the search.

[0683] Step 4:

[0684] The server extracts highly relevant data from past information resources based on the search results. At this stage, it takes emotional information into account and prioritizes selecting data with an appropriate tone and style.

[0685] Step 5:

[0686] The server uses AI generation to combine extracted data with new information to create new information resources. The tone and style of the materials are automatically adjusted to reflect the user's emotions.

[0687] Step 6:

[0688] The server visualizes the generated information resources and creates and incorporates graphs and charts as needed. Emotion-based visual emphasis is also provided.

[0689] Step 7:

[0690] The server transmits the completed information resources to the terminal and provides them to the user. The materials are prepared in a format that the user can easily review.

[0691] Step 8:

[0692] Users can review the completed document on their device and make necessary edits or fine-tune it to suit their mood. For example, they can highlight parts of the document or change the tone.

[0693] (Example 2)

[0694] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0695] In the process of generating information resources, there is a current lack of content provision that takes user emotions into consideration. Conventional systems have difficulty generating materials that match users' emotions and intentions, and flexible adjustments are needed to increase user satisfaction.

[0696] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0697] In this invention, the server includes means for receiving user input and recognizing emotional information based on said input, means for searching past information resources and adjusting the tone and style based on the emotional information, and means for creating new information resources generated by said search and adjustment using a generation AI model. This makes it possible to generate information resources that are in line with the user's emotions and intentions.

[0698] "User input" refers to information provided by users through a terminal, which is used to generate and adjust information resources.

[0699] "Emotional information" refers to data about the user's emotional state, obtained from the user's voice and visual feedback.

[0700] "Past information resources" refer to previously generated information and content stored in databases and storage.

[0701] "Tone and style" refer to elements that shape the appearance and atmosphere of information resources, including formality and visual impression.

[0702] A "generative AI model" refers to an algorithm or program that uses artificial intelligence to generate new content.

[0703] "Visualization" refers to the process of displaying newly generated information resources in a visually easy-to-understand format, making them accessible for viewing.

[0704] In this system, the user selects a theme for document creation using a terminal and enters prompt text. The terminal has a built-in emotion engine that analyzes the user's voice and visual feedback to recognize emotional information. The emotion engine uses voice and image analysis technologies to identify the user's emotions in real time.

[0705] The server receives user input and recognized sentiment information, and searches its database for historical information resources. The search uses SQL queries and search algorithms to identify highly relevant information. Based on the retrieved information, the server generates new information resources by adjusting the tone and style to take sentiment information into account.

[0706] Furthermore, the server uses a generative AI model to create new information resources based on user input prompts and emotional information. The generative AI model utilizes natural language processing technology to generate content that the user desires. For example, if a user requests presentation materials in a calm tone, materials reflecting formal and accurate language will be generated.

[0707] The generated information resources are visualized using visualization tools and organized into an easy-to-understand format. The server sends the final document to the terminal, where the user can review it and make final adjustments to suit their own needs and preferences.

[0708] For example, if a user enters the prompt, "Please create the latest quarterly performance report. The tone should be joyful," the server will generate a report that highlights success stories and emphasizes positive elements. In this way, the system enables the creation of flexible information resources tailored to the user's emotions and intentions.

[0709] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0710] Step 1:

[0711] The user uses a terminal to select a topic for document creation and enter a prompt. This prompt is in text format, such as "Create the latest quarterly earnings report. The tone is joyful." The terminal then prepares to send this input to the server in the appropriate format.

[0712] Step 2:

[0713] An emotion engine built into the device analyzes the user's voice and visual feedback in real time to recognize emotional information. Specifically, voice analysis software analyzes the tone of voice, and image analysis technology identifies emotions from facial expressions. Through this process, emotional information is generated and stored on the device.

[0714] Step 3:

[0715] The terminal sends the user's prompt text and recognized sentiment information to the server. The server receives this data and begins processing it. The received input data includes the prompt text and sentiment information.

[0716] Step 4:

[0717] The server searches the database for historical information resources. During this process, it performs keyword searches and similarity analyses according to the received prompts, extracting relevant information. The output of this process is a list of relevant information as search results.

[0718] Step 5:

[0719] The server uses a generative AI model based on the retrieved information to generate new information resources. At this stage, it takes recognized emotional information into account and adjusts the tone and style. Specifically, the generative AI model combines natural language processing and machine learning techniques to generate content and adjusts its output.

[0720] Step 6:

[0721] The generated information resources are visualized on the server using visualization tools. During visualization, charts and layouts are adjusted and converted into a user-friendly format. This output is the final visualized document.

[0722] Step 7:

[0723] The server sends the completed document to the user's device. The user receives and reviews the document on their device and can make final adjustments to suit their emotional needs as needed. This allows for customization that aligns with the user's intentions.

[0724] (Application Example 2)

[0725] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0726] In today's information society, there is a demand for the rapid and appropriate provision of information that aligns with the emotions and intentions of individual users. In particular, content delivery services tailored to viewers' psychological states and preferences have limitations in terms of diversity and suitability using traditional methods. Therefore, providing appropriate information and content that reflects users' emotions has become a new challenge.

[0727] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0728] In this invention, the server includes means for receiving user input and emotional information and searching a past information set based thereon; means for combining the data from the past information set extracted by the search with a new information set and generating a new information set considering the emotional information; and means for visualizing the generated new information set and presenting it in a format that matches the user's emotions. This makes it possible to provide diverse and appropriate information and content that is in line with the user's emotions.

[0729] "User input" refers to information and instructions entered by the user through a terminal. This information serves as the basic data for the system to generate or retrieve information.

[0730] "Emotional information" refers to data that indicates a user's psychological state and emotions, and is acquired through emotion recognition technology. This information is used to adjust the tone and style of the generated information resources.

[0731] An "information set" refers to a collection of data organized based on a specific category or theme.

[0732] "Past information set" refers to a collection of information accumulated in the past from among existing data sets.

[0733] A "new information set" refers to a newly generated group of information, created by combining existing data and emotional information.

[0734] "Searching methods" refer to the methods and techniques used to extract information from a database.

[0735] "Means of generation" refers to the technology or process for assembling new information resources.

[0736] "Means of visualization" refers to methods and tools for displaying generated information in a way that is easy for users to understand.

[0737] A "system" refers to a platform that integrates these means to generate, search, and display information.

[0738] The embodiment of the invention provides a system that analyzes information and its emotional information entered by a user using a terminal and generates appropriate content. This system uses the following main hardware and software:

[0739] The server receives user input and creates information resources using a generative AI model. The generative AI model has the ability to search for appropriate data from existing information sets and generate new information sets. Furthermore, it adjusts the tone and style of the generated information based on the user's emotional information. Emotional information is acquired through the camera and microphone installed in the device and processed by Google Cloud's emotion recognition API.

[0740] The device captures the user's voice and facial expressions in real time and sends them to the server. This process utilizes the smartphone's camera and microphone. Once emotional information is recognized, the server generates a new set of information accordingly and visualizes it. This visualized information includes rich visuals and interactive elements and is provided to the user as content.

[0741] For example, if a user prompts the system with "I want to watch some relaxing videos," the system recognizes this desire from the user's voice and facial expressions. In response, a playlist containing nature videos and calming music is generated and displayed on the device. Thus, the prompt itself conveys the user's wish in the form of "I want to watch some relaxing videos."

[0742] This system enables the collection and display of information tailored to the user's emotions, realizing personalized content delivery.

[0743] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0744] Step 1:

[0745] The user uses a terminal to input a request for relaxing content, along with a prompt. The input data is text information containing a summary and intent of the content the user is seeking. The terminal receives this input and sends the data to the server.

[0746] Step 2:

[0747] The server analyzes user input data received from the terminal and searches past information sets using a generative AI model. This search involves extracting relevant information from existing databases using keywords associated with the prompt text. The output is the relevant existing information set.

[0748] Step 3:

[0749] The device captures the user's voice and facial expressions using a camera and microphone to acquire emotional information. This is done in real time, and the device analyzes the emotions corresponding to the prompt text entered by the user from the visual and audio data. The input is raw audio and video, and the output is the analyzed emotional information.

[0750] Step 4:

[0751] The server combines acquired emotional information with the retrieved information set to generate a new information set that matches the user's emotions. Specifically, it interprets emotional data using an emotion recognition API, integrates the information using a generative AI model, and adjusts the tone and style. The final output is a customized information set that aligns with the user's emotions.

[0752] Step 5:

[0753] To visualize the newly generated information set, the server sends the results to the terminal. The terminal displays the information, including rich visuals and visually interactive elements, providing content tailored to the user's requests. This output is the content itself, based on the user's prompts.

[0754] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0755] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0756] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0757] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0758] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0759] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0760] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0761] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0762] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0763] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0764] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0765] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0766] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0767] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0768] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0769] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0770] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0771] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0772] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0773] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0774] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0775] The following is further disclosed regarding the embodiments described above.

[0776] (Claim 1)

[0777] A means of receiving user input and searching past information resources,

[0778] A means for generating new information resources by combining data from past information resources extracted by the search with new information resources,

[0779] A means of visualizing newly generated information resources,

[0780] A system that includes this.

[0781] (Claim 2)

[0782] The system according to claim 1, further comprising means for using keywords or similarity analysis to identify suitable historical information resources based on user input.

[0783] (Claim 3)

[0784] The system according to claim 1, further comprising means for storing newly generated information resources and providing them to the user in a format that can be accessed later.

[0785] "Example 1"

[0786] (Claim 1)

[0787] A means of receiving operation instructions from the user and searching for past information,

[0788] A means for generating new information by combining the past information data extracted by the search with new information,

[0789] A means of visually displaying the newly generated information,

[0790] A means of sending newly generated information to a terminal so that the user can review and adjust it,

[0791] A system that includes this.

[0792] (Claim 2)

[0793] The system according to claim 1, further comprising means for using keyword or content matching analysis to identify relevant past information based on user instructions.

[0794] (Claim 3)

[0795] The system according to claim 1, further comprising means for storing newly generated information and providing it to the user in a format that can be accessed at a later date.

[0796] "Application Example 1"

[0797] (Claim 1)

[0798] A means of receiving user input and searching past information resources,

[0799] A means for generating new information resources by combining data from past information resources extracted by the search with new information resources,

[0800] A means of visualizing newly generated information resources,

[0801] A means of quickly creating presentation materials in the real world,

[0802] By analyzing past sales data and market trends, a means of providing new information resources,

[0803] A means by which the user enters prompts using a mobile device,

[0804] A system that includes this.

[0805] (Claim 2)

[0806] The system according to claim 1, further comprising means for using keywords or similarity analysis to identify suitable historical information resources based on user input.

[0807] (Claim 3)

[0808] The system according to claim 1, further comprising means for storing newly generated information resources and providing them to the user in a format that can be accessed later.

[0809] "Example 2 of combining an emotion engine"

[0810] (Claim 1)

[0811] A means for receiving user input and recognizing emotional information based on said input,

[0812] A means of searching past information resources and adjusting tone and style based on emotional information,

[0813] A means for creating new information resources generated by the search and adjustment using a generation AI model,

[0814] A means of visualizing newly generated information resources,

[0815] A system that includes this.

[0816] (Claim 2)

[0817] The system according to claim 1, comprising means for using keywords or similarity analysis to identify suitable historical information resources based on user input, and adjusting information resources based on sentiment information.

[0818] (Claim 3)

[0819] The system according to claim 1, further comprising means for storing newly generated information resources and providing them to the user in a format that can be accessed later.

[0820] "Application example 2 when combining with an emotional engine"

[0821] (Claim 1)

[0822] A means for receiving user input and sentiment information, and for searching a past information set based on that,

[0823] A means for generating a new information set by combining data from a past information set extracted by the search with a new information set and considering emotional information,

[0824] A means of visualizing the newly generated information set and presenting it in a format that matches the user's emotions,

[0825] A system that includes this.

[0826] (Claim 2)

[0827] The system according to claim 1, further comprising means for using keywords, similarity analysis, or sentiment analysis to identify a suitable set of historical information based on user input and sentiment information.

[0828] (Claim 3)

[0829] The system according to claim 1, further comprising means for storing the generated new information set, providing it to the user in a format that can be accessed later, and making recommendations based on the user's sentiment. [Explanation of Symbols]

[0830] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of receiving user input and searching past information resources, A means for generating new information resources by combining data from past information resources extracted by the search with new information resources, A means of visualizing newly generated information resources, A system that includes this.

2. The system according to claim 1, further comprising means for using keywords or similarity analysis to identify suitable historical information resources based on user input.

3. The system according to claim 1, further comprising means for storing newly generated information resources and providing them to the user in a format that can be accessed later.

Images