System

Abstract

A system is provided.SOLUTION: A system comprising: means for collecting data from a plurality of data sources; means for converting the collected data into a uniform format; means for analyzing the data converted into the uniform format; means for automatically generating an output based on the analyzed data; means for specifying a source of the data in the output; means for providing the generated output to a user and receiving a modification from the user; and means for regenerating a final output reflecting the modification.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In today's corporate environment, data is stored in a variety of formats and in different locations, making it extremely time-consuming to find the information you need. This requires a huge amount of time and effort to create documents and analyze data, significantly reducing business efficiency. The purpose of this invention is to improve this situation and automate the process from data collection to output generation, thereby enabling efficient data utilization and improving business efficiency. [Means for solving the problem]

[0005] The present invention solves the above problems by providing a system including the following means.

[0006] A means of collecting data from multiple sources

[0007] A means of converting collected data into a unified format

[0008] A means of analyzing data converted into a unified format

[0009] A means of automatically generating output based on the analyzed data

[0010] A means of citing data sources in output

[0011] A means of providing the generated output to the user and accepting user corrections

[0012] A means to regenerate the final output reflecting the modifications

[0013] The present invention also provides an interactive interface, includes a means for receiving a data collection request from a user in natural language, and a means for analyzing collected data in real time, thereby further improving the efficiency of data collection and analysis, allowing users to significantly reduce the time and effort required for data collection and analysis and quickly and accurately obtain the required output.

[0014] "Data source" refers to the systems, services, and cloud storage from which data is provided that may be stored in different locations or formats.

[0015] "Data collection" refers to the process of automatically obtaining required data from multiple data sources based on specified criteria.

[0016] "Unified format" refers to a data format that converts data collected from different data sources into a common format and makes it possible to analyze it consistently.

[0017] "Analysis" refers to the process of processing collected data based on specific algorithms and rules to extract the necessary information.

[0018] "Output" refers to any output in the form of a report, spreadsheet, presentation, or the like that is generated based on the analyzed data.

[0019] "Data Source" refers to information that clearly indicates from which data source the data included in the Outputs was obtained.

[0020] "User" refers to an end user who requests data collection, analysis, and output generation through this system and uses the resulting output.

[0021] An "interactive interface" refers to an interface that allows a user to instruct a system to collect and analyze data using interactive means such as chat or voice input. [Brief explanation of the drawings]

[0022] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0023] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0024] First, the terms used in the following description will be explained.

[0025] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0026] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0027] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0028] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0029] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0030] [First embodiment]

[0031] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0032] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0033] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0034] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0035] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0036] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0037] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0038] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0039] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0040] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0041] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0042] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0043] The present invention relates to a system for automating data collection, analysis, and output generation in an enterprise environment, which can efficiently collect data from multiple data sources, convert it into a unified format, analyze it, and ultimately automatically generate the output required by the user.

[0044] Explanation of program processing

[0045] The program of this system performs processing in the following procedure.

[0046] 1. Data Source Registration

[0047] During initial setup, the user enters connection information for cloud storage and data warehouses into the system.

[0048] The server uses the entered information to connect to each data source and obtain an API key and authentication token.

[0049] The server stores information about data sources with which a connection has been successfully established in an internal database.

[0050] 2. Data Collection Requests

[0051] A user uses a conversational interface to make a data collection request in natural language (e.g., "Collect sales data for 2023").

[0052] The server receives the request and analyzes the request using a natural language processing engine.

[0053] The server identifies specific data collection conditions (e.g., period, data items) based on the analysis results.

[0054] 3. Data collection and conversion

[0055] The server automatically collects the required data from each data source based on the specified conditions.

[0056] For example, download a spreadsheet from cloud storage and run an SQL query from the data warehouse to get the results.

[0057] The server converts the collected data into a unified format (e.g., CSV, JSON).

[0058] 4. Data Analysis

[0059] The server then applies an analysis algorithm to the data converted into a unified format to extract the necessary information.

[0060] The analysis results are made available to users in the format they request.

[0061] 5. Generating Output

[0062] The server automatically generates output based on the analysis results, which can be in the form of a spreadsheet, PDF report, presentation material, or other format, depending on the user's request.

[0063] Generated outputs will clearly indicate the source from which the data was obtained.

[0064] 6. Providing and Modifying Output

[0065] The server temporarily stores the generated output and notifies the user.

[0066] After receiving the notification, the user logs into the system to view the generated output.

[0067] The user makes any necessary corrections to the output content and sends the corrections to the system.

[0068] The server then incorporates the modifications received from the user and recreates the final output.

[0069] The server provides the final output to the user and shares it via a download link, email, or other means.

[0070] Specific examples

[0071] For example, the following describes an example in which a company's sales department generates a monthly report based on sales data for the past year.

[0072] 1. A user logs in to the system and requests, "Create a monthly report based on sales data from the past year."

[0073] 2. The server parses this request and collects the past year's sales data from a spreadsheet in cloud storage and also retrieves the necessary information from the data warehouse.

[0074] 3. The server converts the collected data into a unified format and applies an analysis algorithm to compile sales figures for each month.

[0075] 4. The server generates a monthly report in spreadsheet format based on the analysis results, and also indicates the source of the data.

[0076] 5. The user reviews the generated report, makes any necessary corrections, and then downloads the final report for use.

[0077] In this way, the present invention efficiently automates a series of operations from data collection to output generation, significantly reducing the workload on the user.

[0078] The processing flow will be explained below.

[0079] Step 1:

[0080] The user accesses the system's initial setup screen and enters connection information for data sources such as cloud storage and data warehouses.

[0081] Step 2:

[0082] The server connects to each data source using the entered connection information to obtain an API key and authentication token, and if successful, stores the data source information in an internal database.

[0083] Step 3:

[0084] A user uses a conversational interface to input a data collection request in natural language (e.g., "Collect monthly sales data for the year 2023").

[0085] Step 4:

[0086] The server sends the user's request to a natural language processing engine, which analyzes the request. Based on the analysis results, specific data collection conditions (e.g., period, data items) are extracted.

[0087] Step 5:

[0088] The server begins collecting data for each data source based on the specified data collection conditions.

[0089] For example, download a spreadsheet from cloud storage and submit SQL queries to a data warehouse to retrieve the required data.

[0090] Step 6:

[0091] The server converts the collected data from each data source into a unified format (e.g., CSV, JSON), which also includes data cleansing and normalization steps.

[0092] Step 7:

[0093] The server inputs the data converted into a unified format into an analysis algorithm, extracting and aggregating data that meets the specified conditions.

[0094] Step 8:

[0095] The server automatically generates output based on the analysis results in the specified output format (e.g., spreadsheet, PDF, presentation), including information on the origin of the collected data.

[0096] Step 9:

[0097] The server temporarily stores the automatically generated output and sends a notification to the user.

[0098] Step 10:

[0099] The user receives the notification sent and accesses the system to view the generated output.

[0100] Step 11:

[0101] The user checks the output content, makes corrections as necessary, and sends the corrections to the system.

[0102] Step 12:

[0103] The server receives the user's modifications and recreates the final output to reflect those modifications.

[0104] Step 13:

[0105] The server then provides the final regenerated output to the user and shares it via a download link, email, or other means.

[0106] Example 1

[0107] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0108] In a corporate environment, collecting data from multiple sources, converting it into a unified format, analyzing it, and automatically generating the final output is a very time-consuming process. It also requires a lot of time and effort to clearly indicate the source of the information and regenerate the output in response to user requests for corrections. Furthermore, the lack of a system that can accept user requests in natural language through a dialogue interface and analyze them in real time impairs user convenience. These issues need to be addressed.

[0109] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0110] In this invention, the server includes: [means for collecting information from multiple information sources]; [means for converting the collected information into a unified format]; and [means for analyzing the information converted into the unified format]. This makes it possible to efficiently automate a series of tasks from collecting information to analyzing it and generating output.

[0111] The "means for collecting information from multiple information sources" is a mechanism for automatically obtaining necessary information from multiple information sources specified by the user.

[0112] "Means for converting collected information into a unified format" refers to a mechanism for converting information collected in different formats into a unified format.

[0113] "Means for analyzing information converted into a unified format" refers to a mechanism for performing the necessary analysis based on information converted into a unified format and extracting meaningful data.

[0114] The "means for automatically generating output based on analyzed information" is a mechanism for automatically generating output in a format requested by a user based on analyzed information.

[0115] "Means for indicating the source of information in the output" refers to a mechanism for indicating from which source the information in the generated output was obtained.

[0116] The "means for providing the generated output to the user and accepting corrections from the user" is a mechanism for providing the generated output to the user and accepting corrections instructed by the user as necessary.

[0117] The "means for regenerating the final output reflecting the corrections" is a mechanism for regenerating the final output based on the corrections provided by the user.

[0118] "Means for providing an interactive interface and receiving information gathering requests in natural language from users" refers to a mechanism for providing an interactive interface that allows users to make information gathering requests in natural language and for receiving such requests.

[0119] "Means for analyzing collected information in real time" refers to a mechanism for instantly analyzing collected information and providing results in real time.

[0120] The present invention provides a system for automating data collection, analysis, and output generation in an enterprise environment. This system efficiently collects information from multiple sources, converts it into a unified format, analyzes it, and finally automatically generates the output required by the user.

[0121] System Overview

[0122] This system consists of a server, terminals, and users. The server performs the following main functions:

[0123] 1. A means of gathering information from multiple sources

[0124] 2. A means of converting the collected information into a unified format

[0125] 3. A means of analyzing information converted into a unified format

[0126] 4. A means to automatically generate output based on the analyzed information

[0127] 5. A means of citing the source of information in your output

[0128] 6. A means of providing the generated output to users and accepting user corrections.

[0129] 7. A means to reproduce the final output reflecting the modifications.

[0130] 8. Means for providing an interactive interface and receiving natural language information gathering requests from users

[0131] 9. Means of analyzing collected information in real time

[0132] Hardware and software used

[0133] This system uses the following hardware and software:

[0134] Servers: Includes database servers, application servers, storage servers, etc.

[0135] Cloud Storage: Common cloud storage services (e.g., Amazon S3, Google Drive).

[0136] Data warehouse: A data warehouse service such as Amazon Redshift or Google BigQuery.

[0137] Natural language processing engines: IBM Watson NLP, Google Cloud Natural Language API, etc.

[0138] Analysis algorithms: Python's pandas library, numpy library, etc.

[0139] Specific examples

[0140] For example, the specific process for generating a monthly report based on sales data for the past year in a company's sales department is shown below.

[0141] 1. A user logs in to the system and requests, "Create a monthly report based on sales data from the past year."

[0142] 2. The server parses this request using a natural language processing engine (e.g., Google Cloud Natural Language API), collects sales data from a spreadsheet in cloud storage (e.g., Google Drive) for the past year, and also retrieves necessary information from a data warehouse (e.g., Amazon Redshift).

[0143] 3. The server converts the collected data into CSV format and uses Python's pandas library to compile sales for each month.

[0144] 4. The server generates a monthly report in Google Sheets based on the analysis results, and also clearly indicates the source of the data.

[0145] 5. The user reviews the generated report, makes any necessary corrections, and then downloads the final report for use.

[0146] Prompt Sentence Examples

[0147] "Generate monthly reports based on the past year's worth of sales data."

[0148] In this way, the present invention efficiently automates a series of operations from data collection to output generation, significantly reducing the workload on the user.

[0149] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0150] Step 1:

[0151] During initial setup, users enter connection information for information sources into the system, including credentials for cloud storage and data warehouses.

[0152] The server connects to each information source based on the entered connection information and obtains an API key and authentication token. For example, it uses the Google Drive API to perform OAuth authentication and obtains an access token.

[0153] The server stores information about sources that have been successfully connected to in an internal database. The input is the connection information provided by the user, and the output is the connection information stored in the database.

[0154] Step 2:

[0155] A user uses a conversational interface to make a data collection request in natural language, for example, "collect sales data for 2023."

[0156] The server receives this request and analyzes it using a natural language processing engine. Specifically, it uses the Google Cloud Natural Language API to tokenize the request and extract important elements (time period, data items).

[0157] The input is the user's natural language request and the output is the parsed collection conditions.

[0158] Step 3:

[0159] The server identifies specific data collection conditions based on the analysis results and automatically collects the necessary data from each information source.

[0160] For example, the server uses the Google Drive API to download a spreadsheet from cloud storage and then runs a SQL query against Amazon Redshift to retrieve the data.

[0161] The input is the specified data collection conditions and the output is the collected raw data.

[0162] Step 4:

[0163] The server converts the collected data into a unified format (e.g., CSV, JSON), specifically using the Python pandas library to unify the data format.

[0164] The input is the raw data collected and the output is the data converted into a unified format.

[0165] Step 5:

[0166] The server then applies an analysis algorithm to the data converted into a unified format to extract the necessary information.

[0167] For example, use Python's pandas library to analyze sales data and calculate the total sales for each month.

[0168] The input is data converted into a unified format, and the output is the analyzed result data.

[0169] Step 6:

[0170] The server automatically generates output based on the analysis results, which can be in the form of a spreadsheet, PDF report, presentation material, or other format, depending on the user's request.

[0171] Specifically, it uses the Google Sheets API to create a spreadsheet and also generates a PDF file if the user requests it.

[0172] The input is the analysis result data, and the output is the generated output.

[0173] Step 7:

[0174] The server will attribute data to the generated output by annotating each section of the output with the source from which the data was obtained.

[0175] Inputs are generated outputs, and outputs are sourced outputs.

[0176] Step 8:

[0177] The server temporarily stores the generated output and notifies the user, for example by sending a link to the output via email or a notification system.

[0178] The input is the output produced, and the output is the notification to the user.

[0179] Step 9:

[0180] After receiving the notification, the user logs into the system to view the generated output, specifically by accessing the web interface and viewing the report.

[0181] The input is the notified output link and the output is the user feedback.

[0182] Step 10:

[0183] The user makes necessary corrections to the output content and submits the corrections to the system, for example, by inputting additional comments or correction instructions into the report content.

[0184] The input is the user's correction instruction, and the output is the correction request.

[0185] Step 11:

[0186] The server reflects the modifications received from the user and regenerates the final output, specifically by reanalyzing the data based on the modifications and generating new output.

[0187] The input is the user's modification request and the output is the regenerated final output.

[0188] Step 12:

[0189] The server provides the final output to the user and shares it via a download link, email, or other means.

[0190] The input is the reproduced final output, and the output is the presentation of the final output to the user.

[0191] (Application example 1)

[0192] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0193] Production management and quality control in factories involve a wide range of tasks, such as collecting and analyzing data obtained from numerous sensors and devices, and generating reports. This process is largely manual, which is inefficient and has a high risk of error. Furthermore, it is difficult to monitor data and detect anomalies in real time, making it difficult to respond flexibly to situations that require a rapid response.

[0194] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0195] In this invention, the server includes: means for collecting data from multiple data sources; means for converting the collected data into a unified format; means for analyzing the data converted into the unified format; means for automatically generating output based on the analyzed data; means for indicating the source of the data in the output; means for providing the generated output to a user and accepting user corrections; means for regenerating the final output reflecting the corrections; means for collecting data from various devices and sensors in a factory environment and automatically generating a production management report; means for collecting data in real time and displaying the analysis results on a dashboard; and means for issuing an alert when an abnormality occurs. This automates the process from data collection to analysis and output generation at the production site, enabling efficient management. Real-time data monitoring and anomaly detection enable rapid problem resolution.

[0196] "Data Source" means a device or system that provides data from an external or internal system or device.

[0197] "Uniform format" refers to a standardized format for converting different types of data into a consistent format.

[0198] "Analysis" refers to the act of processing collected and transformed data to extract meaningful information or results.

[0199] "Output" refers to the output results provided in the form of reports, graphs, etc. generated based on the results of the analyzed data.

[0200] "Source" refers to information indicating the source from which the collected data was obtained.

[0201] A "production management report" refers to a report that compiles information on production efficiency and quality control at the production site.

[0202] "Real-time data" refers to data that is acquired and analyzed immediately, without any delay.

[0203] A "dashboard" is an interface that visually displays data summaries and details in real time.

[0204] An "alert" is a warning message or signal that notifies you of an abnormality or condition that requires attention.

[0205] 1. System Overview

[0206] To implement this invention, it is necessary to build a system that collects data from multiple data sources, converts it into a unified format, analyzes it, and generates output. This system collects data from various devices and sensors in a factory environment and automatically generates production management reports. It also has the ability to collect data in real time, display analysis results on a dashboard, and issue alerts if an abnormality occurs.

[0207] 2. Hardware and Software Used

[0208] Hardware

[0209] Production line sensors (e.g. temperature sensors, humidity sensors)

[0210] Factory equipment (e.g., production machinery, quality inspection equipment)

[0211] Robot control PC

[0212] software

[0213] Database management system (e.g. MySQL)

[0214] Natural language processing engine (e.g. Google Cloud Natural Language API)

[0215] Data analysis libraries (e.g., Pandas, Numpy)

[0216] Dashboard display framework (e.g. Grafana)

[0217] 3. System Operation

[0218] Data collection

[0219] The server stores connection information for each sensor and device and collects data in real time. For example, it periodically obtains temperature data from a temperature sensor and operation data from production machines.

[0220] Data Conversion

[0221] The server converts the collected data into a unified format (e.g., CSV or JSON), allowing data from different formats to be parsed in a consistent manner.

[0222] Data analysis

[0223] The converted data is analyzed using data analysis libraries (Pandas, Numpy), and daily, weekly, and monthly production efficiency and quality control data are extracted as analysis results.

[0224] Output Generation

[0225] The server automatically generates output based on the analysis results, in formats including spreadsheets, PDF reports, dashboard displays, etc. The generated output also clearly indicates the source of the data.

[0226] Real-time monitoring and alert notifications

[0227] The server displays the data in real time on a dashboard, allowing users to intuitively monitor the status of the production site. If an abnormality occurs, an alert is sent to the user, enabling them to take prompt action.

[0228] 4. Specific Examples

[0229] For example, consider a factory production management system that generates an environmental management report based on data from a temperature sensor over the past month. A user uses an interactive interface to send a request such as, "Create an environmental management report based on this month's temperature data." The server analyzes this request, collects the necessary data, and converts it into a unified format. It then applies a data analysis algorithm to generate a monthly environmental management report and provides it to the user.

[0230] Prompt Sentence Examples

[0231] "Collect temperature sensor data for the past month and generate monthly environmental management reports."

[0232] Data collection range: Past month

[0233] Report items: Average temperature, maximum temperature, minimum temperature, temperature fluctuation by date

[0234] In this way, the invention significantly improves efficiency and accuracy on the production floor through automated data collection, analysis, and output generation. Real-time data monitoring and alerting also enable rapid problem resolution.

[0235] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0236] Step 1:

[0237] Data source registration

[0238] The server receives connection information for each device and sensor in the factory provided by the user.

[0239] Specific operation: The user enters connection information such as the IP address and API key of each sensor and device in the factory.

[0240] Input: Connection information entered by the user

[0241] Output: A configuration file or database containing connection information

[0242] The server connects to each data source based on the received connection information, obtains API keys and authentication tokens, and stores them in an internal database.

[0243] Step 2:

[0244] Data collection

[0245] The server collects data in real time from each registered data source.

[0246] Specific operation: The server sends a data collection request to each sensor (temperature sensor, humidity sensor, etc.) and device (production machine, etc.).

[0247] Input: Connection information for each data source

[0248] Output: Collected real-time data (e.g. temperature data, operation data)

[0249] The server periodically collects data and temporarily stores the collected data.

[0250] Step 3:

[0251] Data Conversion

[0252] Convert the collected data into a unified format (e.g., CSV or JSON).

[0253] Specific operation: The data collected by the server is converted into a unified format using a format conversion library.

[0254] Input: Collected real-time data

[0255] Output: Data converted into a unified format

[0256] This allows different forms of data to be parsed consistently.

[0257] Step 4:

[0258] Data analysis

[0259] The server analyzes the data converted into a unified format using data analysis libraries (Pandas, Numpy).

[0260] Specific operation: The server analyzes the data in the unified format using a data analysis algorithm.

[0261] Input: Data converted into a unified format

[0262] Output: Analysis results (e.g. production efficiency, quality control statistics)

[0263] As a result of the analysis, daily, weekly, and monthly production efficiency and quality control data are extracted.

[0264] Step 5:

[0265] Output Generation

[0266] The server automatically generates output based on the analysis results.

[0267] Specific operation: The server creates output such as spreadsheets, PDF reports, and dashboard displays based on the analysis results.

[0268] Input: Analysis results

[0269] Output: The generated output (e.g. spreadsheet, PDF report)

[0270] The output also clearly indicates the source of the data.

[0271] Step 6:

[0272] Providing and modifying generated outputs

[0273] The generated output is provided to the user and corrections are accepted from the user.

[0274] Specific behavior: The server sends the output to the user, who reviews and modifies it.

[0275] Input: Generated output

[0276] Output: Correction requests from the user

[0277] When the user submits a modification request, the server recreates the final output to reflect the modifications.

[0278] Step 7:

[0279] Real-time monitoring and alert notifications

[0280] The server displays the data in real time on a dashboard and sends an alert if an abnormality occurs.

[0281] What it does: The server uses a dashboard framework (e.g., Grafana) to display real-time data and sends alerts to the user if an anomaly is detected.

[0282] Input: Real-time data, analysis results

[0283] Output: Dashboard display, alert notification

[0284] This allows users to intuitively monitor the situation at the production site and quickly resolve problems.

[0285] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0286] This invention relates to a system that automates data collection, analysis, and output generation, as well as dynamically adjusts the process by recognizing user emotions. By combining an emotion engine, this system provides optimal responses and outputs according to the user's emotional state, improving the user experience.

[0287] Explanation of program processing

[0288] The program of this system performs processing in the following procedure.

[0289] 1. Data Source Registration

[0290] During initial setup, the user enters connection information for cloud storage and data warehouses into the system.

[0291] The server uses the entered information to connect to each data source and obtain an API key and authentication token. If successful, the server stores the data source information in its internal database.

[0292] 2. Data Collection Requests

[0293] A user uses a conversational interface to input a data collection request in natural language (e.g., "Collect monthly sales data for the year 2023").

[0294] The server receives the request and analyzes it using a natural language processing engine. Based on the analysis results, specific data collection conditions (e.g., time period, data items) are extracted.

[0295] 3. Emotion Recognition by Emotion Engine

[0296] The device recognizes the user's emotional state while inputting a request using an emotion engine, which analyzes voice tone, facial expressions, text content, etc. to determine the user's emotion.

[0297] The server receives the emotion data sent from the emotion engine and dynamically adjusts the request content and response method.

[0298] 4. Data collection and transformation

[0299] The server begins collecting data for each data source based on the specified data collection conditions.

[0300] For example, download a spreadsheet from cloud storage and submit an SQL query to the data warehouse to retrieve the required data.

[0301] The server converts the collected data into a unified format (e.g., CSV, JSON), which also includes data cleansing and normalization steps.

[0302] 5. Data Analysis

[0303] The server inputs the data converted into a unified format into an analysis algorithm, extracting and aggregating data that meets the specified conditions.

[0304] 6. Generating Output

[0305] The server automatically generates output in the specified output format (e.g., spreadsheet, PDF, presentation) based on the analysis results, including data source information.

[0306] The server optimizes the format and content of the output depending on the user's emotional state: for example, if the user is feeling stressed, it provides a concise summary of the information.

[0307] 7. Providing and Modifying Output

[0308] The server temporarily stores the automatically generated output and sends a notification to the user.

[0309] The user receives the notification sent and accesses the system to view the generated output.

[0310] The user checks the output content, makes corrections as necessary, and sends the corrections to the system.

[0311] The server receives the user's modifications and recreates the final output to reflect those modifications.

[0312] The server then provides the final regenerated output to the user and shares it via a download link, email, or other means.

[0313] Specific examples

[0314] For example, the following describes an example in which a company's sales department generates a monthly report based on sales data for the past year.

[0315] 1. A user logs in to the system and requests, "Create a monthly report based on sales data from the past year."

[0316] 2. The server parses this request and collects the past year's sales data from a spreadsheet in cloud storage and also retrieves the necessary information from the data warehouse.

[0317] 3. The device uses an emotion engine to recognize the emotional state of the user when making a request, for example, detecting that the user is stressed through voice analysis.

[0318] 4. The server converts the collected data into a unified format and applies an analysis algorithm to compile sales figures for each month.

[0319] 5. The server automatically generates a spreadsheet in a concise and easy-to-understand format, taking into account the user's feelings. The generated report also includes information about the source of the data.

[0320] 6. The user reviews the generated report, makes any necessary corrections, and then downloads the final report for use.

[0321] In this way, the system of the present invention, which is combined with an emotion engine, efficiently automates a series of tasks from data collection to output generation, and provides flexible responses according to the user's emotional state.

[0322] The processing flow will be explained below.

[0323] Step 1:

[0324] The user accesses the system's initial setup screen and enters connection information for data sources such as cloud storage and data warehouses.

[0325] Step 2:

[0326] The server connects to each data source using the entered connection information, obtains an API key and authentication token, and if the connection is successful, stores the data source information in an internal database.

[0327] Step 3:

[0328] A user uses a conversational interface to input a data collection request in natural language (e.g., "Collect monthly sales data for the year 2023").

[0329] Step 4:

[0330] The server sends the user's request to a natural language processing engine, which analyzes the request. Based on the analysis results, specific data collection conditions (e.g., period, data items) are extracted.

[0331] Step 5:

[0332] The device uses a camera and microphone to recognize the user's emotional state through an emotion engine while the user is typing a request, for example, using facial expression recognition technology and voice tone analysis.

[0333] Step 6:

[0334] The server receives the emotion data sent from the emotion engine and dynamically adjusts the request content and response method. If the user is feeling stressed, the server simplifies the request and processes it.

[0335] Step 7:

[0336] The server collects data from each data source based on the specified data collection criteria, for example, downloading a spreadsheet from cloud storage and submitting SQL queries to a data warehouse to retrieve data.

[0337] Step 8:

[0338] The server converts the collected data into a unified format (e.g., CSV, JSON), which also includes data cleansing and normalization steps.

[0339] Step 9:

[0340] The server applies an analysis algorithm to the data converted into a unified format, extracting and aggregating data that meets specified conditions.

[0341] Step 10:

[0342] The server automatically generates output in a specified format (e.g., spreadsheet, PDF, presentation) based on the analysis results, and the format and content of the output are optimized based on the user's emotional state.

[0343] Step 11:

[0344] The server temporarily stores the automatically generated output and sends a notification to the user.

[0345] Step 12:

[0346] The user receives the notification sent and accesses the system to view the generated output.

[0347] Step 13:

[0348] The user checks the output content, makes corrections as necessary, and sends the corrections to the system.

[0349] Step 14:

[0350] The server incorporates the modifications received from the user and recreates the final output based on those modifications.

[0351] Step 15:

[0352] The server then provides the final regenerated output to the user and shares it via a download link, email, or other means.

[0353] Example 2

[0354] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0355] Conventional data collection and analysis systems perform uniform processing without considering the user's emotional state, resulting in a poor user experience. Furthermore, because there is no system that dynamically adjusts the process based on the user's emotions, it is difficult to quickly and appropriately provide the information that stressed users want.

[0356] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0357] In this invention, the server includes: [means for collecting data from multiple data sources;] [means for converting the collected data into a unified format;] [means for analyzing the data converted into the unified format;] [means for automatically generating output based on the analyzed data;] [means for indicating the source of the data in the output;] [means for providing the generated output to a user and accepting corrections from the user;] [means for regenerating a final output that reflects the corrections;] [means for recognizing the user's emotional state and dynamically adjusting the process; and [means for analyzing the user's emotions using an emotion engine.] This makes it possible to provide appropriate and prompt output that takes the user's emotional state into consideration.

[0358] "Data Source" means an external or internal system or database that provides data.

[0359] A "uniform format" refers to a data format for converting data of different formats or structures into a consistent format.

[0360] "Analysis" refers to the process of processing collected data for a specific purpose and extracting useful information.

[0361] "Output" refers to results such as reports or reports generated based on analyzed data.

[0362] "User" refers to a person or organization that operates this system and uses the collected data and analysis results.

[0363] An "emotion engine" refers to an algorithm or system that analyzes a user's emotional state and dynamically adjusts processes based on that information.

[0364] "Prompt sentence" refers to a natural language request sentence that a user inputs to a system through an interactive interface.

[0365] This invention relates to a system that automates data collection, analysis, and output generation, as well as dynamically adjusts the process by recognizing user emotions. By combining an emotion engine, this system provides optimal responses and outputs according to the user's emotional state, improving the user experience.

[0366] This system is implemented through the following specific operations by the server, the terminal, and the user.

[0367] When a user first sets up the system, they enter connection information for their cloud storage or data warehouse. For example, they can use Google Drive as their cloud storage or Amazon Redshift as their data warehouse. The server then connects to each using the provided API key or authentication token, and stores the data source information in an internal database. This internal database can be a database management system such as MySQL.

[0368] When a user makes a data collection request, they use a conversational interface (e.g., a chatbot). For example, they input a prompt such as, "Collect monthly sales data for 2023" or "Create a monthly report based on sales data from the past year." This request is sent to the server and analyzed by a natural language processing engine (e.g., Google Cloud Natural Language API). Based on the analysis results, specific data collection conditions (period, data items) are extracted.

[0369] Furthermore, the device uses an emotion engine (e.g., Microsoft Azure's Emotion API) to recognize the user's emotional state while they are inputting a request. For example, it uses a microphone to collect audio and detects whether the user is feeling stressed through voice analysis. This emotional data is sent to the server and used to adjust the process. If the user is feeling stressed, the server will change its response to be more concise and quick.

[0370] Data collection is performed by the server based on specified conditions, such as downloading a specific spreadsheet from Google Drive and sending SQL queries to Amazon Redshift to retrieve the required data. The collected data is then converted into a unified format (CSV or JSON) using a Python library called Pandas.

[0371] The server then inputs the data converted into this unified format into an analysis algorithm to extract and aggregate the data. The analysis algorithm uses a machine learning library such as Scikit-learn. Based on the analysis results, the server automatically generates output in the specified output format (spreadsheet, PDF, presentation, etc.). The Python ReportLab library can be used to generate the output.

[0372] The generated output is temporarily stored by the server and notified to the user. After receiving the notification, the user accesses the system to check the output and make any necessary corrections. The corrections are sent to the system and reflected in the final output. The optimized output is then provided to the user and shared via a download link, email, or other means.

[0373] In this way, the system of the present invention, which is combined with an emotion engine, efficiently automates a series of tasks from data collection to output generation, and provides flexible responses according to the user's emotional state.

[0374] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0375] Step 1: User registers data source

[0376] During initial setup, users enter connection information for cloud storage and data warehouses into the system, including the cloud storage service name, API key, and authentication token.

[0377] The server receives these inputs, connects to each data source, and obtains authentication information. If the connection is successful using the obtained API key or authentication token, the server stores the data source information in an internal database, such as a MySQL database.

[0378] Step 2: Request data collection

[0379] A user uses a conversational interface to input a data collection request, such as the prompt "Collect monthly sales data for the year 2023."

[0380] The server receives this request and analyzes it using a natural language processing engine (for example, Google Cloud Natural Language API). As a result of the analysis, specific data collection conditions, such as "2023," "monthly," and "sales data," are extracted. This clarifies the specific conditions for data collection.

[0381] Step 3: Emotion recognition by the emotion engine

[0382] The device uses an emotion engine (e.g., Microsoft Azure's Emotion API) to recognize the user's emotional state while they are inputting requests, which can include voice and facial expression data.

[0383] The server receives the emotion data sent from the device and detects whether the user is feeling stressed. Based on the detection results, the server dynamically adjusts the request content and response method. For example, if the user is feeling stressed, the server changes the response to a simpler and easier-to-understand format.

[0384] Step 4: Data collection and transformation

[0385] The server collects data from data sources based on specified data collection criteria, with inputs including, for example, spreadsheets from cloud storage or SQL queries to a data warehouse.

[0386] The server converts the collected data into a unified format (e.g., CSV or JSON) using libraries such as Pandas to cleanse and normalize the data. The output is a dataset converted into a unified format.

[0387] Step 5: Data analysis

[0388] The server receives the unified formatted data and feeds it into an analysis algorithm, which includes as input the unified formatted data from the previous step.

[0389] The server uses machine learning libraries such as Scikit-learn to extract and aggregate data that meets the specified criteria. The output is a dataset and aggregated information as the analysis results.

[0390] Step 6: Generate output

[0391] The server automatically generates output based on the analysis results, and includes the analysis result dataset as input.

[0392] The server generates output in the specified format (spreadsheet, PDF, presentation, etc.) using libraries such as ReportLab, while also providing clear information about the origin of the data. The output is automatically generated.

[0393] Step 7: Provide and modify outputs

[0394] The server temporarily stores the generated output and sends notifications to the user, for example by email or push notification.

[0395] The user receives a notification, accesses the system to view the generated output, makes corrections as needed, and submits the corrections to the system.

[0396] The server receives the user's modifications and recreates the final output, which is then provided to the user via a download link, email, or other means.

[0397] (Application example 2)

[0398] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0399] Currently, customer service in brick-and-mortar stores is highly dependent on the proficiency and communication skills of store staff, resulting in inconsistent quality. It is also difficult to instantly grasp a customer's emotional state, especially when they are worried or unsure, and provide the optimal response accordingly. Furthermore, in situations where quick and accurate information provision is required, flexible responses based on emotions are essential, but current systems are unable to adequately address this. This creates a challenge, making it difficult to improve customer satisfaction.

[0400] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0401] In this invention, the server includes: [means for collecting data from multiple data sources;] [means for converting the collected data into a unified format;] [means for recognizing the user's emotional state; and] [means for dynamically adjusting output and response methods according to the recognized emotional state.] This makes it possible [to improve the quality of customer service in physical stores and enable flexible and appropriate interactions according to the customer's emotions].

[0402] "Multiple data sources" refers to a collection of data storage or systems used to obtain data from different locations and formats.

[0403] "Data collection methods" means the processes and tools used to obtain the required information from each data source.

[0404] "Means of converting collected data into a unified format" refers to techniques and methods for converting data of different formats or structures into a consistent format.

[0405] "Means for analyzing data converted into a consistent format" means means for analyzing data converted into a consistent format and extracting useful information.

[0406] "Means for automated output generation" refers to systems or algorithms that automatically generate documents or reports in the required format based on the analyzed data.

[0407] "Means for attributing data in outputs" means a mechanism for attributing the origin or source of data used in the generated outputs.

[0408] "Means for providing the generated output to the user and accepting corrections from the user" refers to the process of displaying and providing the generated output to the user and accepting corrections or feedback from the user.

[0409] "Means for regenerating the final output reflecting the modifications" means a system or process for recreating the final output based on the modifications received from the user.

[0410] "Means for recognizing the user's emotional state" refers to technologies and algorithms that analyze the user's voice, facial expressions, text, etc. to determine their emotional state.

[0411] "Means for dynamically adjusting output and response methods in response to a recognized emotional state" means means for automatically selecting and adjusting appropriate output and response methods based on the user's emotional state.

[0412] As an embodiment of the present invention, a specific system configuration and program processing will be described below. In this system, store clerks who deal with customers wear smart glasses and analyze customer emotions while providing necessary information in real time.

[0413] Hardware and Software Configuration

[0414] Hardware

[0415] Smart glasses: Smart devices with built-in cameras and microphones that can capture customers' facial expressions and voices in real time.

[0416] Server: A central computer system for data collection, analysis, and output generation.

[0417] Cloud storage: A storage system that stores and manages necessary data.

[0418] software

[0419] Natural Language Processing Engine (NLP Engine): Analyzes natural language requests from users, for example, using the Google Natural Language API.

[0420] Emotion recognition engine (emotion engine): Analyzes the user's emotions from captured audio and video. For example, it uses the Microsoft Azure Emotion API.

[0421] Data collection and conversion module: A module that acquires the necessary data from the data source and converts it into a unified format.

[0422] Data analysis module: A module that analyzes the collected and converted data and generates output.

[0423] Explanation of the system processing flow

[0424] The server has a means of collecting data from multiple data sources and converting it into a unified format. The user, a store clerk, wears the smart glasses and serves customers. The customer's questions and requests are captured by a microphone and analyzed by a natural language processing engine. Next, the emotion engine recognizes the customer's emotional state using the smart glasses' camera and microphone.

[0425] The server analyzes the collected data in real time using an analysis module and automatically generates output based on the results. The generated output is provided to the store clerk with the source of the data clearly indicated. It also has a means of accepting corrections and feedback from the user (store clerk) and regenerating the final output that reflects that. This series of processes dynamically adjusts the content of responses and the way information is presented according to the customer's emotional state.

[0426] Specific examples

[0427] For example, consider a situation where a sales associate is asked, "How much of this item do you have in stock?" The microphone in the smart glasses captures the question, and the NLP engine analyzes it. The server collects inventory data from cloud storage and converts it into a unified format.

[0428] The emotion engine analyzes the captured audio and video data and recognizes whether the customer is speaking with a serious expression. The server analyzes inventory status and related promotion information and generates optimal output based on the customer's interests. For example, it could provide concise and easy-to-understand inventory information, as well as recommended products and discount information.

[0429] Through such interactions, customers receive quick and accurate information, increasing their satisfaction.

[0430] Example prompts to input to the generative AI model

[0431] Enter the following into the "Microsoft Azure Emotion API":

[0432] Customer facial expression images and voice data

[0433] Enter the following into "Google Natural Language API":

[0434] Text of the sales associate's question, "How much of this item do you have in stock?"

[0435] As described above, the system of the present invention can effectively handle customer service in physical stores by recognizing the user's emotional state in real time and dynamically adjusting the response content based on that.

[0436] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0437] Step 1:

[0438] Input: The user (store clerk) puts on the smart glasses and enters the login information.

[0439] Processing: The terminal sends login information to the server. The server performs authentication and obtains connection information for data sources (product inventory data, customer history data, etc.).

[0440] Output: Save the data source connection information to the internal database. Display a login confirmation message on the smart glasses display.

[0441] Step 2:

[0442] Input: A user (store associate) makes a data collection request in natural language (e.g., "How much of this item do you have in stock?").

[0443] Processing: The device captures the request content and sends it to the server, which then analyzes it using a natural language processing engine.

[0444] Output: Generates data collection and analysis conditions extracted from the request (e.g., inventory information for a specific product) and passes them to the data collection and transformation module.

[0445] Step 3:

[0446] Input: The camera and microphone in the smart glasses capture the customer's facial images and voice data.

[0447] Processing: The terminal sends the captured data to the emotion recognition engine, which the server uses to analyze the customer's emotional state.

[0448] Output: Generates the parsed emotion data and passes it to the output adjustment module.

[0449] Step 4:

[0450] Input: Data collection conditions identified by the Data Collection and Transformation Module.

[0451] Processing: The server sends API requests to data sources to collect the required data, and converts the collected data into a unified format (e.g., CSV or JSON).

[0452] Output: The data converted into a unified format is passed to the data analysis module.

[0453] Step 5:

[0454] Input: Data converted into a unified format.

[0455] Processing: The server uses a data analysis module to analyze the collected and converted data in real time, extracting requested information such as specific product categories and inventory levels.

[0456] Output: Pass the analysis results to the output generation module.

[0457] Step 6:

[0458] Input: Analysis results and customer sentiment data.

[0459] Processing: The server uses the output generation module to generate the optimal output depending on the customer's emotional state. For example, if the customer is feeling anxious, it provides detailed information that will reassure the customer.

[0460] Output: The generated output is displayed on the smart glasses display.

[0461] Step 7:

[0462] Input: The user (store clerk) checks the provided output and makes any necessary corrections.

[0463] Processing: The terminal sends the modifications to the server, which analyzes them and reflects them in the output generation module.

[0464] Output: The final output reflecting the modifications is reproduced and displayed on the smart glasses display.

[0465] Step 8:

[0466] Input: The final output.

[0467] Processing: The server temporarily stores the final output and generates a share link to other devices or systems as needed.

[0468] Output: The user is provided with a shared link and access information for the file.

[0469] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0470] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0471] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0472] [Second embodiment]

[0473] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0474] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0475] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0476] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0477] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0478] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0479] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0480] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0481] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0482] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0483] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0484] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0485] The present invention relates to a system for automating data collection, analysis, and output generation in an enterprise environment, which can efficiently collect data from multiple data sources, convert it into a unified format, analyze it, and ultimately automatically generate the output required by the user.

[0486] Explanation of program processing

[0487] The program of this system performs processing in the following procedure.

[0488] 1. Data Source Registration

[0489] During initial setup, the user enters connection information for cloud storage and data warehouses into the system.

[0490] The server uses the entered information to connect to each data source and obtain an API key and authentication token.

[0491] The server stores information about data sources with which a connection has been successfully established in an internal database.

[0492] 2. Data Collection Requests

[0493] A user uses a conversational interface to make a data collection request in natural language (e.g., "Collect sales data for 2023").

[0494] The server receives the request and analyzes the request using a natural language processing engine.

[0495] The server identifies specific data collection conditions (e.g., period, data items) based on the analysis results.

[0496] 3. Data collection and conversion

[0497] The server automatically collects the required data from each data source based on the specified conditions.

[0498] For example, download a spreadsheet from cloud storage and run an SQL query from the data warehouse to get the results.

[0499] The server converts the collected data into a unified format (e.g., CSV, JSON).

[0500] 4. Data Analysis

[0501] The server then applies an analysis algorithm to the data converted into a unified format to extract the necessary information.

[0502] The analysis results are made available to users in the format they request.

[0503] 5. Generating Output

[0504] The server automatically generates output based on the analysis results, which can be in the form of a spreadsheet, PDF report, presentation material, or other format, depending on the user's request.

[0505] Generated outputs will clearly indicate the source from which the data was obtained.

[0506] 6. Providing and Modifying Output

[0507] The server temporarily stores the generated output and notifies the user.

[0508] After receiving the notification, the user logs into the system to view the generated output.

[0509] The user makes any necessary corrections to the output content and sends the corrections to the system.

[0510] The server then incorporates the modifications received from the user and recreates the final output.

[0511] The server provides the final output to the user and shares it via a download link, email, or other means.

[0512] Specific examples

[0513] For example, the following describes an example in which a company's sales department generates a monthly report based on sales data for the past year.

[0514] 1. A user logs in to the system and requests, "Create a monthly report based on sales data from the past year."

[0515] 2. The server parses this request and collects the past year's sales data from a spreadsheet in cloud storage and also retrieves the necessary information from the data warehouse.

[0516] 3. The server converts the collected data into a unified format and applies an analysis algorithm to compile sales figures for each month.

[0517] 4. The server generates a monthly report in spreadsheet format based on the analysis results, and also indicates the source of the data.

[0518] 5. The user reviews the generated report, makes any necessary corrections, and then downloads the final report for use.

[0519] In this way, the present invention efficiently automates a series of operations from data collection to output generation, significantly reducing the workload on the user.

[0520] The processing flow will be explained below.

[0521] Step 1:

[0522] The user accesses the system's initial setup screen and enters connection information for data sources such as cloud storage and data warehouses.

[0523] Step 2:

[0524] The server connects to each data source using the entered connection information to obtain an API key and authentication token, and if successful, stores the data source information in an internal database.

[0525] Step 3:

[0526] A user uses a conversational interface to input a data collection request in natural language (e.g., "Collect monthly sales data for the year 2023").

[0527] Step 4:

[0528] The server sends the user's request to a natural language processing engine, which analyzes the request. Based on the analysis results, specific data collection conditions (e.g., period, data items) are extracted.

[0529] Step 5:

[0530] The server begins collecting data for each data source based on the specified data collection conditions.

[0531] For example, download a spreadsheet from cloud storage and submit SQL queries to a data warehouse to retrieve the required data.

[0532] Step 6:

[0533] The server converts the collected data from each data source into a unified format (e.g., CSV, JSON), which also includes data cleansing and normalization steps.

[0534] Step 7:

[0535] The server inputs the data converted into a unified format into an analysis algorithm, extracting and aggregating data that meets the specified conditions.

[0536] Step 8:

[0537] The server automatically generates output based on the analysis results in the specified output format (e.g., spreadsheet, PDF, presentation), including information on the origin of the collected data.

[0538] Step 9:

[0539] The server temporarily stores the automatically generated output and sends a notification to the user.

[0540] Step 10:

[0541] The user receives the notification sent and accesses the system to view the generated output.

[0542] Step 11:

[0543] The user checks the output content, makes corrections as necessary, and sends the corrections to the system.

[0544] Step 12:

[0545] The server receives the user's modifications and recreates the final output to reflect those modifications.

[0546] Step 13:

[0547] The server then provides the final regenerated output to the user and shares it via a download link, email, or other means.

[0548] Example 1

[0549] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0550] In a corporate environment, collecting data from multiple sources, converting it into a unified format, analyzing it, and automatically generating the final output is a very time-consuming process. It also requires a lot of time and effort to clearly indicate the source of the information and regenerate the output in response to user requests for corrections. Furthermore, the lack of a system that can accept user requests in natural language through a dialogue interface and analyze them in real time impairs user convenience. These issues need to be addressed.

[0551] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0552] In this invention, the server includes: [means for collecting information from multiple information sources]; [means for converting the collected information into a unified format]; and [means for analyzing the information converted into the unified format]. This makes it possible to efficiently automate a series of tasks from collecting information to analyzing it and generating output.

[0553] The "means for collecting information from multiple information sources" is a mechanism for automatically obtaining necessary information from multiple information sources specified by the user.

[0554] "Means for converting collected information into a unified format" refers to a mechanism for converting information collected in different formats into a unified format.

[0555] "Means for analyzing information converted into a unified format" refers to a mechanism for performing the necessary analysis based on information converted into a unified format and extracting meaningful data.

[0556] The "means for automatically generating output based on analyzed information" is a mechanism for automatically generating output in a format requested by a user based on analyzed information.

[0557] "Means for indicating the source of information in the output" refers to a mechanism for indicating from which source the information in the generated output was obtained.

[0558] The "means for providing the generated output to the user and accepting corrections from the user" is a mechanism for providing the generated output to the user and accepting corrections instructed by the user as necessary.

[0559] The "means for regenerating the final output reflecting the corrections" is a mechanism for regenerating the final output based on the corrections provided by the user.

[0560] "Means for providing an interactive interface and receiving information gathering requests in natural language from users" refers to a mechanism for providing an interactive interface that allows users to make information gathering requests in natural language and for receiving such requests.

[0561] "Means for analyzing collected information in real time" refers to a mechanism for instantly analyzing collected information and providing results in real time.

[0562] The present invention provides a system for automating data collection, analysis, and output generation in an enterprise environment. This system efficiently collects information from multiple sources, converts it into a unified format, analyzes it, and finally automatically generates the output required by the user.

[0563] System Overview

[0564] This system consists of a server, terminals, and users. The server performs the following main functions:

[0565] 1. A means of gathering information from multiple sources

[0566] 2. A means of converting the collected information into a unified format

[0567] 3. A means of analyzing information converted into a unified format

[0568] 4. A means to automatically generate output based on the analyzed information

[0569] 5. A means of citing the source of information in your output

[0570] 6. A means of providing the generated output to users and accepting user corrections.

[0571] 7. A means to reproduce the final output reflecting the modifications.

[0572] 8. Means for providing an interactive interface and receiving natural language information gathering requests from users

[0573] 9. Means of analyzing collected information in real time

[0574] Hardware and software used

[0575] This system uses the following hardware and software:

[0576] Servers: Includes database servers, application servers, storage servers, etc.

[0577] Cloud Storage: Common cloud storage services (e.g., Amazon S3, Google Drive).

[0578] Data warehouse: A data warehouse service such as Amazon Redshift or Google BigQuery.

[0579] Natural language processing engines: IBM Watson NLP, Google Cloud Natural Language API, etc.

[0580] Analysis algorithms: Python's pandas library, numpy library, etc.

[0581] Specific examples

[0582] For example, the specific process for generating a monthly report based on sales data for the past year in a company's sales department is shown below.

[0583] 1. A user logs in to the system and requests, "Create a monthly report based on sales data from the past year."

[0584] 2. The server parses this request using a natural language processing engine (e.g., Google Cloud Natural Language API), collects sales data from a spreadsheet in cloud storage (e.g., Google Drive) for the past year, and also retrieves necessary information from a data warehouse (e.g., Amazon Redshift).

[0585] 3. The server converts the collected data into CSV format and uses Python's pandas library to compile sales for each month.

[0586] 4. The server generates a monthly report in Google Sheets based on the analysis results, and also clearly indicates the source of the data.

[0587] 5. The user reviews the generated report, makes any necessary corrections, and then downloads the final report for use.

[0588] Prompt Sentence Examples

[0589] "Generate monthly reports based on the past year's worth of sales data."

[0590] In this way, the present invention efficiently automates a series of operations from data collection to output generation, significantly reducing the workload on the user.

[0591] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0592] Step 1:

[0593] During initial setup, users enter connection information for information sources into the system, including credentials for cloud storage and data warehouses.

[0594] The server connects to each information source based on the entered connection information and obtains an API key and authentication token. For example, it uses the Google Drive API to perform OAuth authentication and obtains an access token.

[0595] The server stores information about sources that have been successfully connected to in an internal database. The input is the connection information provided by the user, and the output is the connection information stored in the database.

[0596] Step 2:

[0597] A user uses a conversational interface to make a data collection request in natural language, for example, "collect sales data for 2023."

[0598] The server receives this request and analyzes it using a natural language processing engine. Specifically, it uses the Google Cloud Natural Language API to tokenize the request and extract important elements (time period, data items).

[0599] The input is the user's natural language request and the output is the parsed collection conditions.

[0600] Step 3:

[0601] The server identifies specific data collection conditions based on the analysis results and automatically collects the necessary data from each information source.

[0602] For example, the server uses the Google Drive API to download a spreadsheet from cloud storage and then runs a SQL query against Amazon Redshift to retrieve the data.

[0603] The input is the specified data collection conditions and the output is the collected raw data.

[0604] Step 4:

[0605] The server converts the collected data into a unified format (e.g., CSV, JSON), specifically using the Python pandas library to unify the data format.

[0606] The input is the raw data collected and the output is the data converted into a unified format.

[0607] Step 5:

[0608] The server then applies an analysis algorithm to the data converted into a unified format to extract the necessary information.

[0609] For example, use Python's pandas library to analyze sales data and calculate the total sales for each month.

[0610] The input is data converted into a unified format, and the output is the analyzed result data.

[0611] Step 6:

[0612] The server automatically generates output based on the analysis results, which can be in the form of a spreadsheet, PDF report, presentation material, or other format, depending on the user's request.

[0613] Specifically, it uses the Google Sheets API to create a spreadsheet and also generates a PDF file if the user requests it.

[0614] The input is the analysis result data, and the output is the generated output.

[0615] Step 7:

[0616] The server will attribute data to the generated output by annotating each section of the output with the source from which the data was obtained.

[0617] Inputs are generated outputs, and outputs are sourced outputs.

[0618] Step 8:

[0619] The server temporarily stores the generated output and notifies the user, for example by sending a link to the output via email or a notification system.

[0620] The input is the output produced, and the output is the notification to the user.

[0621] Step 9:

[0622] After receiving the notification, the user logs into the system to view the generated output, specifically by accessing the web interface and viewing the report.

[0623] The input is the notified output link and the output is the user feedback.

[0624] Step 10:

[0625] The user makes necessary corrections to the output content and submits the corrections to the system, for example, by inputting additional comments or correction instructions into the report content.

[0626] The input is the user's correction instruction, and the output is the correction request.

[0627] Step 11:

[0628] The server reflects the modifications received from the user and regenerates the final output, specifically by reanalyzing the data based on the modifications and generating new output.

[0629] The input is the user's modification request and the output is the regenerated final output.

[0630] Step 12:

[0631] The server provides the final output to the user and shares it via a download link, email, or other means.

[0632] The input is the reproduced final output, and the output is the presentation of the final output to the user.

[0633] (Application example 1)

[0634] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0635] Production management and quality control in factories involve a wide range of tasks, such as collecting and analyzing data obtained from numerous sensors and devices, and generating reports. This process is largely manual, which is inefficient and has a high risk of error. Furthermore, it is difficult to monitor data and detect anomalies in real time, making it difficult to respond flexibly to situations that require a rapid response.

[0636] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0637] In this invention, the server includes: means for collecting data from multiple data sources; means for converting the collected data into a unified format; means for analyzing the data converted into the unified format; means for automatically generating output based on the analyzed data; means for indicating the source of the data in the output; means for providing the generated output to a user and accepting user corrections; means for regenerating the final output reflecting the corrections; means for collecting data from various devices and sensors in a factory environment and automatically generating a production management report; means for collecting data in real time and displaying the analysis results on a dashboard; and means for issuing an alert when an abnormality occurs. This automates the process from data collection to analysis and output generation at the production site, enabling efficient management. Real-time data monitoring and anomaly detection enable rapid problem resolution.

[0638] "Data Source" means a device or system that provides data from an external or internal system or device.

[0639] "Uniform format" refers to a standardized format for converting different types of data into a consistent format.

[0640] "Analysis" refers to the act of processing collected and transformed data to extract meaningful information or results.

[0641] "Output" refers to the output results provided in the form of reports, graphs, etc. generated based on the results of the analyzed data.

[0642] "Source" refers to information indicating the source from which the collected data was obtained.

[0643] A "production management report" refers to a report that compiles information on production efficiency and quality control at the production site.

[0644] "Real-time data" refers to data that is acquired and analyzed immediately, without any delay.

[0645] A "dashboard" is an interface that visually displays data summaries and details in real time.

[0646] An "alert" is a warning message or signal that notifies you of an abnormality or condition that requires attention.

[0647] 1. System Overview

[0648] To implement this invention, it is necessary to build a system that collects data from multiple data sources, converts it into a unified format, analyzes it, and generates output. This system collects data from various devices and sensors in a factory environment and automatically generates production management reports. It also has the ability to collect data in real time, display analysis results on a dashboard, and issue alerts if an abnormality occurs.

[0649] 2. Hardware and Software Used

[0650] Hardware

[0651] Production line sensors (e.g. temperature sensors, humidity sensors)

[0652] Factory equipment (e.g., production machinery, quality inspection equipment)

[0653] Robot control PC

[0654] software

[0655] Database management system (e.g. MySQL)

[0656] Natural language processing engine (e.g. Google Cloud Natural Language API)

[0657] Data analysis libraries (e.g., Pandas, Numpy)

[0658] Dashboard display framework (e.g. Grafana)

[0659] 3. System Operation

[0660] Data collection

[0661] The server stores connection information for each sensor and device and collects data in real time. For example, it periodically obtains temperature data from a temperature sensor and operation data from production machines.

[0662] Data Conversion

[0663] The server converts the collected data into a unified format (e.g., CSV or JSON), allowing data from different formats to be parsed in a consistent manner.

[0664] Data analysis

[0665] The converted data is analyzed using data analysis libraries (Pandas, Numpy), and daily, weekly, and monthly production efficiency and quality control data are extracted as analysis results.

[0666] Output Generation

[0667] The server automatically generates output based on the analysis results, in formats including spreadsheets, PDF reports, dashboard displays, etc. The generated output also clearly indicates the source of the data.

[0668] Real-time monitoring and alert notifications

[0669] The server displays the data in real time on a dashboard, allowing users to intuitively monitor the status of the production site. If an abnormality occurs, an alert is sent to the user, enabling them to take prompt action.

[0670] 4. Specific Examples

[0671] For example, consider a factory production management system that generates an environmental management report based on data from a temperature sensor over the past month. A user uses an interactive interface to send a request such as, "Create an environmental management report based on this month's temperature data." The server analyzes this request, collects the necessary data, and converts it into a unified format. It then applies a data analysis algorithm to generate a monthly environmental management report and provides it to the user.

[0672] Prompt Sentence Examples

[0673] "Collect temperature sensor data for the past month and generate monthly environmental management reports."

[0674] Data collection range: Past month

[0675] Report items: Average temperature, maximum temperature, minimum temperature, temperature fluctuation by date

[0676] In this way, the invention significantly improves efficiency and accuracy on the production floor through automated data collection, analysis, and output generation. Real-time data monitoring and alerting also enable rapid problem resolution.

[0677] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0678] Step 1:

[0679] Data source registration

[0680] The server receives connection information for each device and sensor in the factory provided by the user.

[0681] Specific operation: The user enters connection information such as the IP address and API key of each sensor and device in the factory.

[0682] Input: Connection information entered by the user

[0683] Output: A configuration file or database containing connection information

[0684] The server connects to each data source based on the received connection information, obtains API keys and authentication tokens, and stores them in an internal database.

[0685] Step 2:

[0686] Data collection

[0687] The server collects data in real time from each registered data source.

[0688] Specific operation: The server sends a data collection request to each sensor (temperature sensor, humidity sensor, etc.) and device (production machine, etc.).

[0689] Input: Connection information for each data source

[0690] Output: Collected real-time data (e.g. temperature data, operation data)

[0691] The server periodically collects data and temporarily stores the collected data.

[0692] Step 3:

[0693] Data Conversion

[0694] Convert the collected data into a unified format (e.g., CSV or JSON).

[0695] Specific operation: The data collected by the server is converted into a unified format using a format conversion library.

[0696] Input: Collected real-time data

[0697] Output: Data converted into a unified format

[0698] This allows different forms of data to be parsed consistently.

[0699] Step 4:

[0700] Data analysis

[0701] The server analyzes the data converted into a unified format using data analysis libraries (Pandas, Numpy).

[0702] Specific operation: The server analyzes the data in the unified format using a data analysis algorithm.

[0703] Input: Data converted into a unified format

[0704] Output: Analysis results (e.g. production efficiency, quality control statistics)

[0705] As a result of the analysis, daily, weekly, and monthly production efficiency and quality control data are extracted.

[0706] Step 5:

[0707] Output Generation

[0708] The server automatically generates output based on the analysis results.

[0709] Specific operation: The server creates output such as spreadsheets, PDF reports, and dashboard displays based on the analysis results.

[0710] Input: Analysis results

[0711] Output: The generated output (e.g. spreadsheet, PDF report)

[0712] The output also clearly indicates the source of the data.

[0713] Step 6:

[0714] Providing and modifying generated outputs

[0715] The generated output is provided to the user and corrections are accepted from the user.

[0716] Specific behavior: The server sends the output to the user, who reviews and modifies it.

[0717] Input: Generated output

[0718] Output: Correction requests from the user

[0719] When the user submits a modification request, the server recreates the final output to reflect the modifications.

[0720] Step 7:

[0721] Real-time monitoring and alert notifications

[0722] The server displays the data in real time on a dashboard and sends an alert if an abnormality occurs.

[0723] What it does: The server uses a dashboard framework (e.g., Grafana) to display real-time data and sends alerts to the user if an anomaly is detected.

[0724] Input: Real-time data, analysis results

[0725] Output: Dashboard display, alert notification

[0726] This allows users to intuitively monitor the situation at the production site and quickly resolve problems.

[0727] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0728] This invention relates to a system that automates data collection, analysis, and output generation, as well as dynamically adjusts the process by recognizing user emotions. By combining an emotion engine, this system provides optimal responses and outputs according to the user's emotional state, improving the user experience.

[0729] Explanation of program processing

[0730] The program of this system performs processing in the following procedure.

[0731] 1. Data Source Registration

[0732] During initial setup, the user enters connection information for cloud storage and data warehouses into the system.

[0733] The server uses the entered information to connect to each data source and obtain an API key and authentication token. If successful, the server stores the data source information in its internal database.

[0734] 2. Data Collection Requests

[0735] A user uses a conversational interface to input a data collection request in natural language (e.g., "Collect monthly sales data for the year 2023").

[0736] The server receives the request and analyzes it using a natural language processing engine. Based on the analysis results, specific data collection conditions (e.g., time period, data items) are extracted.

[0737] 3. Emotion Recognition by Emotion Engine

[0738] The device recognizes the user's emotional state while inputting a request using an emotion engine, which analyzes voice tone, facial expressions, text content, etc. to determine the user's emotion.

[0739] The server receives the emotion data sent from the emotion engine and dynamically adjusts the request content and response method.

[0740] 4. Data collection and transformation

[0741] The server begins collecting data for each data source based on the specified data collection conditions.

[0742] For example, download a spreadsheet from cloud storage and submit an SQL query to the data warehouse to retrieve the required data.

[0743] The server converts the collected data into a unified format (e.g., CSV, JSON), which also includes data cleansing and normalization steps.

[0744] 5. Data Analysis

[0745] The server inputs the data converted into a unified format into an analysis algorithm, extracting and aggregating data that meets the specified conditions.

[0746] 6. Generating Output

[0747] The server automatically generates output in the specified output format (e.g., spreadsheet, PDF, presentation) based on the analysis results, including data source information.

[0748] The server optimizes the format and content of the output depending on the user's emotional state: for example, if the user is feeling stressed, it provides a concise summary of the information.

[0749] 7. Providing and Modifying Output

[0750] The server temporarily stores the automatically generated output and sends a notification to the user.

[0751] The user receives the notification sent and accesses the system to view the generated output.

[0752] The user checks the output content, makes corrections as necessary, and sends the corrections to the system.

[0753] The server receives the user's modifications and recreates the final output to reflect those modifications.

[0754] The server then provides the final regenerated output to the user and shares it via a download link, email, or other means.

[0755] Specific examples

[0756] For example, the following describes an example in which a company's sales department generates a monthly report based on sales data for the past year.

[0757] 1. A user logs in to the system and requests, "Create a monthly report based on sales data from the past year."

[0758] 2. The server parses this request and collects the past year's sales data from a spreadsheet in cloud storage and also retrieves the necessary information from the data warehouse.

[0759] 3. The device uses an emotion engine to recognize the emotional state of the user when making a request, for example, detecting that the user is stressed through voice analysis.

[0760] 4. The server converts the collected data into a unified format and applies an analysis algorithm to compile sales figures for each month.

[0761] 5. The server automatically generates a spreadsheet in a concise and easy-to-understand format, taking into account the user's feelings. The generated report also includes information about the source of the data.

[0762] 6. The user reviews the generated report, makes any necessary corrections, and then downloads the final report for use.

[0763] In this way, the system of the present invention, which is combined with an emotion engine, efficiently automates a series of tasks from data collection to output generation, and provides flexible responses according to the user's emotional state.

[0764] The processing flow will be explained below.

[0765] Step 1:

[0766] The user accesses the system's initial setup screen and enters connection information for data sources such as cloud storage and data warehouses.

[0767] Step 2:

[0768] The server connects to each data source using the entered connection information, obtains an API key and authentication token, and if the connection is successful, stores the data source information in an internal database.

[0769] Step 3:

[0770] A user uses a conversational interface to input a data collection request in natural language (e.g., "Collect monthly sales data for the year 2023").

[0771] Step 4:

[0772] The server sends the user's request to a natural language processing engine, which analyzes the request. Based on the analysis results, specific data collection conditions (e.g., period, data items) are extracted.

[0773] Step 5:

[0774] The device uses a camera and microphone to recognize the user's emotional state through an emotion engine while the user is typing a request, for example, using facial expression recognition technology and voice tone analysis.

[0775] Step 6:

[0776] The server receives the emotion data sent from the emotion engine and dynamically adjusts the request content and response method. If the user is feeling stressed, the server simplifies the request and processes it.

[0777] Step 7:

[0778] The server collects data from each data source based on the specified data collection criteria, for example, downloading a spreadsheet from cloud storage and submitting SQL queries to a data warehouse to retrieve data.

[0779] Step 8:

[0780] The server converts the collected data into a unified format (e.g., CSV, JSON), which also includes data cleansing and normalization steps.

[0781] Step 9:

[0782] The server applies an analysis algorithm to the data converted into a unified format, extracting and aggregating data that meets specified conditions.

[0783] Step 10:

[0784] The server automatically generates output in a specified format (e.g., spreadsheet, PDF, presentation) based on the analysis results, and the format and content of the output are optimized based on the user's emotional state.

[0785] Step 11:

[0786] The server temporarily stores the automatically generated output and sends a notification to the user.

[0787] Step 12:

[0788] The user receives the notification sent and accesses the system to view the generated output.

[0789] Step 13:

[0790] The user checks the output content, makes corrections as necessary, and sends the corrections to the system.

[0791] Step 14:

[0792] The server incorporates the modifications received from the user and recreates the final output based on those modifications.

[0793] Step 15:

[0794] The server then provides the final regenerated output to the user and shares it via a download link, email, or other means.

[0795] Example 2

[0796] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0797] Conventional data collection and analysis systems perform uniform processing without considering the user's emotional state, resulting in a poor user experience. Furthermore, because there is no system that dynamically adjusts the process based on the user's emotions, it is difficult to quickly and appropriately provide the information that stressed users want.

[0798] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0799] In this invention, the server includes: [means for collecting data from multiple data sources;] [means for converting the collected data into a unified format;] [means for analyzing the data converted into the unified format;] [means for automatically generating output based on the analyzed data;] [means for indicating the source of the data in the output;] [means for providing the generated output to a user and accepting corrections from the user;] [means for regenerating a final output that reflects the corrections;] [means for recognizing the user's emotional state and dynamically adjusting the process; and [means for analyzing the user's emotions using an emotion engine.] This makes it possible to provide appropriate and prompt output that takes the user's emotional state into consideration.

[0800] "Data Source" means an external or internal system or database that provides data.

[0801] A "uniform format" refers to a data format for converting data of different formats or structures into a consistent format.

[0802] "Analysis" refers to the process of processing collected data for a specific purpose and extracting useful information.

[0803] "Output" refers to results such as reports or reports generated based on analyzed data.

[0804] "User" refers to a person or organization that operates this system and uses the collected data and analysis results.

[0805] An "emotion engine" refers to an algorithm or system that analyzes a user's emotional state and dynamically adjusts processes based on that information.

[0806] "Prompt sentence" refers to a natural language request sentence that a user inputs to a system through an interactive interface.

[0807] This invention relates to a system that automates data collection, analysis, and output generation, as well as dynamically adjusts the process by recognizing user emotions. By combining an emotion engine, this system provides optimal responses and outputs according to the user's emotional state, improving the user experience.

[0808] This system is implemented through the following specific operations by the server, the terminal, and the user.

[0809] When a user first sets up the system, they enter connection information for their cloud storage or data warehouse. For example, they can use Google Drive as their cloud storage or Amazon Redshift as their data warehouse. The server then connects to each using the provided API key or authentication token, and stores the data source information in an internal database. This internal database can be a database management system such as MySQL.

[0810] When a user makes a data collection request, they use a conversational interface (e.g., a chatbot). For example, they input a prompt such as, "Collect monthly sales data for 2023" or "Create a monthly report based on sales data from the past year." This request is sent to the server and analyzed by a natural language processing engine (e.g., Google Cloud Natural Language API). Based on the analysis results, specific data collection conditions (period, data items) are extracted.

[0811] Furthermore, the device uses an emotion engine (e.g., Microsoft Azure's Emotion API) to recognize the user's emotional state while they are inputting a request. For example, it uses a microphone to collect audio and detects whether the user is feeling stressed through voice analysis. This emotional data is sent to the server and used to adjust the process. If the user is feeling stressed, the server will change its response to be more concise and quick.

[0812] Data collection is performed by the server based on specified conditions, such as downloading a specific spreadsheet from Google Drive and sending SQL queries to Amazon Redshift to retrieve the required data. The collected data is then converted into a unified format (CSV or JSON) using a Python library called Pandas.

[0813] The server then inputs the data converted into this unified format into an analysis algorithm to extract and aggregate the data. The analysis algorithm uses a machine learning library such as Scikit-learn. Based on the analysis results, the server automatically generates output in the specified output format (spreadsheet, PDF, presentation, etc.). The Python ReportLab library can be used to generate the output.

[0814] The generated output is temporarily stored by the server and notified to the user. After receiving the notification, the user accesses the system to check the output and make any necessary corrections. The corrections are sent to the system and reflected in the final output. The optimized output is then provided to the user and shared via a download link, email, or other means.

[0815] In this way, the system of the present invention, which is combined with an emotion engine, efficiently automates a series of tasks from data collection to output generation, and provides flexible responses according to the user's emotional state.

[0816] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0817] Step 1: User registers data source

[0818] During initial setup, users enter connection information for cloud storage and data warehouses into the system, including the cloud storage service name, API key, and authentication token.

[0819] The server receives these inputs, connects to each data source, and obtains authentication information. If the connection is successful using the obtained API key or authentication token, the server stores the data source information in an internal database, such as a MySQL database.

[0820] Step 2: Request data collection

[0821] A user uses a conversational interface to input a data collection request, such as the prompt "Collect monthly sales data for the year 2023."

[0822] The server receives this request and analyzes it using a natural language processing engine (for example, Google Cloud Natural Language API). As a result of the analysis, specific data collection conditions, such as "2023," "monthly," and "sales data," are extracted. This clarifies the specific conditions for data collection.

[0823] Step 3: Emotion recognition by the emotion engine

[0824] The device uses an emotion engine (e.g., Microsoft Azure's Emotion API) to recognize the user's emotional state while they are inputting requests, which can include voice and facial expression data.

[0825] The server receives the emotion data sent from the device and detects whether the user is feeling stressed. Based on the detection results, the server dynamically adjusts the request content and response method. For example, if the user is feeling stressed, the server changes the response to a simpler and easier-to-understand format.

[0826] Step 4: Data collection and transformation

[0827] The server collects data from data sources based on specified data collection criteria, with inputs including, for example, spreadsheets from cloud storage or SQL queries to a data warehouse.

[0828] The server converts the collected data into a unified format (e.g., CSV or JSON) using libraries such as Pandas to cleanse and normalize the data. The output is a dataset converted into a unified format.

[0829] Step 5: Data analysis

[0830] The server receives the unified formatted data and feeds it into an analysis algorithm, which includes as input the unified formatted data from the previous step.

[0831] The server uses machine learning libraries such as Scikit-learn to extract and aggregate data that meets the specified criteria. The output is a dataset and aggregated information as the analysis results.

[0832] Step 6: Generate output

[0833] The server automatically generates output based on the analysis results, and includes the analysis result dataset as input.

[0834] The server generates output in the specified format (spreadsheet, PDF, presentation, etc.) using libraries such as ReportLab, while also providing clear information about the origin of the data. The output is automatically generated.

[0835] Step 7: Provide and modify outputs

[0836] The server temporarily stores the generated output and sends notifications to the user, for example by email or push notification.

[0837] The user receives a notification, accesses the system to view the generated output, makes corrections as needed, and submits the corrections to the system.

[0838] The server receives the user's modifications and recreates the final output, which is then provided to the user via a download link, email, or other means.

[0839] (Application example 2)

[0840] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0841] Currently, customer service in brick-and-mortar stores is highly dependent on the proficiency and communication skills of store staff, resulting in inconsistent quality. It is also difficult to instantly grasp a customer's emotional state, especially when they are worried or unsure, and provide the optimal response accordingly. Furthermore, in situations where quick and accurate information provision is required, flexible responses based on emotions are essential, but current systems are unable to adequately address this. This creates a challenge, making it difficult to improve customer satisfaction.

[0842] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0843] In this invention, the server includes: [means for collecting data from multiple data sources;] [means for converting the collected data into a unified format;] [means for recognizing the user's emotional state; and] [means for dynamically adjusting output and response methods according to the recognized emotional state.] This makes it possible [to improve the quality of customer service in physical stores and enable flexible and appropriate interactions according to the customer's emotions].

[0844] "Multiple data sources" refers to a collection of data storage or systems used to obtain data from different locations and formats.

[0845] "Data collection methods" means the processes and tools used to obtain the required information from each data source.

[0846] "Means of converting collected data into a unified format" refers to techniques and methods for converting data of different formats or structures into a consistent format.

[0847] "Means for analyzing data converted into a consistent format" means means for analyzing data converted into a consistent format and extracting useful information.

[0848] "Means for automated output generation" refers to systems or algorithms that automatically generate documents or reports in the required format based on the analyzed data.

[0849] "Means for attributing data in outputs" means a mechanism for attributing the origin or source of data used in the generated outputs.

[0850] "Means for providing the generated output to the user and accepting corrections from the user" refers to the process of displaying and providing the generated output to the user and accepting corrections or feedback from the user.

[0851] "Means for regenerating the final output reflecting the modifications" means a system or process for recreating the final output based on the modifications received from the user.

[0852] "Means for recognizing the user's emotional state" refers to technologies and algorithms that analyze the user's voice, facial expressions, text, etc. to determine their emotional state.

[0853] "Means for dynamically adjusting output and response methods in response to a recognized emotional state" means means for automatically selecting and adjusting appropriate output and response methods based on the user's emotional state.

[0854] As an embodiment of the present invention, a specific system configuration and program processing will be described below. In this system, store clerks who deal with customers wear smart glasses and analyze customer emotions while providing necessary information in real time.

[0855] Hardware and Software Configuration

[0856] Hardware

[0857] Smart glasses: Smart devices with built-in cameras and microphones that can capture customers' facial expressions and voices in real time.

[0858] Server: A central computer system for data collection, analysis, and output generation.

[0859] Cloud storage: A storage system that stores and manages necessary data.

[0860] software

[0861] Natural Language Processing Engine (NLP Engine): Analyzes natural language requests from users, for example, using the Google Natural Language API.

[0862] Emotion recognition engine (emotion engine): Analyzes the user's emotions from captured audio and video. For example, it uses the Microsoft Azure Emotion API.

[0863] Data collection and conversion module: A module that acquires the necessary data from the data source and converts it into a unified format.

[0864] Data analysis module: A module that analyzes the collected and converted data and generates output.

[0865] Explanation of the system processing flow

[0866] The server has a means of collecting data from multiple data sources and converting it into a unified format. The user, a store clerk, wears the smart glasses and serves customers. The customer's questions and requests are captured by a microphone and analyzed by a natural language processing engine. Next, the emotion engine recognizes the customer's emotional state using the smart glasses' camera and microphone.

[0867] The server analyzes the collected data in real time using an analysis module and automatically generates output based on the results. The generated output is provided to the store clerk with the source of the data clearly indicated. It also has a means of accepting corrections and feedback from the user (store clerk) and regenerating the final output that reflects that. This series of processes dynamically adjusts the content of responses and the way information is presented according to the customer's emotional state.

[0868] Specific examples

[0869] For example, consider a situation where a sales associate is asked, "How much of this item do you have in stock?" The microphone in the smart glasses captures the question, and the NLP engine analyzes it. The server collects inventory data from cloud storage and converts it into a unified format.

[0870] The emotion engine analyzes the captured audio and video data and recognizes whether the customer is speaking with a serious expression. The server analyzes inventory status and related promotion information and generates optimal output based on the customer's interests. For example, it could provide concise and easy-to-understand inventory information, as well as recommended products and discount information.

[0871] Through such interactions, customers receive quick and accurate information, increasing their satisfaction.

[0872] Example prompts to input to the generative AI model

[0873] Enter the following into the "Microsoft Azure Emotion API":

[0874] Customer facial expression images and voice data

[0875] Enter the following into "Google Natural Language API":

[0876] Text of the sales associate's question, "How much of this item do you have in stock?"

[0877] As described above, the system of the present invention can effectively handle customer service in physical stores by recognizing the user's emotional state in real time and dynamically adjusting the response content based on that.

[0878] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0879] Step 1:

[0880] Input: The user (store clerk) puts on the smart glasses and enters the login information.

[0881] Processing: The terminal sends login information to the server. The server performs authentication and obtains connection information for data sources (product inventory data, customer history data, etc.).

[0882] Output: Save the data source connection information to the internal database. Display a login confirmation message on the smart glasses display.

[0883] Step 2:

[0884] Input: A user (store associate) makes a data collection request in natural language (e.g., "How much of this item do you have in stock?").

[0885] Processing: The device captures the request content and sends it to the server, which then analyzes it using a natural language processing engine.

[0886] Output: Generates data collection and analysis conditions extracted from the request (e.g., inventory information for a specific product) and passes them to the data collection and transformation module.

[0887] Step 3:

[0888] Input: The camera and microphone in the smart glasses capture the customer's facial images and voice data.

[0889] Processing: The terminal sends the captured data to the emotion recognition engine, which the server uses to analyze the customer's emotional state.

[0890] Output: Generates the parsed emotion data and passes it to the output adjustment module.

[0891] Step 4:

[0892] Input: Data collection conditions identified by the Data Collection and Transformation Module.

[0893] Processing: The server sends API requests to data sources to collect the required data, and converts the collected data into a unified format (e.g., CSV or JSON).

[0894] Output: The data converted into a unified format is passed to the data analysis module.

[0895] Step 5:

[0896] Input: Data converted into a unified format.

[0897] Processing: The server uses a data analysis module to analyze the collected and converted data in real time, extracting requested information such as specific product categories and inventory levels.

[0898] Output: Pass the analysis results to the output generation module.

[0899] Step 6:

[0900] Input: Analysis results and customer sentiment data.

[0901] Processing: The server uses the output generation module to generate the optimal output depending on the customer's emotional state. For example, if the customer is feeling anxious, it provides detailed information that will reassure the customer.

[0902] Output: The generated output is displayed on the smart glasses display.

[0903] Step 7:

[0904] Input: The user (store clerk) checks the provided output and makes any necessary corrections.

[0905] Processing: The terminal sends the modifications to the server, which analyzes them and reflects them in the output generation module.

[0906] Output: The final output reflecting the modifications is reproduced and displayed on the smart glasses display.

[0907] Step 8:

[0908] Input: The final output.

[0909] Processing: The server temporarily stores the final output and generates a share link to other devices or systems as needed.

[0910] Output: The user is provided with a shared link and access information for the file.

[0911] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0912] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0913] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0914] [Third embodiment]

[0915] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0916] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0917] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0918] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0919] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0920] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0921] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0922] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0923] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0924] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0925] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0926] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0927] The present invention relates to a system for automating data collection, analysis, and output generation in an enterprise environment, which can efficiently collect data from multiple data sources, convert it into a unified format, analyze it, and ultimately automatically generate the output required by the user.

[0928] Explanation of program processing

[0929] The program of this system performs processing in the following procedure.

[0930] 1. Data Source Registration

[0931] During initial setup, the user enters connection information for cloud storage and data warehouses into the system.

[0932] The server uses the entered information to connect to each data source and obtain an API key and authentication token.

[0933] The server stores information about data sources with which a connection has been successfully established in an internal database.

[0934] 2. Data Collection Requests

[0935] A user uses a conversational interface to make a data collection request in natural language (e.g., "Collect sales data for 2023").

[0936] The server receives the request and analyzes the request using a natural language processing engine.

[0937] The server identifies specific data collection conditions (e.g., period, data items) based on the analysis results.

[0938] 3. Data collection and conversion

[0939] The server automatically collects the required data from each data source based on the specified conditions.

[0940] For example, download a spreadsheet from cloud storage and run an SQL query from the data warehouse to get the results.

[0941] The server converts the collected data into a unified format (e.g., CSV, JSON).

[0942] 4. Data Analysis

[0943] The server then applies an analysis algorithm to the data converted into a unified format to extract the necessary information.

[0944] The analysis results are made available to users in the format they request.

[0945] 5. Generating Output

[0946] The server automatically generates output based on the analysis results, which can be in the form of a spreadsheet, PDF report, presentation material, or other format, depending on the user's request.

[0947] Generated outputs will clearly indicate the source from which the data was obtained.

[0948] 6. Providing and Modifying Output

[0949] The server temporarily stores the generated output and notifies the user.

[0950] After receiving the notification, the user logs into the system to view the generated output.

[0951] The user makes any necessary corrections to the output content and sends the corrections to the system.

[0952] The server then incorporates the modifications received from the user and recreates the final output.

[0953] The server provides the final output to the user and shares it via a download link, email, or other means.

[0954] Specific examples

[0955] For example, the following describes an example in which a company's sales department generates a monthly report based on sales data for the past year.

[0956] 1. A user logs in to the system and requests, "Create a monthly report based on sales data from the past year."

[0957] 2. The server parses this request and collects the past year's sales data from a spreadsheet in cloud storage and also retrieves the necessary information from the data warehouse.

[0958] 3. The server converts the collected data into a unified format and applies an analysis algorithm to compile sales figures for each month.

[0959] 4. The server generates a monthly report in spreadsheet format based on the analysis results, and also indicates the source of the data.

[0960] 5. The user reviews the generated report, makes any necessary corrections, and then downloads the final report for use.

[0961] In this way, the present invention efficiently automates a series of operations from data collection to output generation, significantly reducing the workload on the user.

[0962] The processing flow will be explained below.

[0963] Step 1:

[0964] The user accesses the system's initial setup screen and enters connection information for data sources such as cloud storage and data warehouses.

[0965] Step 2:

[0966] The server connects to each data source using the entered connection information to obtain an API key and authentication token, and if successful, stores the data source information in an internal database.

[0967] Step 3:

[0968] A user uses a conversational interface to input a data collection request in natural language (e.g., "Collect monthly sales data for the year 2023").

[0969] Step 4:

[0970] The server sends the user's request to a natural language processing engine, which analyzes the request. Based on the analysis results, specific data collection conditions (e.g., period, data items) are extracted.

[0971] Step 5:

[0972] The server begins collecting data for each data source based on the specified data collection conditions.

[0973] For example, download a spreadsheet from cloud storage and submit SQL queries to a data warehouse to retrieve the required data.

[0974] Step 6:

[0975] The server converts the collected data from each data source into a unified format (e.g., CSV, JSON), which also includes data cleansing and normalization steps.

[0976] Step 7:

[0977] The server inputs the data converted into a unified format into an analysis algorithm, extracting and aggregating data that meets the specified conditions.

[0978] Step 8:

[0979] The server automatically generates output based on the analysis results in the specified output format (e.g., spreadsheet, PDF, presentation), including information on the origin of the collected data.

[0980] Step 9:

[0981] The server temporarily stores the automatically generated output and sends a notification to the user.

[0982] Step 10:

[0983] The user receives the notification sent and accesses the system to view the generated output.

[0984] Step 11:

[0985] The user checks the output content, makes corrections as necessary, and sends the corrections to the system.

[0986] Step 12:

[0987] The server receives the user's modifications and recreates the final output to reflect those modifications.

[0988] Step 13:

[0989] The server then provides the final regenerated output to the user and shares it via a download link, email, or other means.

[0990] Example 1

[0991] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0992] In a corporate environment, collecting data from multiple sources, converting it into a unified format, analyzing it, and automatically generating the final output is a very time-consuming process. It also requires a lot of time and effort to clearly indicate the source of the information and regenerate the output in response to user requests for corrections. Furthermore, the lack of a system that can accept user requests in natural language through a dialogue interface and analyze them in real time impairs user convenience. These issues need to be addressed.

[0993] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0994] In this invention, the server includes: [means for collecting information from multiple information sources]; [means for converting the collected information into a unified format]; and [means for analyzing the information converted into the unified format]. This makes it possible to efficiently automate a series of tasks from collecting information to analyzing it and generating output.

[0995] The "means for collecting information from multiple information sources" is a mechanism for automatically obtaining necessary information from multiple information sources specified by the user.

[0996] "Means for converting collected information into a unified format" refers to a mechanism for converting information collected in different formats into a unified format.

[0997] "Means for analyzing information converted into a unified format" refers to a mechanism for performing the necessary analysis based on information converted into a unified format and extracting meaningful data.

[0998] The "means for automatically generating output based on analyzed information" is a mechanism for automatically generating output in a format requested by a user based on analyzed information.

[0999] "Means for indicating the source of information in the output" refers to a mechanism for indicating from which source the information in the generated output was obtained.

[1000] The "means for providing the generated output to the user and accepting corrections from the user" is a mechanism for providing the generated output to the user and accepting corrections instructed by the user as necessary.

[1001] The "means for regenerating the final output reflecting the corrections" is a mechanism for regenerating the final output based on the corrections provided by the user.

[1002] "Means for providing an interactive interface and receiving information gathering requests in natural language from users" refers to a mechanism for providing an interactive interface that allows users to make information gathering requests in natural language and for receiving such requests.

[1003] "Means for analyzing collected information in real time" refers to a mechanism for instantly analyzing collected information and providing results in real time.

[1004] The present invention provides a system for automating data collection, analysis, and output generation in an enterprise environment. This system efficiently collects information from multiple sources, converts it into a unified format, analyzes it, and finally automatically generates the output required by the user.

[1005] System Overview

[1006] This system consists of a server, terminals, and users. The server performs the following main functions:

[1007] 1. A means of gathering information from multiple sources

[1008] 2. A means of converting the collected information into a unified format

[1009] 3. A means of analyzing information converted into a unified format

[1010] 4. A means to automatically generate output based on the analyzed information

[1011] 5. A means of citing the source of information in your output

[1012] 6. A means of providing the generated output to users and accepting user corrections.

[1013] 7. A means to reproduce the final output reflecting the modifications.

[1014] 8. Means for providing an interactive interface and receiving natural language information gathering requests from users

[1015] 9. Means of analyzing collected information in real time

[1016] Hardware and software used

[1017] This system uses the following hardware and software:

[1018] Servers: Includes database servers, application servers, storage servers, etc.

[1019] Cloud Storage: Common cloud storage services (e.g., Amazon S3, Google Drive).

[1020] Data warehouse: A data warehouse service such as Amazon Redshift or Google BigQuery.

[1021] Natural language processing engines: IBM Watson NLP, Google Cloud Natural Language API, etc.

[1022] Analysis algorithms: Python's pandas library, numpy library, etc.

[1023] Specific examples

[1024] For example, the specific process for generating a monthly report based on sales data for the past year in a company's sales department is shown below.

[1025] 1. A user logs in to the system and requests, "Create a monthly report based on sales data from the past year."

[1026] 2. The server parses this request using a natural language processing engine (e.g., Google Cloud Natural Language API), collects sales data from a spreadsheet in cloud storage (e.g., Google Drive) for the past year, and also retrieves necessary information from a data warehouse (e.g., Amazon Redshift).

[1027] 3. The server converts the collected data into CSV format and uses Python's pandas library to compile sales for each month.

[1028] 4. The server generates a monthly report in Google Sheets based on the analysis results, and also clearly indicates the source of the data.

[1029] 5. The user reviews the generated report, makes any necessary corrections, and then downloads the final report for use.

[1030] Prompt Sentence Examples

[1031] "Generate monthly reports based on the past year's worth of sales data."

[1032] In this way, the present invention efficiently automates a series of operations from data collection to output generation, significantly reducing the workload on the user.

[1033] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1034] Step 1:

[1035] During initial setup, users enter connection information for information sources into the system, including credentials for cloud storage and data warehouses.

[1036] The server connects to each information source based on the entered connection information and obtains an API key and authentication token. For example, it uses the Google Drive API to perform OAuth authentication and obtains an access token.

[1037] The server stores information about sources that have been successfully connected to in an internal database. The input is the connection information provided by the user, and the output is the connection information stored in the database.

[1038] Step 2:

[1039] A user uses a conversational interface to make a data collection request in natural language, for example, "collect sales data for 2023."

[1040] The server receives this request and analyzes it using a natural language processing engine. Specifically, it uses the Google Cloud Natural Language API to tokenize the request and extract important elements (time period, data items).

[1041] The input is the user's natural language request and the output is the parsed collection conditions.

[1042] Step 3:

[1043] The server identifies specific data collection conditions based on the analysis results and automatically collects the necessary data from each information source.

[1044] For example, the server uses the Google Drive API to download a spreadsheet from cloud storage and then runs a SQL query against Amazon Redshift to retrieve the data.

[1045] The input is the specified data collection conditions and the output is the collected raw data.

[1046] Step 4:

[1047] The server converts the collected data into a unified format (e.g., CSV, JSON), specifically using the Python pandas library to unify the data format.

[1048] The input is the raw data collected and the output is the data converted into a unified format.

[1049] Step 5:

[1050] The server then applies an analysis algorithm to the data converted into a unified format to extract the necessary information.

[1051] For example, use Python's pandas library to analyze sales data and calculate the total sales for each month.

[1052] The input is data converted into a unified format, and the output is the analyzed result data.

[1053] Step 6:

[1054] The server automatically generates output based on the analysis results, which can be in the form of a spreadsheet, PDF report, presentation material, or other format, depending on the user's request.

[1055] Specifically, it uses the Google Sheets API to create a spreadsheet and also generates a PDF file if the user requests it.

[1056] The input is the analysis result data, and the output is the generated output.

[1057] Step 7:

[1058] The server will attribute data to the generated output by annotating each section of the output with the source from which the data was obtained.

[1059] Inputs are generated outputs, and outputs are sourced outputs.

[1060] Step 8:

[1061] The server temporarily stores the generated output and notifies the user, for example by sending a link to the output via email or a notification system.

[1062] The input is the output produced, and the output is the notification to the user.

[1063] Step 9:

[1064] After receiving the notification, the user logs into the system to view the generated output, specifically by accessing the web interface and viewing the report.

[1065] The input is the notified output link and the output is the user feedback.

[1066] Step 10:

[1067] The user makes necessary corrections to the output content and submits the corrections to the system, for example, by inputting additional comments or correction instructions into the report content.

[1068] The input is the user's correction instruction, and the output is the correction request.

[1069] Step 11:

[1070] The server reflects the modifications received from the user and regenerates the final output, specifically by reanalyzing the data based on the modifications and generating new output.

[1071] The input is the user's modification request and the output is the regenerated final output.

[1072] Step 12:

[1073] The server provides the final output to the user and shares it via a download link, email, or other means.

[1074] The input is the reproduced final output, and the output is the presentation of the final output to the user.

[1075] (Application example 1)

[1076] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1077] Production management and quality control in factories involve a wide range of tasks, such as collecting and analyzing data obtained from numerous sensors and devices, and generating reports. This process is largely manual, which is inefficient and has a high risk of error. Furthermore, it is difficult to monitor data and detect anomalies in real time, making it difficult to respond flexibly to situations that require a rapid response.

[1078] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1079] In this invention, the server includes: means for collecting data from multiple data sources; means for converting the collected data into a unified format; means for analyzing the data converted into the unified format; means for automatically generating output based on the analyzed data; means for indicating the source of the data in the output; means for providing the generated output to a user and accepting user corrections; means for regenerating the final output reflecting the corrections; means for collecting data from various devices and sensors in a factory environment and automatically generating a production management report; means for collecting data in real time and displaying the analysis results on a dashboard; and means for issuing an alert when an abnormality occurs. This automates the process from data collection to analysis and output generation at the production site, enabling efficient management. Real-time data monitoring and anomaly detection enable rapid problem resolution.

[1080] "Data Source" means a device or system that provides data from an external or internal system or device.

[1081] "Uniform format" refers to a standardized format for converting different types of data into a consistent format.

[1082] "Analysis" refers to the act of processing collected and transformed data to extract meaningful information or results.

[1083] "Output" refers to the output results provided in the form of reports, graphs, etc. generated based on the results of the analyzed data.

[1084] "Source" refers to information indicating the source from which the collected data was obtained.

[1085] A "production management report" refers to a report that compiles information on production efficiency and quality control at the production site.

[1086] "Real-time data" refers to data that is acquired and analyzed immediately, without any delay.

[1087] A "dashboard" is an interface that visually displays data summaries and details in real time.

[1088] An "alert" is a warning message or signal that notifies you of an abnormality or condition that requires attention.

[1089] 1. System Overview

[1090] To implement this invention, it is necessary to build a system that collects data from multiple data sources, converts it into a unified format, analyzes it, and generates output. This system collects data from various devices and sensors in a factory environment and automatically generates production management reports. It also has the ability to collect data in real time, display analysis results on a dashboard, and issue alerts if an abnormality occurs.

[1091] 2. Hardware and Software Used

[1092] Hardware

[1093] Production line sensors (e.g. temperature sensors, humidity sensors)

[1094] Factory equipment (e.g., production machinery, quality inspection equipment)

[1095] Robot control PC

[1096] software

[1097] Database management system (e.g. MySQL)

[1098] Natural language processing engine (e.g. Google Cloud Natural Language API)

[1099] Data analysis libraries (e.g., Pandas, Numpy)

[1100] Dashboard display framework (e.g. Grafana)

[1101] 3. System Operation

[1102] Data collection

[1103] The server stores connection information for each sensor and device and collects data in real time. For example, it periodically obtains temperature data from a temperature sensor and operation data from production machines.

[1104] Data Conversion

[1105] The server converts the collected data into a unified format (e.g., CSV or JSON), allowing data from different formats to be parsed in a consistent manner.

[1106] Data analysis

[1107] The converted data is analyzed using data analysis libraries (Pandas, Numpy), and daily, weekly, and monthly production efficiency and quality control data are extracted as analysis results.

[1108] Output Generation

[1109] The server automatically generates output based on the analysis results, in formats including spreadsheets, PDF reports, dashboard displays, etc. The generated output also clearly indicates the source of the data.

[1110] Real-time monitoring and alert notifications

[1111] The server displays the data in real time on a dashboard, allowing users to intuitively monitor the status of the production site. If an abnormality occurs, an alert is sent to the user, enabling them to take prompt action.

[1112] 4. Specific Examples

[1113] For example, consider a factory production management system that generates an environmental management report based on data from a temperature sensor over the past month. A user uses an interactive interface to send a request such as, "Create an environmental management report based on this month's temperature data." The server analyzes this request, collects the necessary data, and converts it into a unified format. It then applies a data analysis algorithm to generate a monthly environmental management report and provides it to the user.

[1114] Prompt Sentence Examples

[1115] "Collect temperature sensor data for the past month and generate monthly environmental management reports."

[1116] Data collection range: Past month

[1117] Report items: Average temperature, maximum temperature, minimum temperature, temperature fluctuation by date

[1118] In this way, the invention significantly improves efficiency and accuracy on the production floor through automated data collection, analysis, and output generation. Real-time data monitoring and alerting also enable rapid problem resolution.

[1119] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1120] Step 1:

[1121] Data source registration

[1122] The server receives connection information for each device and sensor in the factory provided by the user.

[1123] Specific operation: The user enters connection information such as the IP address and API key of each sensor and device in the factory.

[1124] Input: Connection information entered by the user

[1125] Output: A configuration file or database containing connection information

[1126] The server connects to each data source based on the received connection information, obtains API keys and authentication tokens, and stores them in an internal database.

[1127] Step 2:

[1128] Data collection

[1129] The server collects data in real time from each registered data source.

[1130] Specific operation: The server sends a data collection request to each sensor (temperature sensor, humidity sensor, etc.) and device (production machine, etc.).

[1131] Input: Connection information for each data source

[1132] Output: Collected real-time data (e.g. temperature data, operation data)

[1133] The server periodically collects data and temporarily stores the collected data.

[1134] Step 3:

[1135] Data Conversion

[1136] Convert the collected data into a unified format (e.g., CSV or JSON).

[1137] Specific operation: The data collected by the server is converted into a unified format using a format conversion library.

[1138] Input: Collected real-time data

[1139] Output: Data converted into a unified format

[1140] This allows different forms of data to be parsed consistently.

[1141] Step 4:

[1142] Data analysis

[1143] The server analyzes the data converted into a unified format using data analysis libraries (Pandas, Numpy).

[1144] Specific operation: The server analyzes the data in the unified format using a data analysis algorithm.

[1145] Input: Data converted into a unified format

[1146] Output: Analysis results (e.g. production efficiency, quality control statistics)

[1147] As a result of the analysis, daily, weekly, and monthly production efficiency and quality control data are extracted.

[1148] Step 5:

[1149] Output Generation

[1150] The server automatically generates output based on the analysis results.

[1151] Specific operation: The server creates output such as spreadsheets, PDF reports, and dashboard displays based on the analysis results.

[1152] Input: Analysis results

[1153] Output: The generated output (e.g. spreadsheet, PDF report)

[1154] The output also clearly indicates the source of the data.

[1155] Step 6:

[1156] Providing and modifying generated outputs

[1157] The generated output is provided to the user and corrections are accepted from the user.

[1158] Specific behavior: The server sends the output to the user, who reviews and modifies it.

[1159] Input: Generated output

[1160] Output: Correction requests from the user

[1161] When the user submits a modification request, the server recreates the final output to reflect the modifications.

[1162] Step 7:

[1163] Real-time monitoring and alert notifications

[1164] The server displays the data in real time on a dashboard and sends an alert if an abnormality occurs.

[1165] What it does: The server uses a dashboard framework (e.g., Grafana) to display real-time data and sends alerts to the user if an anomaly is detected.

[1166] Input: Real-time data, analysis results

[1167] Output: Dashboard display, alert notification

[1168] This allows users to intuitively monitor the situation at the production site and quickly resolve problems.

[1169] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1170] This invention relates to a system that automates data collection, analysis, and output generation, as well as dynamically adjusts the process by recognizing user emotions. By combining an emotion engine, this system provides optimal responses and outputs according to the user's emotional state, improving the user experience.

[1171] Explanation of program processing

[1172] The program of this system performs processing in the following procedure.

[1173] 1. Data Source Registration

[1174] During initial setup, the user enters connection information for cloud storage and data warehouses into the system.

[1175] The server uses the entered information to connect to each data source and obtain an API key and authentication token. If successful, the server stores the data source information in its internal database.

[1176] 2. Data Collection Requests

[1177] A user uses a conversational interface to input a data collection request in natural language (e.g., "Collect monthly sales data for the year 2023").

[1178] The server receives the request and analyzes it using a natural language processing engine. Based on the analysis results, specific data collection conditions (e.g., time period, data items) are extracted.

[1179] 3. Emotion Recognition by Emotion Engine

[1180] The device recognizes the user's emotional state while inputting a request using an emotion engine, which analyzes voice tone, facial expressions, text content, etc. to determine the user's emotion.

[1181] The server receives the emotion data sent from the emotion engine and dynamically adjusts the request content and response method.

[1182] 4. Data collection and transformation

[1183] The server begins collecting data for each data source based on the specified data collection conditions.

[1184] For example, download a spreadsheet from cloud storage and submit an SQL query to the data warehouse to retrieve the required data.

[1185] The server converts the collected data into a unified format (e.g., CSV, JSON), which also includes data cleansing and normalization steps.

[1186] 5. Data Analysis

[1187] The server inputs the data converted into a unified format into an analysis algorithm, extracting and aggregating data that meets the specified conditions.

[1188] 6. Generating Output

[1189] The server automatically generates output in the specified output format (e.g., spreadsheet, PDF, presentation) based on the analysis results, including data source information.

[1190] The server optimizes the format and content of the output depending on the user's emotional state: for example, if the user is feeling stressed, it provides a concise summary of the information.

[1191] 7. Providing and Modifying Output

[1192] The server temporarily stores the automatically generated output and sends a notification to the user.

[1193] The user receives the notification sent and accesses the system to view the generated output.

[1194] The user checks the output content, makes corrections as necessary, and sends the corrections to the system.

[1195] The server receives the user's modifications and recreates the final output to reflect those modifications.

[1196] The server then provides the final regenerated output to the user and shares it via a download link, email, or other means.

[1197] Specific examples

[1198] For example, the following describes an example in which a company's sales department generates a monthly report based on sales data for the past year.

[1199] 1. A user logs in to the system and requests, "Create a monthly report based on sales data from the past year."

[1200] 2. The server parses this request and collects the past year's sales data from a spreadsheet in cloud storage and also retrieves the necessary information from the data warehouse.

[1201] 3. The device uses an emotion engine to recognize the emotional state of the user when making a request, for example, detecting that the user is stressed through voice analysis.

[1202] 4. The server converts the collected data into a unified format and applies an analysis algorithm to compile sales figures for each month.

[1203] 5. The server automatically generates a spreadsheet in a concise and easy-to-understand format, taking into account the user's feelings. The generated report also includes information about the source of the data.

[1204] 6. The user reviews the generated report, makes any necessary corrections, and then downloads the final report for use.

[1205] In this way, the system of the present invention, which is combined with an emotion engine, efficiently automates a series of tasks from data collection to output generation, and provides flexible responses according to the user's emotional state.

[1206] The processing flow will be explained below.

[1207] Step 1:

[1208] The user accesses the system's initial setup screen and enters connection information for data sources such as cloud storage and data warehouses.

[1209] Step 2:

[1210] The server connects to each data source using the entered connection information, obtains an API key and authentication token, and if the connection is successful, stores the data source information in an internal database.

[1211] Step 3:

[1212] A user uses a conversational interface to input a data collection request in natural language (e.g., "Collect monthly sales data for the year 2023").

[1213] Step 4:

[1214] The server sends the user's request to a natural language processing engine, which analyzes the request. Based on the analysis results, specific data collection conditions (e.g., period, data items) are extracted.

[1215] Step 5:

[1216] The device uses a camera and microphone to recognize the user's emotional state through an emotion engine while the user is typing a request, for example, using facial expression recognition technology and voice tone analysis.

[1217] Step 6:

[1218] The server receives the emotion data sent from the emotion engine and dynamically adjusts the request content and response method. If the user is feeling stressed, the server simplifies the request and processes it.

[1219] Step 7:

[1220] The server collects data from each data source based on the specified data collection criteria, for example, downloading a spreadsheet from cloud storage and submitting SQL queries to a data warehouse to retrieve data.

[1221] Step 8:

[1222] The server converts the collected data into a unified format (e.g., CSV, JSON), which also includes data cleansing and normalization steps.

[1223] Step 9:

[1224] The server applies an analysis algorithm to the data converted into a unified format, extracting and aggregating data that meets specified conditions.

[1225] Step 10:

[1226] The server automatically generates output in a specified format (e.g., spreadsheet, PDF, presentation) based on the analysis results, and the format and content of the output are optimized based on the user's emotional state.

[1227] Step 11:

[1228] The server temporarily stores the automatically generated output and sends a notification to the user.

[1229] Step 12:

[1230] The user receives the notification sent and accesses the system to view the generated output.

[1231] Step 13:

[1232] The user checks the output content, makes corrections as necessary, and sends the corrections to the system.

[1233] Step 14:

[1234] The server incorporates the modifications received from the user and recreates the final output based on those modifications.

[1235] Step 15:

[1236] The server then provides the final regenerated output to the user and shares it via a download link, email, or other means.

[1237] Example 2

[1238] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1239] Conventional data collection and analysis systems perform uniform processing without considering the user's emotional state, resulting in a poor user experience. Furthermore, because there is no system that dynamically adjusts the process based on the user's emotions, it is difficult to quickly and appropriately provide the information that stressed users want.

[1240] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1241] In this invention, the server includes: [means for collecting data from multiple data sources;] [means for converting the collected data into a unified format;] [means for analyzing the data converted into the unified format;] [means for automatically generating output based on the analyzed data;] [means for indicating the source of the data in the output;] [means for providing the generated output to a user and accepting corrections from the user;] [means for regenerating a final output that reflects the corrections;] [means for recognizing the user's emotional state and dynamically adjusting the process; and [means for analyzing the user's emotions using an emotion engine.] This makes it possible to provide appropriate and prompt output that takes the user's emotional state into consideration.

[1242] "Data Source" means an external or internal system or database that provides data.

[1243] A "uniform format" refers to a data format for converting data of different formats or structures into a consistent format.

[1244] "Analysis" refers to the process of processing collected data for a specific purpose and extracting useful information.

[1245] "Output" refers to results such as reports or reports generated based on analyzed data.

[1246] "User" refers to a person or organization that operates this system and uses the collected data and analysis results.

[1247] An "emotion engine" refers to an algorithm or system that analyzes a user's emotional state and dynamically adjusts processes based on that information.

[1248] "Prompt sentence" refers to a natural language request sentence that a user inputs to a system through an interactive interface.

[1249] This invention relates to a system that automates data collection, analysis, and output generation, as well as dynamically adjusts the process by recognizing user emotions. By combining an emotion engine, this system provides optimal responses and outputs according to the user's emotional state, improving the user experience.

[1250] This system is implemented through the following specific operations by the server, the terminal, and the user.

[1251] When a user first sets up the system, they enter connection information for their cloud storage or data warehouse. For example, they can use Google Drive as their cloud storage or Amazon Redshift as their data warehouse. The server then connects to each using the provided API key or authentication token, and stores the data source information in an internal database. This internal database can be a database management system such as MySQL.

[1252] When a user makes a data collection request, they use a conversational interface (e.g., a chatbot). For example, they input a prompt such as, "Collect monthly sales data for 2023" or "Create a monthly report based on sales data from the past year." This request is sent to the server and analyzed by a natural language processing engine (e.g., Google Cloud Natural Language API). Based on the analysis results, specific data collection conditions (period, data items) are extracted.

[1253] Furthermore, the device uses an emotion engine (e.g., Microsoft Azure's Emotion API) to recognize the user's emotional state while they are inputting a request. For example, it uses a microphone to collect audio and detects whether the user is feeling stressed through voice analysis. This emotional data is sent to the server and used to adjust the process. If the user is feeling stressed, the server will change its response to be more concise and quick.

[1254] Data collection is performed by the server based on specified conditions, such as downloading a specific spreadsheet from Google Drive and sending SQL queries to Amazon Redshift to retrieve the required data. The collected data is then converted into a unified format (CSV or JSON) using a Python library called Pandas.

[1255] The server then inputs the data converted into this unified format into an analysis algorithm to extract and aggregate the data. The analysis algorithm uses a machine learning library such as Scikit-learn. Based on the analysis results, the server automatically generates output in the specified output format (spreadsheet, PDF, presentation, etc.). The Python ReportLab library can be used to generate the output.

[1256] The generated output is temporarily stored by the server and notified to the user. After receiving the notification, the user accesses the system to check the output and make any necessary corrections. The corrections are sent to the system and reflected in the final output. The optimized output is then provided to the user and shared via a download link, email, or other means.

[1257] In this way, the system of the present invention, which is combined with an emotion engine, efficiently automates a series of tasks from data collection to output generation, and provides flexible responses according to the user's emotional state.

[1258] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1259] Step 1: User registers data source

[1260] During initial setup, users enter connection information for cloud storage and data warehouses into the system, including the cloud storage service name, API key, and authentication token.

[1261] The server receives these inputs, connects to each data source, and obtains authentication information. If the connection is successful using the obtained API key or authentication token, the server stores the data source information in an internal database, such as a MySQL database.

[1262] Step 2: Request data collection

[1263] A user uses a conversational interface to input a data collection request, such as the prompt "Collect monthly sales data for the year 2023."

[1264] The server receives this request and analyzes it using a natural language processing engine (for example, Google Cloud Natural Language API). As a result of the analysis, specific data collection conditions, such as "2023," "monthly," and "sales data," are extracted. This clarifies the specific conditions for data collection.

[1265] Step 3: Emotion recognition by the emotion engine

[1266] The device uses an emotion engine (e.g., Microsoft Azure's Emotion API) to recognize the user's emotional state while they are inputting requests, which can include voice and facial expression data.

[1267] The server receives the emotion data sent from the device and detects whether the user is feeling stressed. Based on the detection results, the server dynamically adjusts the request content and response method. For example, if the user is feeling stressed, the server changes the response to a simpler and easier-to-understand format.

[1268] Step 4: Data collection and transformation

[1269] The server collects data from data sources based on specified data collection criteria, with inputs including, for example, spreadsheets from cloud storage or SQL queries to a data warehouse.

[1270] The server converts the collected data into a unified format (e.g., CSV or JSON) using libraries such as Pandas to cleanse and normalize the data. The output is a dataset converted into a unified format.

[1271] Step 5: Data analysis

[1272] The server receives the unified formatted data and feeds it into an analysis algorithm, which includes as input the unified formatted data from the previous step.

[1273] The server uses machine learning libraries such as Scikit-learn to extract and aggregate data that meets the specified criteria. The output is a dataset and aggregated information as the analysis results.

[1274] Step 6: Generate output

[1275] The server automatically generates output based on the analysis results, and includes the analysis result dataset as input.

[1276] The server generates output in the specified format (spreadsheet, PDF, presentation, etc.) using libraries such as ReportLab, while also providing clear information about the origin of the data. The output is automatically generated.

[1277] Step 7: Provide and modify outputs

[1278] The server temporarily stores the generated output and sends notifications to the user, for example by email or push notification.

[1279] The user receives a notification, accesses the system to view the generated output, makes corrections as needed, and submits the corrections to the system.

[1280] The server receives the user's modifications and recreates the final output, which is then provided to the user via a download link, email, or other means.

[1281] (Application example 2)

[1282] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1283] Currently, customer service in brick-and-mortar stores is highly dependent on the proficiency and communication skills of store staff, resulting in inconsistent quality. It is also difficult to instantly grasp a customer's emotional state, especially when they are worried or unsure, and provide the optimal response accordingly. Furthermore, in situations where quick and accurate information provision is required, flexible responses based on emotions are essential, but current systems are unable to adequately address this. This creates a challenge, making it difficult to improve customer satisfaction.

[1284] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1285] In this invention, the server includes: [means for collecting data from multiple data sources;] [means for converting the collected data into a unified format;] [means for recognizing the user's emotional state; and] [means for dynamically adjusting output and response methods according to the recognized emotional state.] This makes it possible [to improve the quality of customer service in physical stores and enable flexible and appropriate interactions according to the customer's emotions].

[1286] "Multiple data sources" refers to a collection of data storage or systems used to obtain data from different locations and formats.

[1287] "Data collection methods" means the processes and tools used to obtain the required information from each data source.

[1288] "Means of converting collected data into a unified format" refers to techniques and methods for converting data of different formats or structures into a consistent format.

[1289] "Means for analyzing data converted into a consistent format" means means for analyzing data converted into a consistent format and extracting useful information.

[1290] "Means for automated output generation" refers to systems or algorithms that automatically generate documents or reports in the required format based on the analyzed data.

[1291] "Means for attributing data in outputs" means a mechanism for attributing the origin or source of data used in the generated outputs.

[1292] "Means for providing the generated output to the user and accepting corrections from the user" refers to the process of displaying and providing the generated output to the user and accepting corrections or feedback from the user.

[1293] "Means for regenerating the final output reflecting the modifications" means a system or process for recreating the final output based on the modifications received from the user.

[1294] "Means for recognizing the user's emotional state" refers to technologies and algorithms that analyze the user's voice, facial expressions, text, etc. to determine their emotional state.

[1295] "Means for dynamically adjusting output and response methods in response to a recognized emotional state" means means for automatically selecting and adjusting appropriate output and response methods based on the user's emotional state.

[1296] As an embodiment of the present invention, a specific system configuration and program processing will be described below. In this system, store clerks who deal with customers wear smart glasses and analyze customer emotions while providing necessary information in real time.

[1297] Hardware and Software Configuration

[1298] Hardware

[1299] Smart glasses: Smart devices with built-in cameras and microphones that can capture customers' facial expressions and voices in real time.

[1300] Server: A central computer system for data collection, analysis, and output generation.

[1301] Cloud storage: A storage system that stores and manages necessary data.

[1302] software

[1303] Natural Language Processing Engine (NLP Engine): Analyzes natural language requests from users, for example, using the Google Natural Language API.

[1304] Emotion recognition engine (emotion engine): Analyzes the user's emotions from captured audio and video. For example, it uses the Microsoft Azure Emotion API.

[1305] Data collection and conversion module: A module that acquires the necessary data from the data source and converts it into a unified format.

[1306] Data analysis module: A module that analyzes the collected and converted data and generates output.

[1307] Explanation of the system processing flow

[1308] The server has a means of collecting data from multiple data sources and converting it into a unified format. The user, a store clerk, wears the smart glasses and serves customers. The customer's questions and requests are captured by a microphone and analyzed by a natural language processing engine. Next, the emotion engine recognizes the customer's emotional state using the smart glasses' camera and microphone.

[1309] The server analyzes the collected data in real time using an analysis module and automatically generates output based on the results. The generated output is provided to the store clerk with the source of the data clearly indicated. It also has a means of accepting corrections and feedback from the user (store clerk) and regenerating the final output that reflects that. This series of processes dynamically adjusts the content of responses and the way information is presented according to the customer's emotional state.

[1310] Specific examples

[1311] For example, consider a situation where a sales associate is asked, "How much of this item do you have in stock?" The microphone in the smart glasses captures the question, and the NLP engine analyzes it. The server collects inventory data from cloud storage and converts it into a unified format.

[1312] The emotion engine analyzes the captured audio and video data and recognizes whether the customer is speaking with a serious expression. The server analyzes inventory status and related promotion information and generates optimal output based on the customer's interests. For example, it could provide concise and easy-to-understand inventory information, as well as recommended products and discount information.

[1313] Through such interactions, customers receive quick and accurate information, increasing their satisfaction.

[1314] Example prompts to input to the generative AI model

[1315] Enter the following into the "Microsoft Azure Emotion API":

[1316] Customer facial expression images and voice data

[1317] Enter the following into "Google Natural Language API":

[1318] Text of the sales associate's question, "How much of this item do you have in stock?"

[1319] As described above, the system of the present invention can effectively handle customer service in physical stores by recognizing the user's emotional state in real time and dynamically adjusting the response content based on that.

[1320] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1321] Step 1:

[1322] Input: The user (store clerk) puts on the smart glasses and enters the login information.

[1323] Processing: The terminal sends login information to the server. The server performs authentication and obtains connection information for data sources (product inventory data, customer history data, etc.).

[1324] Output: Save the data source connection information to the internal database. Display a login confirmation message on the smart glasses display.

[1325] Step 2:

[1326] Input: A user (store associate) makes a data collection request in natural language (e.g., "How much of this item do you have in stock?").

[1327] Processing: The device captures the request content and sends it to the server, which then analyzes it using a natural language processing engine.

[1328] Output: Generates data collection and analysis conditions extracted from the request (e.g., inventory information for a specific product) and passes them to the data collection and transformation module.

[1329] Step 3:

[1330] Input: The camera and microphone in the smart glasses capture the customer's facial images and voice data.

[1331] Processing: The terminal sends the captured data to the emotion recognition engine, which the server uses to analyze the customer's emotional state.

[1332] Output: Generates the parsed emotion data and passes it to the output adjustment module.

[1333] Step 4:

[1334] Input: Data collection conditions identified by the Data Collection and Transformation Module.

[1335] Processing: The server sends API requests to data sources to collect the required data, and converts the collected data into a unified format (e.g., CSV or JSON).

[1336] Output: The data converted into a unified format is passed to the data analysis module.

[1337] Step 5:

[1338] Input: Data converted into a unified format.

[1339] Processing: The server uses a data analysis module to analyze the collected and converted data in real time, extracting requested information such as specific product categories and inventory levels.

[1340] Output: Pass the analysis results to the output generation module.

[1341] Step 6:

[1342] Input: Analysis results and customer sentiment data.

[1343] Processing: The server uses the output generation module to generate the optimal output depending on the customer's emotional state. For example, if the customer is feeling anxious, it provides detailed information that will reassure the customer.

[1344] Output: The generated output is displayed on the smart glasses display.

[1345] Step 7:

[1346] Input: The user (store clerk) checks the provided output and makes any necessary corrections.

[1347] Processing: The terminal sends the modifications to the server, which analyzes them and reflects them in the output generation module.

[1348] Output: The final output reflecting the modifications is reproduced and displayed on the smart glasses display.

[1349] Step 8:

[1350] Input: The final output.

[1351] Processing: The server temporarily stores the final output and generates a share link to other devices or systems as needed.

[1352] Output: The user is provided with a shared link and access information for the file.

[1353] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1354] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1355] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1356] [Fourth embodiment]

[1357] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1358] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1359] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1360] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1361] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1362] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1363] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1364] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1365] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1366] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1367] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1368] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1369] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1370] The present invention relates to a system for automating data collection, analysis, and output generation in an enterprise environment, which can efficiently collect data from multiple data sources, convert it into a unified format, analyze it, and ultimately automatically generate the output required by the user.

[1371] Explanation of program processing

[1372] The program of this system performs processing in the following procedure.

[1373] 1. Data Source Registration

[1374] During initial setup, the user enters connection information for cloud storage and data warehouses into the system.

[1375] The server uses the entered information to connect to each data source and obtain an API key and authentication token.

[1376] The server stores information about data sources with which a connection has been successfully established in an internal database.

[1377] 2. Data Collection Requests

[1378] A user uses a conversational interface to make a data collection request in natural language (e.g., "Collect sales data for 2023").

[1379] The server receives the request and analyzes the request using a natural language processing engine.

[1380] The server identifies specific data collection conditions (e.g., period, data items) based on the analysis results.

[1381] 3. Data collection and conversion

[1382] The server automatically collects the required data from each data source based on the specified conditions.

[1383] For example, download a spreadsheet from cloud storage and run an SQL query from the data warehouse to get the results.

[1384] The server converts the collected data into a unified format (e.g., CSV, JSON).

[1385] 4. Data Analysis

[1386] The server then applies an analysis algorithm to the data converted into a unified format to extract the necessary information.

[1387] The analysis results are made available to users in the format they request.

[1388] 5. Generating Output

[1389] The server automatically generates output based on the analysis results, which can be in the form of a spreadsheet, PDF report, presentation material, or other format, depending on the user's request.

[1390] Generated outputs will clearly indicate the source from which the data was obtained.

[1391] 6. Providing and Modifying Output

[1392] The server temporarily stores the generated output and notifies the user.

[1393] After receiving the notification, the user logs into the system to view the generated output.

[1394] The user makes any necessary corrections to the output content and sends the corrections to the system.

[1395] The server then incorporates the modifications received from the user and recreates the final output.

[1396] The server provides the final output to the user and shares it via a download link, email, or other means.

[1397] Specific examples

[1398] For example, the following describes an example in which a company's sales department generates a monthly report based on sales data for the past year.

[1399] 1. A user logs in to the system and requests, "Create a monthly report based on sales data from the past year."

[1400] 2. The server parses this request and collects the past year's sales data from a spreadsheet in cloud storage and also retrieves the necessary information from the data warehouse.

[1401] 3. The server converts the collected data into a unified format and applies an analysis algorithm to compile sales figures for each month.

[1402] 4. The server generates a monthly report in spreadsheet format based on the analysis results, and also indicates the source of the data.

[1403] 5. The user reviews the generated report, makes any necessary corrections, and then downloads the final report for use.

[1404] In this way, the present invention efficiently automates a series of operations from data collection to output generation, significantly reducing the workload on the user.

[1405] The processing flow will be explained below.

[1406] Step 1:

[1407] The user accesses the system's initial setup screen and enters connection information for data sources such as cloud storage and data warehouses.

[1408] Step 2:

[1409] The server connects to each data source using the entered connection information to obtain an API key and authentication token, and if successful, stores the data source information in an internal database.

[1410] Step 3:

[1411] A user uses a conversational interface to input a data collection request in natural language (e.g., "Collect monthly sales data for the year 2023").

[1412] Step 4:

[1413] The server sends the user's request to a natural language processing engine, which analyzes the request. Based on the analysis results, specific data collection conditions (e.g., period, data items) are extracted.

[1414] Step 5:

[1415] The server begins collecting data for each data source based on the specified data collection conditions.

[1416] For example, download a spreadsheet from cloud storage and submit SQL queries to a data warehouse to retrieve the required data.

[1417] Step 6:

[1418] The server converts the collected data from each data source into a unified format (e.g., CSV, JSON), which also includes data cleansing and normalization steps.

[1419] Step 7:

[1420] The server inputs the data converted into a unified format into an analysis algorithm, extracting and aggregating data that meets the specified conditions.

[1421] Step 8:

[1422] The server automatically generates output based on the analysis results in the specified output format (e.g., spreadsheet, PDF, presentation), including information on the origin of the collected data.

[1423] Step 9:

[1424] The server temporarily stores the automatically generated output and sends a notification to the user.

[1425] Step 10:

[1426] The user receives the notification sent and accesses the system to view the generated output.

[1427] Step 11:

[1428] The user checks the output content, makes corrections as necessary, and sends the corrections to the system.

[1429] Step 12:

[1430] The server receives the user's modifications and recreates the final output to reflect those modifications.

[1431] Step 13:

[1432] The server then provides the final regenerated output to the user and shares it via a download link, email, or other means.

[1433] Example 1

[1434] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1435] In a corporate environment, collecting data from multiple sources, converting it into a unified format, analyzing it, and automatically generating the final output is a very time-consuming process. It also requires a lot of time and effort to clearly indicate the source of the information and regenerate the output in response to user requests for corrections. Furthermore, the lack of a system that can accept user requests in natural language through a dialogue interface and analyze them in real time impairs user convenience. These issues need to be addressed.

[1436] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1437] In this invention, the server includes: [means for collecting information from multiple information sources]; [means for converting the collected information into a unified format]; and [means for analyzing the information converted into the unified format]. This makes it possible to efficiently automate a series of tasks from collecting information to analyzing it and generating output.

[1438] The "means for collecting information from multiple information sources" is a mechanism for automatically obtaining necessary information from multiple information sources specified by the user.

[1439] "Means for converting collected information into a unified format" refers to a mechanism for converting information collected in different formats into a unified format.

[1440] "Means for analyzing information converted into a unified format" refers to a mechanism for performing the necessary analysis based on information converted into a unified format and extracting meaningful data.

[1441] The "means for automatically generating output based on analyzed information" is a mechanism for automatically generating output in a format requested by a user based on analyzed information.

[1442] "Means for indicating the source of information in the output" refers to a mechanism for indicating from which source the information in the generated output was obtained.

[1443] The "means for providing the generated output to the user and accepting corrections from the user" is a mechanism for providing the generated output to the user and accepting corrections instructed by the user as necessary.

[1444] The "means for regenerating the final output reflecting the corrections" is a mechanism for regenerating the final output based on the corrections provided by the user.

[1445] "Means for providing an interactive interface and receiving information gathering requests in natural language from users" refers to a mechanism for providing an interactive interface that allows users to make information gathering requests in natural language and for receiving such requests.

[1446] "Means for analyzing collected information in real time" refers to a mechanism for instantly analyzing collected information and providing results in real time.

[1447] The present invention provides a system for automating data collection, analysis, and output generation in an enterprise environment. This system efficiently collects information from multiple sources, converts it into a unified format, analyzes it, and finally automatically generates the output required by the user.

[1448] System Overview

[1449] This system consists of a server, terminals, and users. The server performs the following main functions:

[1450] 1. A means of gathering information from multiple sources

[1451] 2. A means of converting the collected information into a unified format

[1452] 3. A means of analyzing information converted into a unified format

[1453] 4. A means to automatically generate output based on the analyzed information

[1454] 5. A means of citing the source of information in your output

[1455] 6. A means of providing the generated output to users and accepting user corrections.

[1456] 7. A means to reproduce the final output reflecting the modifications.

[1457] 8. Means for providing an interactive interface and receiving natural language information gathering requests from users

[1458] 9. Means of analyzing collected information in real time

[1459] Hardware and software used

[1460] This system uses the following hardware and software:

[1461] Servers: Includes database servers, application servers, storage servers, etc.

[1462] Cloud Storage: Common cloud storage services (e.g., Amazon S3, Google Drive).

[1463] Data warehouse: A data warehouse service such as Amazon Redshift or Google BigQuery.

[1464] Natural language processing engines: IBM Watson NLP, Google Cloud Natural Language API, etc.

[1465] Analysis algorithms: Python's pandas library, numpy library, etc.

[1466] Specific examples

[1467] For example, the specific process for generating a monthly report based on sales data for the past year in a company's sales department is shown below.

[1468] 1. A user logs in to the system and requests, "Create a monthly report based on sales data from the past year."

[1469] 2. The server parses this request using a natural language processing engine (e.g., Google Cloud Natural Language API), collects sales data from a spreadsheet in cloud storage (e.g., Google Drive) for the past year, and also retrieves necessary information from a data warehouse (e.g., Amazon Redshift).

[1470] 3. The server converts the collected data into CSV format and uses Python's pandas library to compile sales for each month.

[1471] 4. The server generates a monthly report in Google Sheets based on the analysis results, and also clearly indicates the source of the data.

[1472] 5. The user reviews the generated report, makes any necessary corrections, and then downloads the final report for use.

[1473] Prompt Sentence Examples

[1474] "Generate monthly reports based on the past year's worth of sales data."

[1475] In this way, the present invention efficiently automates a series of operations from data collection to output generation, significantly reducing the workload on the user.

[1476] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1477] Step 1:

[1478] During initial setup, users enter connection information for information sources into the system, including credentials for cloud storage and data warehouses.

[1479] The server connects to each information source based on the entered connection information and obtains an API key and authentication token. For example, it uses the Google Drive API to perform OAuth authentication and obtains an access token.

[1480] The server stores information about sources that have been successfully connected to in an internal database. The input is the connection information provided by the user, and the output is the connection information stored in the database.

[1481] Step 2:

[1482] A user uses a conversational interface to make a data collection request in natural language, for example, "collect sales data for 2023."

[1483] The server receives this request and analyzes it using a natural language processing engine. Specifically, it uses the Google Cloud Natural Language API to tokenize the request and extract important elements (time period, data items).

[1484] The input is the user's natural language request and the output is the parsed collection conditions.

[1485] Step 3:

[1486] The server identifies specific data collection conditions based on the analysis results and automatically collects the necessary data from each information source.

[1487] For example, the server uses the Google Drive API to download a spreadsheet from cloud storage and then runs a SQL query against Amazon Redshift to retrieve the data.

[1488] The input is the specified data collection conditions and the output is the collected raw data.

[1489] Step 4:

[1490] The server converts the collected data into a unified format (e.g., CSV, JSON), specifically using the Python pandas library to unify the data format.

[1491] The input is the raw data collected and the output is the data converted into a unified format.

[1492] Step 5:

[1493] The server then applies an analysis algorithm to the data converted into a unified format to extract the necessary information.

[1494] For example, use Python's pandas library to analyze sales data and calculate the total sales for each month.

[1495] The input is data converted into a unified format, and the output is the analyzed result data.

[1496] Step 6:

[1497] The server automatically generates output based on the analysis results, which can be in the form of a spreadsheet, PDF report, presentation material, or other format, depending on the user's request.

[1498] Specifically, it uses the Google Sheets API to create a spreadsheet and also generates a PDF file if the user requests it.

[1499] The input is the analysis result data, and the output is the generated output.

[1500] Step 7:

[1501] The server will attribute data to the generated output by annotating each section of the output with the source from which the data was obtained.

[1502] Inputs are generated outputs, and outputs are sourced outputs.

[1503] Step 8:

[1504] The server temporarily stores the generated output and notifies the user, for example by sending a link to the output via email or a notification system.

[1505] The input is the output produced, and the output is the notification to the user.

[1506] Step 9:

[1507] After receiving the notification, the user logs into the system to view the generated output, specifically by accessing the web interface and viewing the report.

[1508] The input is the notified output link and the output is the user feedback.

[1509] Step 10:

[1510] The user makes necessary corrections to the output content and submits the corrections to the system, for example, by inputting additional comments or correction instructions into the report content.

[1511] The input is the user's correction instruction, and the output is the correction request.

[1512] Step 11:

[1513] The server reflects the modifications received from the user and regenerates the final output, specifically by reanalyzing the data based on the modifications and generating new output.

[1514] The input is the user's modification request and the output is the regenerated final output.

[1515] Step 12:

[1516] The server provides the final output to the user and shares it via a download link, email, or other means.

[1517] The input is the reproduced final output, and the output is the presentation of the final output to the user.

[1518] (Application example 1)

[1519] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1520] Production management and quality control in factories involve a wide range of tasks, such as collecting and analyzing data obtained from numerous sensors and devices, and generating reports. This process is largely manual, which is inefficient and has a high risk of error. Furthermore, it is difficult to monitor data and detect anomalies in real time, making it difficult to respond flexibly to situations that require a rapid response.

[1521] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1522] In this invention, the server includes: means for collecting data from multiple data sources; means for converting the collected data into a unified format; means for analyzing the data converted into the unified format; means for automatically generating output based on the analyzed data; means for indicating the source of the data in the output; means for providing the generated output to a user and accepting user corrections; means for regenerating the final output reflecting the corrections; means for collecting data from various devices and sensors in a factory environment and automatically generating a production management report; means for collecting data in real time and displaying the analysis results on a dashboard; and means for issuing an alert when an abnormality occurs. This automates the process from data collection to analysis and output generation at the production site, enabling efficient management. Real-time data monitoring and anomaly detection enable rapid problem resolution.

[1523] "Data Source" means a device or system that provides data from an external or internal system or device.

[1524] "Uniform format" refers to a standardized format for converting different types of data into a consistent format.

[1525] "Analysis" refers to the act of processing collected and transformed data to extract meaningful information or results.

[1526] "Output" refers to the output results provided in the form of reports, graphs, etc. generated based on the results of the analyzed data.

[1527] "Source" refers to information indicating the source from which the collected data was obtained.

[1528] A "production management report" refers to a report that compiles information on production efficiency and quality control at the production site.

[1529] "Real-time data" refers to data that is acquired and analyzed immediately, without any delay.

[1530] A "dashboard" is an interface that visually displays data summaries and details in real time.

[1531] An "alert" is a warning message or signal that notifies you of an abnormality or condition that requires attention.

[1532] 1. System Overview

[1533] To implement this invention, it is necessary to build a system that collects data from multiple data sources, converts it into a unified format, analyzes it, and generates output. This system collects data from various devices and sensors in a factory environment and automatically generates production management reports. It also has the ability to collect data in real time, display analysis results on a dashboard, and issue alerts if an abnormality occurs.

[1534] 2. Hardware and Software Used

[1535] Hardware

[1536] Production line sensors (e.g. temperature sensors, humidity sensors)

[1537] Factory equipment (e.g., production machinery, quality inspection equipment)

[1538] Robot control PC

[1539] software

[1540] Database management system (e.g. MySQL)

[1541] Natural language processing engine (e.g. Google Cloud Natural Language API)

[1542] Data analysis libraries (e.g., Pandas, Numpy)

[1543] Dashboard display framework (e.g. Grafana)

[1544] 3. System Operation

[1545] Data collection

[1546] The server stores connection information for each sensor and device and collects data in real time. For example, it periodically obtains temperature data from a temperature sensor and operation data from production machines.

[1547] Data Conversion

[1548] The server converts the collected data into a unified format (e.g., CSV or JSON), allowing data from different formats to be parsed in a consistent manner.

[1549] Data analysis

[1550] The converted data is analyzed using data analysis libraries (Pandas, Numpy), and daily, weekly, and monthly production efficiency and quality control data are extracted as analysis results.

[1551] Output Generation

[1552] The server automatically generates output based on the analysis results, in formats including spreadsheets, PDF reports, dashboard displays, etc. The generated output also clearly indicates the source of the data.

[1553] Real-time monitoring and alert notifications

[1554] The server displays the data in real time on a dashboard, allowing users to intuitively monitor the status of the production site. If an abnormality occurs, an alert is sent to the user, enabling them to take prompt action.

[1555] 4. Specific Examples

[1556] For example, consider a factory production management system that generates an environmental management report based on data from a temperature sensor over the past month. A user uses an interactive interface to send a request such as, "Create an environmental management report based on this month's temperature data." The server analyzes this request, collects the necessary data, and converts it into a unified format. It then applies a data analysis algorithm to generate a monthly environmental management report and provides it to the user.

[1557] Prompt Sentence Examples

[1558] "Collect temperature sensor data for the past month and generate monthly environmental management reports."

[1559] Data collection range: Past month

[1560] Report items: Average temperature, maximum temperature, minimum temperature, temperature fluctuation by date

[1561] In this way, the invention significantly improves efficiency and accuracy on the production floor through automated data collection, analysis, and output generation. Real-time data monitoring and alerting also enable rapid problem resolution.

[1562] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1563] Step 1:

[1564] Data source registration

[1565] The server receives connection information for each device and sensor in the factory provided by the user.

[1566] Specific operation: The user enters connection information such as the IP address and API key of each sensor and device in the factory.

[1567] Input: Connection information entered by the user

[1568] Output: A configuration file or database containing connection information

[1569] The server connects to each data source based on the received connection information, obtains API keys and authentication tokens, and stores them in an internal database.

[1570] Step 2:

[1571] Data collection

[1572] The server collects data in real time from each registered data source.

[1573] Specific operation: The server sends a data collection request to each sensor (temperature sensor, humidity sensor, etc.) and device (production machine, etc.).

[1574] Input: Connection information for each data source

[1575] Output: Collected real-time data (e.g. temperature data, operation data)

[1576] The server periodically collects data and temporarily stores the collected data.

[1577] Step 3:

[1578] Data Conversion

[1579] Convert the collected data into a unified format (e.g., CSV or JSON).

[1580] Specific operation: The data collected by the server is converted into a unified format using a format conversion library.

[1581] Input: Collected real-time data

[1582] Output: Data converted into a unified format

[1583] This allows different forms of data to be parsed consistently.

[1584] Step 4:

[1585] Data analysis

[1586] The server analyzes the data converted into a unified format using data analysis libraries (Pandas, Numpy).

[1587] Specific operation: The server analyzes the data in the unified format using a data analysis algorithm.

[1588] Input: Data converted into a unified format

[1589] Output: Analysis results (e.g. production efficiency, quality control statistics)

[1590] As a result of the analysis, daily, weekly, and monthly production efficiency and quality control data are extracted.

[1591] Step 5:

[1592] Output Generation

[1593] The server automatically generates output based on the analysis results.

[1594] Specific operation: The server creates output such as spreadsheets, PDF reports, and dashboard displays based on the analysis results.

[1595] Input: Analysis results

[1596] Output: The generated output (e.g. spreadsheet, PDF report)

[1597] The output also clearly indicates the source of the data.

[1598] Step 6:

[1599] Providing and modifying generated outputs

[1600] The generated output is provided to the user and corrections are accepted from the user.

[1601] Specific behavior: The server sends the output to the user, who reviews and modifies it.

[1602] Input: Generated output

[1603] Output: Correction requests from the user

[1604] When the user submits a modification request, the server recreates the final output to reflect the modifications.

[1605] Step 7:

[1606] Real-time monitoring and alert notifications

[1607] The server displays the data in real time on a dashboard and sends an alert if an abnormality occurs.

[1608] What it does: The server uses a dashboard framework (e.g., Grafana) to display real-time data and sends alerts to the user if an anomaly is detected.

[1609] Input: Real-time data, analysis results

[1610] Output: Dashboard display, alert notification

[1611] This allows users to intuitively monitor the situation at the production site and quickly resolve problems.

[1612] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1613] This invention relates to a system that automates data collection, analysis, and output generation, as well as dynamically adjusts the process by recognizing user emotions. By combining an emotion engine, this system provides optimal responses and outputs according to the user's emotional state, improving the user experience.

[1614] Explanation of program processing

[1615] The program of this system performs processing in the following procedure.

[1616] 1. Data Source Registration

[1617] During initial setup, the user enters connection information for cloud storage and data warehouses into the system.

[1618] The server uses the entered information to connect to each data source and obtain an API key and authentication token. If successful, the server stores the data source information in its internal database.

[1619] 2. Data Collection Requests

[1620] A user uses a conversational interface to input a data collection request in natural language (e.g., "Collect monthly sales data for the year 2023").

[1621] The server receives the request and analyzes it using a natural language processing engine. Based on the analysis results, specific data collection conditions (e.g., time period, data items) are extracted.

[1622] 3. Emotion Recognition by Emotion Engine

[1623] The device recognizes the user's emotional state while inputting a request using an emotion engine, which analyzes voice tone, facial expressions, text content, etc. to determine the user's emotion.

[1624] The server receives the emotion data sent from the emotion engine and dynamically adjusts the request content and response method.

[1625] 4. Data collection and transformation

[1626] The server begins collecting data for each data source based on the specified data collection conditions.

[1627] For example, download a spreadsheet from cloud storage and submit an SQL query to the data warehouse to retrieve the required data.

[1628] The server converts the collected data into a unified format (e.g., CSV, JSON), which also includes data cleansing and normalization steps.

[1629] 5. Data Analysis

[1630] The server inputs the data converted into a unified format into an analysis algorithm, extracting and aggregating data that meets the specified conditions.

[1631] 6. Generating Output

[1632] The server automatically generates output in the specified output format (e.g., spreadsheet, PDF, presentation) based on the analysis results, including data source information.

[1633] The server optimizes the format and content of the output depending on the user's emotional state: for example, if the user is feeling stressed, it provides a concise summary of the information.

[1634] 7. Providing and Modifying Output

[1635] The server temporarily stores the automatically generated output and sends a notification to the user.

[1636] The user receives the notification sent and accesses the system to view the generated output.

[1637] The user checks the output content, makes corrections as necessary, and sends the corrections to the system.

[1638] The server receives the user's modifications and recreates the final output to reflect those modifications.

[1639] The server then provides the final regenerated output to the user and shares it via a download link, email, or other means.

[1640] Specific examples

[1641] For example, the following describes an example in which a company's sales department generates a monthly report based on sales data for the past year.

[1642] 1. A user logs in to the system and requests, "Create a monthly report based on sales data from the past year."

[1643] 2. The server parses this request and collects the past year's sales data from a spreadsheet in cloud storage and also retrieves the necessary information from the data warehouse.

[1644] 3. The device uses an emotion engine to recognize the emotional state of the user when making a request, for example, detecting that the user is stressed through voice analysis.

[1645] 4. The server converts the collected data into a unified format and applies an analysis algorithm to compile sales figures for each month.

[1646] 5. The server automatically generates a spreadsheet in a concise and easy-to-understand format, taking into account the user's feelings. The generated report also includes information about the source of the data.

[1647] 6. The user reviews the generated report, makes any necessary corrections, and then downloads the final report for use.

[1648] In this way, the system of the present invention, which is combined with an emotion engine, efficiently automates a series of tasks from data collection to output generation, and provides flexible responses according to the user's emotional state.

[1649] The processing flow will be explained below.

[1650] Step 1:

[1651] The user accesses the system's initial setup screen and enters connection information for data sources such as cloud storage and data warehouses.

[1652] Step 2:

[1653] The server connects to each data source using the entered connection information, obtains an API key and authentication token, and if the connection is successful, stores the data source information in an internal database.

[1654] Step 3:

[1655] A user uses a conversational interface to input a data collection request in natural language (e.g., "Collect monthly sales data for the year 2023").

[1656] Step 4:

[1657] The server sends the user's request to a natural language processing engine, which analyzes the request. Based on the analysis results, specific data collection conditions (e.g., period, data items) are extracted.

[1658] Step 5:

[1659] The device uses a camera and microphone to recognize the user's emotional state through an emotion engine while the user is typing a request, for example, using facial expression recognition technology and voice tone analysis.

[1660] Step 6:

[1661] The server receives the emotion data sent from the emotion engine and dynamically adjusts the request content and response method. If the user is feeling stressed, the server simplifies the request and processes it.

[1662] Step 7:

[1663] The server collects data from each data source based on the specified data collection criteria, for example, downloading a spreadsheet from cloud storage and submitting SQL queries to a data warehouse to retrieve data.

[1664] Step 8:

[1665] The server converts the collected data into a unified format (e.g., CSV, JSON), which also includes data cleansing and normalization steps.

[1666] Step 9:

[1667] The server applies an analysis algorithm to the data converted into a unified format, extracting and aggregating data that meets specified conditions.

[1668] Step 10:

[1669] The server automatically generates output in a specified format (e.g., spreadsheet, PDF, presentation) based on the analysis results, and the format and content of the output are optimized based on the user's emotional state.

[1670] Step 11:

[1671] The server temporarily stores the automatically generated output and sends a notification to the user.

[1672] Step 12:

[1673] The user receives the notification sent and accesses the system to view the generated output.

[1674] Step 13:

[1675] The user checks the output content, makes corrections as necessary, and sends the corrections to the system.

[1676] Step 14:

[1677] The server incorporates the modifications received from the user and recreates the final output based on those modifications.

[1678] Step 15:

[1679] The server then provides the final regenerated output to the user and shares it via a download link, email, or other means.

[1680] Example 2

[1681] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1682] Conventional data collection and analysis systems perform uniform processing without considering the user's emotional state, resulting in a poor user experience. Furthermore, because there is no system that dynamically adjusts the process based on the user's emotions, it is difficult to quickly and appropriately provide the information that stressed users want.

[1683] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1684] In this invention, the server includes: [means for collecting data from multiple data sources;] [means for converting the collected data into a unified format;] [means for analyzing the data converted into the unified format;] [means for automatically generating output based on the analyzed data;] [means for indicating the source of the data in the output;] [means for providing the generated output to a user and accepting corrections from the user;] [means for regenerating a final output that reflects the corrections;] [means for recognizing the user's emotional state and dynamically adjusting the process; and [means for analyzing the user's emotions using an emotion engine.] This makes it possible to provide appropriate and prompt output that takes the user's emotional state into consideration.

[1685] "Data Source" means an external or internal system or database that provides data.

[1686] A "uniform format" refers to a data format for converting data of different formats or structures into a consistent format.

[1687] "Analysis" refers to the process of processing collected data for a specific purpose and extracting useful information.

[1688] "Output" refers to results such as reports or reports generated based on analyzed data.

[1689] "User" refers to a person or organization that operates this system and uses the collected data and analysis results.

[1690] An "emotion engine" refers to an algorithm or system that analyzes a user's emotional state and dynamically adjusts processes based on that information.

[1691] "Prompt sentence" refers to a natural language request sentence that a user inputs to a system through an interactive interface.

[1692] This invention relates to a system that automates data collection, analysis, and output generation, as well as dynamically adjusts the process by recognizing user emotions. By combining an emotion engine, this system provides optimal responses and outputs according to the user's emotional state, improving the user experience.

[1693] This system is implemented through the following specific operations by the server, the terminal, and the user.

[1694] When a user first sets up the system, they enter connection information for their cloud storage or data warehouse. For example, they can use Google Drive as their cloud storage or Amazon Redshift as their data warehouse. The server then connects to each using the provided API key or authentication token, and stores the data source information in an internal database. This internal database can be a database management system such as MySQL.

[1695] When a user makes a data collection request, they use a conversational interface (e.g., a chatbot). For example, they input a prompt such as, "Collect monthly sales data for 2023" or "Create a monthly report based on sales data from the past year." This request is sent to the server and analyzed by a natural language processing engine (e.g., Google Cloud Natural Language API). Based on the analysis results, specific data collection conditions (period, data items) are extracted.

[1696] Furthermore, the device uses an emotion engine (e.g., Microsoft Azure's Emotion API) to recognize the user's emotional state while they are inputting a request. For example, it uses a microphone to collect audio and detects whether the user is feeling stressed through voice analysis. This emotional data is sent to the server and used to adjust the process. If the user is feeling stressed, the server will change its response to be more concise and quick.

[1697] Data collection is performed by the server based on specified conditions, such as downloading a specific spreadsheet from Google Drive and sending SQL queries to Amazon Redshift to retrieve the required data. The collected data is then converted into a unified format (CSV or JSON) using a Python library called Pandas.

[1698] The server then inputs the data converted into this unified format into an analysis algorithm to extract and aggregate the data. The analysis algorithm uses a machine learning library such as Scikit-learn. Based on the analysis results, the server automatically generates output in the specified output format (spreadsheet, PDF, presentation, etc.). The Python ReportLab library can be used to generate the output.

[1699] The generated output is temporarily stored by the server and notified to the user. After receiving the notification, the user accesses the system to check the output and make any necessary corrections. The corrections are sent to the system and reflected in the final output. The optimized output is then provided to the user and shared via a download link, email, or other means.

[1700] In this way, the system of the present invention, which is combined with an emotion engine, efficiently automates a series of tasks from data collection to output generation, and provides flexible responses according to the user's emotional state.

[1701] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1702] Step 1: User registers data source

[1703] During initial setup, users enter connection information for cloud storage and data warehouses into the system, including the cloud storage service name, API key, and authentication token.

[1704] The server receives these inputs, connects to each data source, and obtains authentication information. If the connection is successful using the obtained API key or authentication token, the server stores the data source information in an internal database, such as a MySQL database.

[1705] Step 2: Request data collection

[1706] A user uses a conversational interface to input a data collection request, such as the prompt "Collect monthly sales data for the year 2023."

[1707] The server receives this request and analyzes it using a natural language processing engine (for example, Google Cloud Natural Language API). As a result of the analysis, specific data collection conditions, such as "2023," "monthly," and "sales data," are extracted. This clarifies the specific conditions for data collection.

[1708] Step 3: Emotion recognition by the emotion engine

[1709] The device uses an emotion engine (e.g., Microsoft Azure's Emotion API) to recognize the user's emotional state while they are inputting requests, which can include voice and facial expression data.

[1710] The server receives the emotion data sent from the device and detects whether the user is feeling stressed. Based on the detection results, the server dynamically adjusts the request content and response method. For example, if the user is feeling stressed, the server changes the response to a simpler and easier-to-understand format.

[1711] Step 4: Data collection and transformation

[1712] The server collects data from data sources based on specified data collection criteria, with inputs including, for example, spreadsheets from cloud storage or SQL queries to a data warehouse.

[1713] The server converts the collected data into a unified format (e.g., CSV or JSON) using libraries such as Pandas to cleanse and normalize the data. The output is a dataset converted into a unified format.

[1714] Step 5: Data analysis

[1715] The server receives the unified formatted data and feeds it into an analysis algorithm, which includes as input the unified formatted data from the previous step.

[1716] The server uses machine learning libraries such as Scikit-learn to extract and aggregate data that meets the specified criteria. The output is a dataset and aggregated information as the analysis results.

[1717] Step 6: Generate output

[1718] The server automatically generates output based on the analysis results, and includes the analysis result dataset as input.

[1719] The server generates output in the specified format (spreadsheet, PDF, presentation, etc.) using libraries such as ReportLab, while also providing clear information about the origin of the data. The output is automatically generated.

[1720] Step 7: Provide and modify outputs

[1721] The server temporarily stores the generated output and sends notifications to the user, for example by email or push notification.

[1722] The user receives a notification, accesses the system to view the generated output, makes corrections as needed, and submits the corrections to the system.

[1723] The server receives the user's modifications and recreates the final output, which is then provided to the user via a download link, email, or other means.

[1724] (Application example 2)

[1725] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1726] Currently, customer service in brick-and-mortar stores is highly dependent on the proficiency and communication skills of store staff, resulting in inconsistent quality. It is also difficult to instantly grasp a customer's emotional state, especially when they are worried or unsure, and provide the optimal response accordingly. Furthermore, in situations where quick and accurate information provision is required, flexible responses based on emotions are essential, but current systems are unable to adequately address this. This creates a challenge, making it difficult to improve customer satisfaction.

[1727] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1728] In this invention, the server includes: [means for collecting data from multiple data sources;] [means for converting the collected data into a unified format;] [means for recognizing the user's emotional state; and] [means for dynamically adjusting output and response methods according to the recognized emotional state.] This makes it possible [to improve the quality of customer service in physical stores and enable flexible and appropriate interactions according to the customer's emotions].

[1729] "Multiple data sources" refers to a collection of data storage or systems used to obtain data from different locations and formats.

[1730] "Data collection methods" means the processes and tools used to obtain the required information from each data source.

[1731] "Means of converting collected data into a unified format" refers to techniques and methods for converting data of different formats or structures into a consistent format.

[1732] "Means for analyzing data converted into a consistent format" means means for analyzing data converted into a consistent format and extracting useful information.

[1733] "Means for automated output generation" refers to systems or algorithms that automatically generate documents or reports in the required format based on the analyzed data.

[1734] "Means for attributing data in outputs" means a mechanism for attributing the origin or source of data used in the generated outputs.

[1735] "Means for providing the generated output to the user and accepting corrections from the user" refers to the process of displaying and providing the generated output to the user and accepting corrections or feedback from the user.

[1736] "Means for regenerating the final output reflecting the modifications" means a system or process for recreating the final output based on the modifications received from the user.

[1737] "Means for recognizing the user's emotional state" refers to technologies and algorithms that analyze the user's voice, facial expressions, text, etc. to determine their emotional state.

[1738] "Means for dynamically adjusting output and response methods in response to a recognized emotional state" means means for automatically selecting and adjusting appropriate output and response methods based on the user's emotional state.

[1739] As an embodiment of the present invention, a specific system configuration and program processing will be described below. In this system, store clerks who deal with customers wear smart glasses and analyze customer emotions while providing necessary information in real time.

[1740] Hardware and Software Configuration

[1741] Hardware

[1742] Smart glasses: Smart devices with built-in cameras and microphones that can capture customers' facial expressions and voices in real time.

[1743] Server: A central computer system for data collection, analysis, and output generation.

[1744] Cloud storage: A storage system that stores and manages necessary data.

[1745] software

[1746] Natural Language Processing Engine (NLP Engine): Analyzes natural language requests from users, for example, using the Google Natural Language API.

[1747] Emotion recognition engine (emotion engine): Analyzes the user's emotions from captured audio and video. For example, it uses the Microsoft Azure Emotion API.

[1748] Data collection and conversion module: A module that acquires the necessary data from the data source and converts it into a unified format.

[1749] Data analysis module: A module that analyzes the collected and converted data and generates output.

[1750] Explanation of the system processing flow

[1751] The server has a means of collecting data from multiple data sources and converting it into a unified format. The user, a store clerk, wears the smart glasses and serves customers. The customer's questions and requests are captured by a microphone and analyzed by a natural language processing engine. Next, the emotion engine recognizes the customer's emotional state using the smart glasses' camera and microphone.

[1752] The server analyzes the collected data in real time using an analysis module and automatically generates output based on the results. The generated output is provided to the store clerk with the source of the data clearly indicated. It also has a means of accepting corrections and feedback from the user (store clerk) and regenerating the final output that reflects that. This series of processes dynamically adjusts the content of responses and the way information is presented according to the customer's emotional state.

[1753] Specific examples

[1754] For example, consider a situation where a sales associate is asked, "How much of this item do you have in stock?" The microphone in the smart glasses captures the question, and the NLP engine analyzes it. The server collects inventory data from cloud storage and converts it into a unified format.

[1755] The emotion engine analyzes the captured audio and video data and recognizes whether the customer is speaking with a serious expression. The server analyzes inventory status and related promotion information and generates optimal output based on the customer's interests. For example, it could provide concise and easy-to-understand inventory information, as well as recommended products and discount information.

[1756] Through such interactions, customers receive quick and accurate information, increasing their satisfaction.

[1757] Example prompts to input to the generative AI model

[1758] Enter the following into the "Microsoft Azure Emotion API":

[1759] Customer facial expression images and voice data

[1760] Enter the following into "Google Natural Language API":

[1761] Text of the sales associate's question, "How much of this item do you have in stock?"

[1762] As described above, the system of the present invention can effectively handle customer service in physical stores by recognizing the user's emotional state in real time and dynamically adjusting the response content based on that.

[1763] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1764] Step 1:

[1765] Input: The user (store clerk) puts on the smart glasses and enters the login information.

[1766] Processing: The terminal sends login information to the server. The server performs authentication and obtains connection information for data sources (product inventory data, customer history data, etc.).

[1767] Output: Save the data source connection information to the internal database. Display a login confirmation message on the smart glasses display.

[1768] Step 2:

[1769] Input: A user (store associate) makes a data collection request in natural language (e.g., "How much of this item do you have in stock?").

[1770] Processing: The device captures the request content and sends it to the server, which then analyzes it using a natural language processing engine.

[1771] Output: Generates data collection and analysis conditions extracted from the request (e.g., inventory information for a specific product) and passes them to the data collection and transformation module.

[1772] Step 3:

[1773] Input: The camera and microphone in the smart glasses capture the customer's facial images and voice data.

[1774] Processing: The terminal sends the captured data to the emotion recognition engine, which the server uses to analyze the customer's emotional state.

[1775] Output: Generates the parsed emotion data and passes it to the output adjustment module.

[1776] Step 4:

[1777] Input: Data collection conditions identified by the Data Collection and Transformation Module.

[1778] Processing: The server sends API requests to data sources to collect the required data, and converts the collected data into a unified format (e.g., CSV or JSON).

[1779] Output: The data converted into a unified format is passed to the data analysis module.

[1780] Step 5:

[1781] Input: Data converted into a unified format.

[1782] Processing: The server uses a data analysis module to analyze the collected and converted data in real time, extracting requested information such as specific product categories and inventory levels.

[1783] Output: Pass the analysis results to the output generation module.

[1784] Step 6:

[1785] Input: Analysis results and customer sentiment data.

[1786] Processing: The server uses the output generation module to generate the optimal output depending on the customer's emotional state. For example, if the customer is feeling anxious, it provides detailed information that will reassure the customer.

[1787] Output: The generated output is displayed on the smart glasses display.

[1788] Step 7:

[1789] Input: The user (store clerk) checks the provided output and makes any necessary corrections.

[1790] Processing: The terminal sends the modifications to the server, which analyzes them and reflects them in the output generation module.

[1791] Output: The final output reflecting the modifications is reproduced and displayed on the smart glasses display.

[1792] Step 8:

[1793] Input: The final output.

[1794] Processing: The server temporarily stores the final output and generates a share link to other devices or systems as needed.

[1795] Output: The user is provided with a shared link and access information for the file.

[1796] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1797] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1798] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1799] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1800] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1801] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1802] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1803] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1804] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1805] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1806] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1807] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1808] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1809] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1810] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1811] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1812] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1813] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1814] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1815] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1816] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1817] The following is further disclosed regarding the above embodiment.

[1818] (Claim 1)

[1819] [means for collecting data from multiple data sources;

[1820] [Means of converting collected data into a unified format;

[1821] [Means for analyzing data converted into a unified format;

[1822] [Means for automatically generating output based on the analyzed data; and

[1823] [Means of indicating the source of data in the output and

[1824] [Means for providing the generated output to the user and accepting modifications from the user; and

[1825] [Means to regenerate the final output reflecting the modifications, and

[1826] A system including:

[1827] (Claim 2)

[1828] 10. The system of claim 1, further comprising means for providing an interactive interface and receiving a data collection request from a user in natural language.

[1829] (Claim 3)

[1830] The system of claim 1, further comprising means for analyzing the collected data in real time.

[1831] "Example 1"

[1832] (Claim 1)

[1833] [Means of collecting information from multiple sources;

[1834] [Means of converting the collected information into a unified format;

[1835] [Means for analyzing the information converted into a unified format;

[1836] [Means for automatically generating output based on the analyzed information; and

[1837] [Means of indicating the source of information in the output,

[1838] [Means for providing the generated output to the user and accepting modifications from the user; and

[1839] [Means to regenerate the final output reflecting the modifications, and

[1840] A system including:

[1841] (Claim 2)

[1842] The system of claim 1, further comprising means for providing an interactive interface and receiving a natural language request from a user to gather information.

[1843] (Claim 3)

[1844] [The system of claim 1, further comprising means for analyzing the collected information in real time.

[1845] "Application Example 1"

[1846] (Claim 1)

[1847] [means for collecting data from multiple data sources;

[1848] [Means of converting collected data into a unified format;

[1849] [Means for analyzing data converted into a unified format;

[1850] [Means for automatically generating output based on the analyzed data; and

[1851] [Means of indicating the source of data in the output and

[1852] [Means for providing the generated output to the user and accepting modifications from the user; and

[1853] [Means to regenerate the final output reflecting the modifications, and

[1854] [In a factory environment, a means to collect data from each device and sensor and automatically generate production management reports,

[1855] [Means of collecting data in real time and displaying the analysis results on a dashboard,

[1856] [Means to notify alerts when abnormalities occur, and

[1857] A system including:

[1858] (Claim 2)

[1859] The system of claim 1, providing an interactive interface and receiving a data collection request from a user in natural language.

[1860] (Claim 3)

[1861] [The system of claim 1, wherein the collected data is analyzed in real time to generate reports on production efficiency and quality control.

[1862] "Example 2: Combining Emotion Engines"

[1863] (Claim 1)

[1864] [means for collecting data from multiple data sources;

[1865] [Means of converting collected data into a unified format;

[1866] [Means for analyzing data converted into a unified format;

[1867] [Means for automatically generating output based on the analyzed data; and

[1868] [Means of indicating the source of data in the output and

[1869] [Means for providing the generated output to the user and accepting modifications from the user; and

[1870] [Means to regenerate the final output reflecting the modifications, and

[1871] [Means for recognizing the user's emotional state and dynamically adjusting the process;

[1872] [Means for analyzing user emotions using an emotion engine;

[1873] A system including:

[1874] (Claim 2)

[1875] 10. The system of claim 1, further comprising means for providing an interactive interface and receiving a data collection request from a user in natural language.

[1876] (Claim 3)

[1877] The system of claim 1, further comprising means for analyzing the collected data in real time.

[1878] "Application example 2 when combining emotion engines"

[1879] (Claim 1)

[1880] [means for collecting data from multiple data sources;

[1881] [Means of converting collected data into a unified format;

[1882] [Means for analyzing data converted into a unified format;

[1883] [Means for automatically generating output based on the analyzed data; and

[1884] [Means of indicating the source of data in the output and

[1885] [Means for providing the generated output to the user and accepting modifications from the user; and

[1886] [Means to regenerate the final output reflecting the modifications, and

[1887] [means for recognizing the emotional state of a user;

[1888] [Means for dynamically adjusting output and response methods depending on perceived emotional states; and

[1889] A system including:

[1890] (Claim 2)

[1891] 10. The system of claim 1, further comprising means for providing an interactive interface and receiving a data collection request from a user in natural language.

[1892] (Claim 3)

[1893] The system of claim 1, further comprising means for analyzing the collected data in real time. [Explanation of symbols]

[1894] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for collecting data from a plurality of data sources; A means of converting the collected data into a unified format; means for analyzing the data converted into the unified format; a means for automatically generating output based on the analyzed data; and A means to clearly indicate the source of data in the output, and means for providing the generated output to a user and accepting modifications from the user; A means to reproduce the final output reflecting the modifications; and A system including:

2. 10. The system of claim 1, further comprising means for providing an interactive interface and receiving a natural language request for data collection from a user.

3. 10. The system of claim 1, further comprising means for analyzing the collected data in real time.

Images