System

Abstract

A system is provided.SOLUTION: A system comprising: means for collecting market data, public information, and in-house data; means for parsing user-entered natural language prompts to generate typed prompts; means for querying data sources based on the typed prompts to obtain needed data; means for integrating, cleaning, and pre-processing the obtained data; means for visualizing the integrated data in a user-specified format; and means for providing an interface for a user to filter and customize the data visualization results.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] The problem that this invention aims to solve is the difficulty that businesspeople have in quickly and easily accessing market data, public information, and internal closed data, and in centrally managing and visualizing the data. In particular, data collection, data analysis, and visualization of the results require a great deal of time and specialized knowledge, resulting in reduced business efficiency and delays in prompt management decisions. To solve this problem, there is a need to provide a system that allows users to access the necessary data simply by entering prompts, and automatically collect, analyze, and visualize the data. [Means for solving the problem]

[0005] The present invention solves the above-mentioned problems by taking the following measures. Specifically, the present invention provides a system including: means for collecting market data, public information, and internal company data; means for analyzing natural language prompts entered by a user to generate typed prompts; means for executing queries against data sources based on the typed prompts to obtain required data; means for integrating, cleaning, and preprocessing the obtained data; means for visualizing the integrated data in a user-specified format; and means for providing an interface for users to filter and customize the data visualization results. This system automates the process from data collection to visualization, allowing users to easily and quickly analyze data and make management decisions.

[0006] "Market Data" means publicly available market data, such as market trends, competitive information, and economic indicators.

[0007] "Public information" is information that is publicly available and accessible to anyone in a database or on the Internet.

[0008] "Internal data" refers to data such as confidential information, sales performance, and customer information that is collected and maintained within a company.

[0009] A "prompt" is a natural language sentence containing a question or command that a user enters into a system.

[0010] "Typification" is the process of parsing natural language input and converting it into a structured form that the system can understand.

[0011] A "query" is an inquiry or request sent to a database or API to retrieve specific data.

[0012] "Visualization" refers to the visual display of data using graphs, tables, dashboards, etc.

[0013] "Interface" refers to the screens and designs that facilitate information exchange and operation between the user and the system.

[0014] "Cleaning" is the process of correcting or removing missing or outlier values ​​in data to improve data quality.

[0015] "Preprocessing" refers to preparatory work such as formatting data and standardizing the format that is carried out before data analysis and model building. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] MODE FOR CARRYING OUT THE INVENTION

[0038] The present invention provides a system that allows business people to quickly and easily access market data, public information, and internal company data, and efficiently manages and visualizes the data in a unified manner. The following describes an embodiment of this system.

[0039] System Overview

[0040] The system mainly consists of a server, a terminal, and a user. The server collects, analyzes, integrates, and visualizes data, while the terminal provides an interface for users to enter prompts and check the visualization results. Users can automatically perform a series of processes from data collection to visualization simply by entering prompts.

[0041] What the program does

[0042] 1. Initial settings and data source registration

[0043] Server: The server stores API endpoints, authentication keys, database connection strings, etc. in configuration files, which establish connections to market data, public information, and internal databases.

[0044] User: The user provides the necessary API keys and database credentials to the server, which receives them and stores them securely using security features.

[0045] 2. Typing prompts

[0046] Terminal: The user types a natural language prompt into the terminal (e.g., "Sales analysis using market data and internal sales data for 2023").

[0047] Terminal: The terminal receives user input and parses the prompt using a natural language processing (NLP) module. The resulting prompt is then generated as a typed prompt and sent to the server.

[0048] 3. Data collection and processing

[0049] Server: The server receives the typed prompt and starts the data retrieval task. It sends an HTTP request to the market data API to retrieve market data for 2023. At the same time, it runs an SQL query against the internal database to retrieve sales data for 2023.

[0050] Server: The acquired data is stored in temporary storage and undergoes cleaning and pre-processing, which includes imputing missing values, standardizing data formats, and removing unnecessary data.

[0051] Server: The cleaned data is aggregated and joined using a common key (e.g., date or product ID).

[0052] 4. Data Visualization

[0053] Server: Passes the integrated data to a visualization tool and visualizes it in the format specified by the user (graph, table, dashboard, etc.). For example, it generates graphs using libraries such as Matplotlib or Tableau.

[0054] Terminal: The visualized data is sent to the terminal and displayed to the user, who can modify and customize the visualization using pull-down menus and filtering functions.

[0055] 5. Providing an Interface

[0056] Server: Provides a user-friendly interface that allows users to smoothly input data, complete prompts, view results, and filter and customize them.

[0057] Users: Using the provided interface, users can quickly analyze data and make management decisions and develop strategies. For example, they can adjust sales strategies based on the results of sales data analysis.

[0058] In this way, the system of the present invention automates the process from data collection to visualization, and provides an environment in which users can perform data analysis without needing specialized knowledge.

[0059] The processing flow will be explained below.

[0060] Step 1:

[0061] Server: The server configures the connection to the market data API and the internal database. Specifically, it stores the API endpoint, authentication key, and database connection string in a configuration file and authenticates against the API and database.

[0062] Step 2:

[0063] User: The user uses a device to send authentication information (API keys, database credentials) to the server, which receives it and stores it securely using security features.

[0064] Step 3:

[0065] Terminal: The user enters a natural language prompt into the terminal, for example, "Sales analysis using market data and internal sales data for 2023."

[0066] Step 4:

[0067] Terminal: The terminal receives the input natural language prompt and uses a natural language processing (NLP) module to parse and format the prompt. As a result of the prompt parsing, a formatted prompt is generated, including the data source to be used, the target time period, etc.

[0068] Step 5:

[0069] Terminal: Sends a typed prompt to the server.

[0070] Step 6:

[0071] Server: The server receives a typed prompt and begins the task of retrieving data by sending an HTTP request to the market data API to retrieve market data for the specified period.

[0072] Step 7:

[0073] Server: At the same time, the server runs a SQL query against the internal database to retrieve sales data for the specified period. For example, it runs the following SQL query:

[0074] sql

[0075] SELECT FROM sales WHERE year = 2023

[0076] Step 8:

[0077] Server: The acquired data is stored in temporary storage. After storage, data cleaning begins. Specifically, it performs tasks such as filling in missing values, standardizing data formats, and removing outliers.

[0078] Step 9:

[0079] Server: The cleaned data is consolidated using a common key (e.g., date or product ID), and the consolidated data is centralized and organized into a form suitable for analysis.

[0080] Step 10:

[0081] Server: Passes the consolidated data to visualization tools, for example, using visualization libraries such as Matplotlib or Tableau to generate graphs and tables.

[0082] Step 11:

[0083] Server: Sends visualized data to the device, where the data is presented in the format selected by the user (bar graph, line graph, dashboard, etc.).

[0084] Step 12:

[0085] Terminal: The terminal displays the visualization results to the user, who can further customize the displayed data using pull-down menus and filtering options.

[0086] Step 13:

[0087] Server: Ensures that data can be filtered, sorted, and prompted for additional input through a user-friendly interface, allowing users to easily explore data details and make quick business decisions based on the analysis results.

[0088] Through the above steps, this system provides an environment in which users can easily and efficiently analyze data and support rapid decision-making.

[0089] Example 1

[0090] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0091] In today's business environment, it is important to quickly and effectively integrate and analyze data from different sources. However, it is not easy to collect large amounts of market data, public information, and internal data, efficiently extract and integrate the necessary information, and visualize it in an easy-to-understand format. This task requires advanced expertise and a lot of time, placing a heavy burden on many businesspeople. In addition, the lack of intuitive ways to manipulate and customize visualized data makes it difficult to utilize analysis results in management decisions.

[0092] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0093] In this invention, the server includes: [means for collecting market data, public information, and internal company data;] [means for analyzing user-input natural language prompts and generating styled prompts using a generative AI model; and] [means for executing queries against data sources based on the styled prompts to obtain required data.] This makes it possible [to efficiently and effectively collect, integrate, analyze, and visualize required data from different data sources].

[0094] "Market data" refers to data that includes market trends, pricing information, supply and demand conditions, etc.

[0095] "Public information" is data that is publicly available and accessible to anyone, such as government statistics or news articles.

[0096] "Internal data" refers to data generated and managed within a company or organization. Examples include sales data, customer information, and inventory data.

[0097] A "natural language prompt" is an instruction or question entered by a user using natural language.

[0098] A "generative AI model" is a model that uses artificial intelligence to analyze prompts, understand their meaning, and generate structured prompts.

[0099] A "typed prompt" is a natural language prompt that has been parsed and converted into a specific format, making it easier for the system to extract the exact information needed.

[0100] A "Data Source" is an external or internal source from which data is obtained, such as an API, a database, or a CSV file.

[0101] A "query" is a query statement used to obtain information from a data source such as a database or API.

[0102] "Data cleaning" is the process of preparing data for analysis by filling in missing values, standardizing formats, and removing unnecessary data.

[0103] "Preprocessing" refers to a series of preparation steps that are performed before data analysis, including data cleaning and format conversion.

[0104] "Visualization" refers to the visual presentation of data in the form of graphs, tables, dashboards, etc.

[0105] An "interface" is a screen or function that allows a user to interact with a system, such as providing a prompt input screen or data filtering functionality.

[0106] MODE FOR CARRYING OUT THE INVENTION

[0107] This invention provides a system that allows business people to quickly and easily access market data, public information, and internal company data, and efficiently manage and visualize them in a centralized manner. In implementing this system, three parties - a server, a terminal, and a user - work together, each of which plays a specific role.

[0108] Hardware and software used

[0109] Hardware: The server uses a cloud infrastructure (e.g., Amazon Web Services, Microsoft® Azure®, etc.) equipped with a high-performance CPU and large amounts of memory. The terminals are PCs, tablets, smartphones, etc. operated by users.

[0110] Software: The following will be used:

[0111] Natural language processing modules (e.g., spaCy, Transformers)

[0112] API management tools (libraries that execute HTTP requests)

[0113] Database management system (e.g. PostgreSQL)

[0114] Libraries for data cleaning and preprocessing (e.g., Pandas)

[0115] Data visualization tools (e.g., Matplotlib, Tableau)

[0116] Web frameworks (e.g. Django, Flask)

[0117] What the program does

[0118] Initial Settings and Data Source Registration

[0119] The server stores API endpoints, authentication keys, and database connection strings in configuration files that establish connections to market data, public information, and internal company databases.

[0120] Users enter the necessary API keys and database authentication information through a dedicated interface and send it to the server, which then securely stores the received authentication information using its security features.

[0121] Prompt typing

[0122] The terminal provides an interface for users to enter natural language prompts, such as "Sales analysis using market data and internal sales data for 2023."

[0123] The device uses a natural language processing module (NLP) to parse the prompt and generate a typed prompt, which is then sent to the server.

[0124] Data collection and processing

[0125] The server receives the typed prompt and starts the data collection task, making an HTTP request to the market data API to retrieve market data for 2023. At the same time, it runs an SQL query against the internal database to retrieve sales data for 2023.

[0126] The server stores the acquired data in temporary storage and performs data cleaning and preprocessing, such as standardizing data formats, filling in missing values, and removing unnecessary data. The cleaned data is then integrated using a common key.

[0127] Data Visualization

[0128] The server passes the integrated data to a visualization tool, which visualizes it in the format specified by the user (e.g., graph, table, dashboard). For example, a sales progress graph can be generated using libraries such as Matplotlib or Tableau.

[0129] The terminal receives the visualized data and displays it to the user, who can access it through an interface that allows filtering and customization of the visualization results.

[0130] Providing an interface

[0131] The server provides a user-friendly interface that allows users to easily input data, enter prompts, view results, filter and customize, etc.

[0132] Users can quickly analyze data using the provided interface and use it to make business decisions and develop strategies. For example, they can adjust their sales strategies based on the results of sales data analysis.

[0133] As described above, the present invention automates the process from data collection to visualization, and provides an environment in which users can perform data analysis without needing specialized knowledge.

[0134] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0135] Step 1:

[0136] Initial Settings and Data Source Registration

[0137] The server saves the API endpoint, authentication key, and database connection string in a configuration file to establish connections to market data, public information, and internal data. The input is the API endpoint URL, authentication key, and database connection information, and the connection configuration is based on this. The output is the connection configuration information saved.

[0138] The user enters the necessary API keys and database credentials into a dedicated interface and sends them to the server. The input is the user-provided credentials sent to the server. The output is the credentials securely stored on the server.

[0139] Step 2:

[0140] Prompt typing

[0141] The terminal provides an interface for the user to input a prompt sentence in natural language. For example, the user might input "Sales analysis using market data and internal sales data for 2023." The input is the prompt sentence entered by the user in natural language.

[0142] The terminal parses the input prompt sentence using a natural language processing (NLP) module. As a result of the parsing, a typed prompt is generated and sent to the server. The output is the typed prompt sentence.

[0143] Step 3:

[0144] Data collection

[0145] The server initiates the data collection task based on the typed prompt, sending an HTTP request to the market data API to retrieve market data for the year 2023. The input is the typed prompt, and the API request is generated based on this information. The output is the retrieved market data stored in temporary storage.

[0146] The server runs SQL queries against the internal database to retrieve sales data for the year 2023. Similarly, the input is a typed prompt, and an SQL query is generated based on this information. As an output, the retrieved internal sales data is saved in temporary storage.

[0147] Step 4:

[0148] Data cleaning and preprocessing

[0149] The server temporarily stores the acquired market data and internal sales data, and then cleans and preprocesses the data. Specifically, it standardizes formats, fills in missing values, and deletes unnecessary data. The input is the acquired market data and internal sales data, and the output is cleaned, integrated data.

[0150] Step 5:

[0151] Data integration

[0152] The server aggregates the cleaned data, joining the data using a common key (e.g., date or product ID) to generate the aggregated data. The inputs are the cleaned market data and internal company data, and the output is the aggregated data.

[0153] Step 6:

[0154] Data Visualization

[0155] The server passes the integrated data to a visualization tool, which visualizes it in the format specified by the user (e.g., graph, table, dashboard, etc.). For example, a graph of sales trends is generated using a library such as Matplotlib or Tableau. The input is the integrated data, and the visualized data is generated as the output.

[0156] The terminal receives the visualized data and displays it to the user, who uses an interface to filter and customize the displayed graphs and tables. The input is the visualized data, and the output is the user-customized visualization.

[0157] Step 7:

[0158] Providing an interface

[0159] The server provides a user-friendly interface, allowing users to easily input data, enter prompts, check results, filter, customize, etc. The input is the information for the interface design, and the output is the operation screen that the user can use.

[0160] Users can use the provided interface to quickly analyze data and use it for business decisions and strategic planning. For example, they can adjust their sales strategy based on the results of sales data analysis. The input is the user's operations, and the output is specific analysis results.

[0161] (Application example 1)

[0162] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0163] In conventional factory operations, a wide variety of data is generated from various sources, but it has been difficult to efficiently collect, integrate, and visualize this data. It has been particularly difficult to monitor and analyze important performance indicators, such as the number of abnormalities and product pass rates, in real time. As a result, delays have occurred in maintenance and optimization of production lines, hindering efficient operations. To solve these issues, a system is needed that can quickly collect and analyze real-time data generated within the factory and visualize the results in an easily understandable format.

[0164] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0165] In this invention, the server includes: [means for collecting market data, public information, and internal company data;] [means for analyzing user-input natural language prompts to generate typed prompts;] [means for executing queries on data sources based on the typed prompts to obtain required data;] [means for collecting real-time data from sensors and machine operating status APIs within the factory; and [means for analyzing the collected data and visualizing production line performance indicators such as the number of abnormalities and the pass rate.] This enables efficient collection, integration, and cleaning of various data generated within the factory, and also enables real-time visualization of important performance indicators such as the number of abnormalities and the pass rate, enabling rapid response to optimize factory operations.

[0166] "Market Data" refers to various statistical information, trends, and other data related to the market.

[0167] "Public information" is information that is publicly available, including company financial reports, news articles, industry reports, etc.

[0168] "Internal data" refers to various data generated within a company, including business reports, sales data, product inspection data, and the like.

[0169] A "natural language prompt" refers to an instruction or question that a user enters in natural language.

[0170] A "typed prompt" refers to a prompt that is parsed from a natural language prompt and converted into a format that is easy for the system to understand.

[0171] "Data source" refers to the location or system from which data is obtained. Examples include APIs, databases, sensors, etc.

[0172] A "query" is an instruction to a database or API to retrieve specific data.

[0173] "Data integration" refers to the process of combining data from different sources into a single, consistent format.

[0174] "Cleaning" refers to the preprocessing of data to detect and correct incomplete, inaccurate, or unnecessary data.

[0175] "Preprocessing" refers to various preparatory tasks carried out before data analysis or visualization, including data normalization and conversion.

[0176] "Visualization" refers to the process of representing data in a visual format, such as a graph or chart, to allow users to intuitively understand the information.

[0177] "Filtering" refers to the process of specifying specific conditions when displaying or analyzing data and displaying only the necessary data.

[0178] "Customization" refers to tailoring the appearance and behavior of a system to meet the needs and requirements of the user.

[0179] "Interface" refers to the screen and input means that allow the user to operate the system.

[0180] A "sensor" refers to a device that measures physical environmental variables (such as temperature, humidity, and vibration) and outputs the data as an electronic signal.

[0181] "Machine operating status API" refers to a program interface for obtaining machine operating status and performance data.

[0182] "Real-time data" refers to the latest data available at the current time.

[0183] "Performance indicators" are standards for measuring and evaluating the efficiency and effectiveness of a system or process.

[0184] The "number of abnormal events" indicates the frequency of abnormal events occurring within a certain period of time.

[0185] The "pass rate" indicates the percentage of products that meet the standards and are judged to have passed the inspection.

[0186] The present invention provides a system for quickly collecting, analyzing, and integrating real-time data generated in a factory, and visualizing the data. Specific embodiments for carrying out the present invention will be described below.

[0187] System Overview

[0188] The system mainly consists of a server, a terminal, and a user. The server collects, analyzes, integrates, and visualizes data, while the terminal provides an interface for users to enter prompts and check the visualization results. By entering prompts, users can easily automate the entire process from data collection to visualization.

[0189] Hardware and software used

[0190] Hardware:

[0191] Sensors: Devices that measure physical environmental variables (temperature, vibration, pressure, etc.) within the factory

[0192] Server: Collects, analyzes, stores, and visualizes data

[0193] Terminal: A device where users can enter prompts and view visualization results.

[0194] Machine Operation Status API: Interface for obtaining the machine operation status

[0195] software:

[0196] Natural Language Processing (NLP) libraries: spaCy, NLTK, etc.

[0197] Data collection and cleaning: Pandas library (Python)

[0198] Database connection: SQLAlchemy (Python)

[0199] Visualization libraries: Matplotlib, Seaborn (Python)

[0200] Process Details

[0201] 1. Data collection and integration

[0202] The server collects real-time data from sensors installed in the factory and machine health APIs, such as temperature data from sensors and machine uptime information, while user-entered prompts are parsed and converted into typed prompts using natural language processing.

[0203] 2. Data cleaning and preprocessing

[0204] The collected data is stored in temporary storage and undergoes cleaning and preprocessing, such as imputing missing values, standardizing data formats, and removing inaccurate data.

[0205] 3. Data Visualization

[0206] The preprocessed data is integrated and visualized in a specified format (graph, chart, etc.). For example, data on the number of abnormalities or product pass rates can be visually displayed over time.

[0207] 4. Providing a user interface

[0208] The terminal displays the visualization results to the user, providing a filterable and customizable interface through which the user can quickly and efficiently analyze the data.

[0209] Prompt Sentence Examples

[0210] Below are some examples of prompt sentences.

[0211] "Analyze this month's product pass rate and number of machine abnormalities"

[0212] "Sales analysis using 2023 market data and internal sales data"

[0213] "What time of day this month are you seeing the most anomalies?"

[0214] The system of the present invention improves factory operations, enabling real-time data monitoring and analysis, which allows for quicker maintenance and production line adjustments and more efficient factory operations.

[0215] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0216] Step 1:

[0217] The server collects real-time data from sensors in the factory and machine operation status APIs. This includes temperature data, vibration data, and operating time information. The data sent from the sensors is accumulated on the server via HTTP requests. The server stores the acquired data in temporary storage.

[0218] Input: Real-time data from sensors and machine health APIs

[0219] Output: Raw data stored in temporary storage

[0220] Step 2:

[0221] The user inputs a natural language prompt through the terminal, for example, "Analyze this month's product pass rate and machine abnormality count." The terminal receives this natural language prompt, analyzes it using a natural language processing (NLP) module, and generates a typed prompt.

[0222] Input: User's natural language prompt

[0223] Output: Typed prompt

[0224] Step 3:

[0225] The server receives typed prompts and runs queries against the collected data, for example using a database query to get temperature data or anomaly counts for a specific time period, and the results of this query are stored in temporary storage in the form of a data frame.

[0226] Input: Typed prompts, raw data stored in temporary storage

[0227] Output: The specific data retrieved by the query

[0228] Step 4:

[0229] The server cleans and preprocesses the acquired data, including imputing missing values, removing outliers, and standardizing data formats. It uses the Pandas library to clean the data table and convert it into a format suitable for analysis.

[0230] Input: The specific data retrieved by the query

[0231] Output: Cleaned and preprocessed data

[0232] Step 5:

[0233] The server analyzes the preprocessed data and visualizes it in the specified format. For example, it can display the number of abnormalities or product pass rates as a line graph or bar graph. It generates graphs using the Matplotlib or Seaborn libraries and saves them as image files.

[0234] Input: Cleaned and preprocessed data

[0235] Output: Visualized graph or chart

[0236] Step 6:

[0237] The server then sends the generated visualization data to the terminal, which receives it and displays it graphically to the user, who can then use the provided interface to view the results and filter and customize them as needed.

[0238] Input: A visualized graph or chart

[0239] Output: A graph or chart displayed on the user's terminal

[0240] Step 7:

[0241] Based on the displayed data, users can optimize their production lines and create maintenance plans. For example, they can identify time periods when there are many abnormalities and concentrate maintenance work during those periods. The results of user operations are also saved on the server and reflected in future data analysis.

[0242] Input: User interactions and results

[0243] Output: Insights for maintenance planning and production line optimization

[0244] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0245] MODE FOR CARRYING OUT THE INVENTION

[0246] This invention provides a system that enables businesspeople to integrate market data, public information, and internal company data, and customizes data analysis by recognizing user emotions. The system is primarily composed of a server, terminals, and users, and is combined with an emotion engine. This makes it possible to dynamically adjust the data visualization method and presentation content according to the user's emotions.

[0247] System Overview

[0248] The system consists of the following main components:

[0249] 1. Server: The central processing unit responsible for data collection, analysis, integration, visualization, and emotion recognition.

[0250] 2. Terminal: A device that provides an interface for users to enter prompts and view visualization results.

[0251] 3. User: The person who uses the system to perform data analysis.

[0252] 4. Emotion Engine: A module that recognizes emotions from user input and behavior and customizes data analysis and visualization.

[0253] What the program does

[0254] 1. Initial settings and data source registration

[0255] Server: The server configures the connection to the market data API and the internal database, storing the API endpoint, authentication key, and database connection string in a configuration file, which then authenticates against the API and database.

[0256] User: The user provides credentials (API keys or database credentials) to the server, which stores them securely using security features.

[0257] 2. Typing prompts

[0258] Terminal: The user types a natural language prompt into the terminal, for example, "Sales analysis using market data and internal sales data for 2023."

[0259] Terminal: The terminal receives the natural language prompt, parses and formats the prompt using the NLP module, and sends the formatted prompt to the server.

[0260] 3. Data collection and processing

[0261] Server: The server initiates the data retrieval task based on a typed prompt. It sends an HTTP request to the market data API to retrieve market data for the specified period. At the same time, the server executes an SQL query against the internal database to retrieve sales data for the specified period.

[0262] Server: The acquired data is stored in temporary storage and undergoes cleaning and preprocessing, such as imputing missing values, standardizing data formats, and removing outliers.

[0263] Server: The cleaned data is integrated using a common key, and the integrated data is centralized and made suitable for analysis.

[0264] 4. Data Visualization

[0265] Server: Passes the integrated data to the visualization tool and visualizes it in the format specified by the user. It generates graphs and tables using visualization libraries such as Matplotlib and Tableau.

[0266] Server: Sends the visualized data to the terminal and displays it to the user, who can modify and customize the visualization using pull-down menus and filtering options.

[0267] 5. Emotion Engine

[0268] Server: Monitors user input and behavior and uses an emotion engine to recognize user emotions, for example, inferring stress or satisfaction from the user's typing speed and keystroke patterns.

[0269] Server: Dynamically adjusts the visualization and presentation of data based on the perceived emotion. If the user is nervous, it might provide a simpler graph or add a more detailed explanation.

[0270] 6. Providing an Interface

[0271] Server: Provides a user-friendly interface that allows users to smoothly input data, complete prompts, view results, filter and customize them, etc.

[0272] Users: Use a customized interface based on emotion recognition to quickly analyze data and make business decisions. For example, if a user is feeling stressed, the system will provide a simplified visualization and guide subsequent operations.

[0273] In this way, the system of the present invention automates the process from data collection to visualization, and further provides data analysis customized according to the user's emotions, thereby providing an environment in which users can efficiently utilize data without the need for specialized knowledge.

[0274] The processing flow will be explained below.

[0275] Step 1:

[0276] Server: The server configures the connection to the market data API and the internal database. Specifically, it stores the API endpoints, authentication keys, and database connection strings in configuration files and establishes the connection to the API and database.

[0277] Step 2:

[0278] User: The user uses a device to send their credentials (API key or database credentials) to the server, which receives them and stores them securely using security features.

[0279] Step 3:

[0280] Terminal: The user types a natural language prompt into the terminal (e.g., "Sales analysis using market data and internal sales data for 2023").

[0281] Step 4:

[0282] Terminal: The terminal receives the input natural language prompt and parses it using a natural language processing (NLP) module, generating a typed prompt as a result, which it then sends to the server:

[0283] {

[0284] "action": "Sales analysis",

[0285] "data_sources": ["Market Data API", "Internal Database"],

[0286] "year": 2023

[0287] }

[0288] Step 5:

[0289] Server: The server receives the typed prompt and sends an HTTP request to the market data API to retrieve market data for the specified period. It also executes an SQL query against the internal database to retrieve sales data for the specified period. For example, it executes the following SQL query:

[0290] sql

[0291] SELECT FROM sales WHERE year = 2023

[0292] Step 6:

[0293] Server: The acquired data is stored in temporary storage. After storage, data cleaning is performed. Specifically, missing values ​​are filled, the data format is standardized, and outliers are removed.

[0294] Step 7:

[0295] Server: The cleaned data is integrated using a common key (date or product ID), and the integrated data is centralized and organized into a form suitable for analysis.

[0296] Step 8:

[0297] Server: Passes the consolidated data to a visualization tool and visualizes it in the format specified by the user. For example, it generates graphs and tables using libraries such as Matplotlib or Tableau.

[0298] Step 9:

[0299] Server: Sends visualized data to the device, where it is displayed in the format selected by the user (bar graph, line graph, dashboard, etc.).

[0300] Step 10:

[0301] Terminal: The terminal displays the visualization results to the user, who can customize the displayed data using pull-down menus and filtering options.

[0302] Step 11:

[0303] Server: Uses an emotion engine to recognize emotions from user input and behavior, for example, inferring stress or satisfaction from a user's typing speed and keystroke patterns.

[0304] Step 12:

[0305] Server: Dynamically adjusts the visualization and presentation of data based on the user's emotions as recognized by the emotion engine. For example, if the user is nervous, provide a simpler graph or add a detailed explanation.

[0306] Step 13:

[0307] Server: Sends customized visualization results to the device, presenting data in a way that is easier for users to understand.

[0308] Step 14:

[0309] Users: Through a customized interface, they can quickly analyze data and make business decisions, filtering and re-prompting as needed. For example, they can adjust marketing strategies for specific products or services based on the results of analyzing sales data.

[0310] In this way, the system of the present invention automates the process from data collection to visualization, and further provides customization according to the user's emotions, thereby providing an environment in which users can efficiently utilize data without the need for specialized knowledge.

[0311] Example 2

[0312] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0313] In conventional data analysis systems, market data, public information, and internal data are collected and analyzed separately, which makes data integration time-consuming and labor-intensive, making efficient data analysis difficult. Furthermore, they lack the functionality to dynamically adjust data analysis taking into account the user's emotions and stress levels, making them less user-friendly.

[0314] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0315] In this invention, the server includes: [means for collecting market data, public information, and internal company data]; [means for analyzing user-input natural language prompts to generate typed prompts]; [means for executing queries against data sources based on the typed prompts to obtain required data]; [means for integrating, cleaning, and preprocessing the acquired data]; [means for visualizing the integrated data in a user-specified format]; [means for providing an interface for users to filter and customize data visualization results]; and [means for monitoring user input and behavior, recognizing emotions, and dynamically adjusting data analysis and visualization.] This makes it possible [to simplify data integration processing and achieve efficient data analysis]. It also makes it possible [to recognize user emotions and customize data analysis in real time, improving operability and providing a stress-free data analysis environment].

[0316] "Market Data" refers to information relating to economic activity and transactions in a particular market.

[0317] "Public information" refers to data and information that is publicly available, including, for example, government statistics, news, and reports.

[0318] "Internal data" refers to data generated within a company, including sales data, customer information, and financial data.

[0319] "User input" refers to all input operations performed by users of the system via their terminals.

[0320] A "natural language prompt" refers to a sentence of instructions or requests that a user enters in everyday language.

[0321] A "typed prompt" refers to a prompt that has been parsed from a natural language prompt and converted into a format that is easy for the system to process.

[0322] "Data source" refers to the origin or source from which data is obtained, including APIs and databases.

[0323] A "query" is a request made to a database or API to search or retrieve data.

[0324] "Cleaning" refers to the processing of data to improve its quality by correcting or removing missing or outlier values.

[0325] "Preprocessing" refers to preparatory work such as data standardization, conversion, and filtering that is carried out before data analysis.

[0326] "Visualization" refers to the process of displaying data in a visual format such as a graph, chart, or table.

[0327] "Filtering" refers to the process of selecting and displaying data based on specific conditions.

[0328] "Customization" refers to the operation of changing the system or the display format of data according to the user's needs.

[0329] "Interface" refers to the input / output means by which a user and a system interact with each other.

[0330] "Emotion recognition" refers to the process of identifying emotions from user input and behavior and having the system respond accordingly.

[0331] MODE FOR CARRYING OUT THE INVENTION

[0332] The present invention provides a system that enables businesspeople to handle market data, public information, and internal company data in an integrated manner, and further customizes data analysis by recognizing user emotions. The system is primarily composed of a server, a terminal, and a user, combined with an emotion recognition engine. Specific embodiments for implementing this system are described in detail below.

[0333] Hardware and Software Configuration

[0334] 1. Server

[0335] Hardware: Dedicated servers with powerful CPUs, large amounts of RAM and storage.

[0336] Software: Software for data collection, analysis, synthesis, visualization, and emotion recognition, including databases (e.g., MySQL®, PostgreSQL), programming languages ​​(e.g., Python), data visualization libraries (e.g., Matplotlib, Tableau), natural language processing engines (e.g., spaCy, NLTK), and emotion recognition models.

[0337] 2. Terminal

[0338] Hardware: The input and display devices used by users, such as PCs, tablets, and smartphones.

[0339] Software: Web browsers and applications for providing a user interface. Specific examples include web browsers (e.g., GOOGLE CHROME (registered trademark), Mozilla Firefox) and user interface frameworks (e.g., React, Vue.js).

[0340] 3. Users

[0341] Business people using systems to analyze data.

[0342] Data processing and calculation process

[0343] The server uses a natural language processing engine to parse the natural language prompts received from the user and generate typed prompts. Based on the typed prompts, the server queries market data APIs and internal databases to retrieve the required data. The retrieved data is stored in temporary storage, after which it is cleaned and pre-processed. Cleaning includes missing value imputation, data format standardization, and outlier removal. Pre-processing includes data standardization, transformation, and filtering.

[0344] The integrated data is visualized using a data visualization library in the form of graphs, charts, tables, etc. The generated visualization results are sent to the terminal, providing an interface for users to filter and customize the data based on specific conditions.

[0345] Additionally, the server monitors user input and behavior and uses an emotion recognition engine to identify the user's emotions. Based on the emotion, the system dynamically adjusts the visualization and presentation of the data. For example, if the user is feeling stressed, the system will provide a simpler graph and add a detailed explanation.

[0346] Specific examples

[0347] Example 1: Sales analysis of market data and internal sales data

[0348] User: Enter "Sales analysis using 2023 market data and internal sales data."

[0349] Terminal: Parses the prompt and sends the typed prompt to the server.

[0350] Server: Collects data from specified data sources, cleans and integrates it, uses a data visualization library to generate a time series graph of sales, and sends it to the terminal.

[0351] Server: Monitors the user's emotions and, if they are nervous, provides a simple graph and pops up a detailed explanation.

[0352] Example 2: Customer Satisfaction Analysis

[0353] User: Type "Create Customer Satisfaction Report for Q4 2022."

[0354] Terminal: Type natural language prompts and send them to the server.

[0355] Server: Acquires data on customer satisfaction (such as survey results), cleans and integrates them, and uses Tableau to generate a heat map of customer satisfaction and send it to the device.

[0356] Server: Adjust the color and detail of the heatmap based on user sentiment.

[0357] In this way, the system of the present invention not only automates the process of collecting, processing, and visualizing the necessary data based on the user's natural language prompts, but also customizes data analysis according to the user's emotions, thereby achieving fast and efficient data analysis.

[0358] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0359] Step 1:

[0360] Initial setup and registration of authentication information

[0361] Server: Store your market data API endpoints, authentication keys, and internal database connection strings in a configuration file, which will connect and authenticate to the API and database.

[0362] User: Enters API key and authentication information on the initial setup screen of the system. The server encrypts and stores this information securely and returns a connection success message.

[0363] Input: API endpoint, authentication key, database connection information

[0364] Output: Connection successful message

[0365] Specific operation: The user enters authentication information into the input form and clicks the submit button. The server sets up the connection based on the entered information and returns a message when the connection is successful.

[0366] Step 2:

[0367] Entering and parsing user prompts

[0368] User: Uses a terminal interface to enter natural language prompts. Example: "Sales analysis using market data and internal sales data for 2023."

[0369] Terminal: Parses the input natural language prompt using an NLP module (e.g. spaCy, NLTK), generates a typed prompt, and sends it to the server.

[0370] Input: Natural language prompt (e.g., "Sales analysis using market data and internal sales data for 2023")

[0371] Output: Typed prompt

[0372] Specific operation: The user enters a prompt, and the device analyzes the natural language prompt to extract key information such as "market data," "2023," "sales data," and "sales analysis," then formats and sends it to the server.

[0373] Step 3:

[0374] Data collection

[0375] Server: Initiates data collection tasks based on typed prompts, sends HTTP requests to the market data API to retrieve market data for the specified period, and simultaneously executes SQL queries against the internal database to retrieve the required sales data.

[0376] Input: Typed prompt

[0377] Output: Collected data (market data, internal sales data)

[0378] Specific operation: The server sends an HTTP GET request to "api.marketdata.com" to retrieve market data for 2023. At the same time, it executes the query "SELECT FROM sales_data WHERE year = 2023" against the internal database to retrieve sales data.

[0379] Step 4:

[0380] Data cleaning and preprocessing

[0381] Server: Stores the acquired data in temporary storage and performs cleaning and preprocessing, such as imputing missing values, standardizing data formats, and detecting and removing outliers.

[0382] Input: Collected data

[0383] Output: Cleaned data

[0384] Specific operation: The server checks the collected data and performs missing value imputation (e.g., replacing "0" or "null" with the appropriate value), formatting standardization (e.g., standardizing date formats), and outlier removal (e.g., removing abnormally high or low values).

[0385] Step 5:

[0386] Data Integration and Transformation

[0387] Server: The cleaned data is consolidated using a common key (e.g., date) and made suitable for analysis.

[0388] Input: Cleaned data

[0389] Output: Integrated data

[0390] What it does: The server joins market data and sales data based on "date" to create a unified data set, enabling more comprehensive analysis.

[0391] Step 6:

[0392] Data Visualization

[0393] Server: Passes the integrated data to a visualization tool (e.g., Matplotlib, Tableau) and visualizes it in the format specified by the user.

[0394] Server: The visualized data is sent to the terminal and displayed to the user, who can filter and customize it.

[0395] Input: Integrated data

[0396] Output: Visualization results (graphs and charts)

[0397] What it does: The server uses Matplotlib to generate a line graph of monthly sales and sends it to the terminal where the user can view the graph. The user can select a specific month from a dropdown menu to view more details.

[0398] Step 7:

[0399] Emotion Recognition and Adaptive Presentation

[0400] Server: Monitors user input and actions and uses an emotion recognition engine to infer emotions. Example: Determine stress levels based on typing speed and keystrokes.

[0401] Server: Dynamically adjusts visualization and presentation content based on the recognized emotion. Example: If the user is nervous, present a simple graph and add a detailed explanation.

[0402] Input: User operation data (typing speed, keystroke patterns)

[0403] Output: Adaptive visualization results

[0404] Specific operation: The server analyzes input speed information and, if it determines that the user is feeling stressed, it provides a simple bar graph that makes subtraction easy to understand and displays a detailed explanation in a pop-up.

[0405] Step 8:

[0406] Providing and customizing the interface

[0407] Server: Provides a user-friendly interface that allows for easy data entry, prompt entry, results review, filtering, and customization.

[0408] Users: Use a customized interface to quickly analyze data and make decisions.

[0409] Input: None (fixed interface)

[0410] Output: User-friendly interface

[0411] What it does: Users upload data on the data dashboard screen, select the appropriate filter options from the interface, and the graphs are updated in real time.

[0412] (Application example 2)

[0413] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0414] Modern logistics centers are swept up in a wide variety of data, and require systems that can efficiently collect, analyze, and visualize it. However, when employees are confronted with vast amounts of data, they can become stressed and fatigued, and there is a lack of ways to present information accordingly. This situation reduces work efficiency and becomes a source of stress in the workplace.

[0415] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0416] In this invention, the server includes: [means for collecting market data, public information, and business data;] [means for analyzing natural language prompts entered by a user to generate typed prompts; and] [means for recognizing a user's emotions using an emotion engine and dynamically adjusting the data visualization method.] This enables [employees to efficiently process massive amounts of data, and furthermore, the emotional information presentation method improves work efficiency and reduces stress].

[0417] "Market data" is a general term for external data related to commercial activities, such as market trends, trading information, and competitive analysis.

[0418] "Public Information" means data or information that is publicly available, including websites and public reports.

[0419] "Business data" refers to data generated within a company related to its daily operations, including sales information, inventory data, customer information, and so on.

[0420] "User" refers to the person who operates the system and performs data analysis.

[0421] "Natural language prompt" refers to a natural language instruction or question that a user enters into a system.

[0422] A "typed prompt" is an instruction that has been parsed from a natural language prompt and converted into a structure that is easy for the system to understand.

[0423] "Data Source" refers to the source from which data is obtained, including market data APIs and internal databases.

[0424] A "query" is a request to retrieve specific information from a database or API.

[0425] "Cleaning" refers to the process of preparing data by removing invalid values, missing values, and outliers before analyzing the data.

[0426] "Preprocessing" refers to the preparation of data before analysis, and includes standardizing formats and scaling.

[0427] "Visualization" refers to the process of displaying acquired and integrated data in a visual format such as a graph or chart.

[0428] "Filtering" is the process of extracting parts of data that meet specific conditions and using them for display or analysis.

[0429] "Customization" refers to adjusting the display content and functions of a system according to the user's requirements and circumstances.

[0430] "Interface" refers to the operation screen and input means used by users when interacting directly with the system.

[0431] The "emotion engine" is a module that analyzes the user's emotional state and dynamically adjusts the system's output and display.

[0432] "Smart glasses" refer to wearable devices that visually display information and allow users to check data in real time by wearing them.

[0433] This invention aims to enable employees working at logistics centers to efficiently collect, analyze, and visualize market data, business data, and real-time inventory information. In addition, by utilizing an emotion engine, it provides a method for presenting information according to the employee's emotional state, thereby improving work efficiency and reducing stress.

[0434] System Configuration

[0435] Hardware Configuration

[0436] Server: The central processing unit that collects, analyzes, consolidates, and visualizes data from market data APIs and operational databases.

[0437] Terminal: A device where a user inputs data and sees results, including smart glasses.

[0438] Smart glasses: Wearable devices that visually present data to the user in real time.

[0439] Software Configuration

[0440] Market Data Collection Module: Collects market data from external APIs.

[0441] Database Query Module: Executes queries against business databases to retrieve required data.

[0442] Data Cleaning and Integration Module: Cleans, pre-processes, and integrates the collected data.

[0443] Emotion engine: Analyzes data collected from the smart glasses' sensors and recognizes the user's emotions.

[0444] Visualization module: Visualizes the integrated data according to the user's emotional state.

[0445] Interface module: Provides an interface that allows users to filter and customize the data visualization results.

[0446] Processing flow

[0447] The server collects data from market data APIs and operational databases, cleans and pre-processes it, then integrates the data and creates visualizations based on the user's natural language prompts. The emotion engine recognizes the user's emotional state from the smart glasses' sensors and adjusts how the data is presented. Through the smart glasses, users can visually view the information.

[0448] Specifically, the user types the following natural language prompt into the terminal:

[0449] I want to conduct sales analysis using market data for 2023 and internal sales data.

[0450] The server analyzes the prompts and collects and analyzes the necessary data. After cleaning and preprocessing, the data is visualized by the emotion engine, taking into account the user's emotional state, and displayed on the smart glasses.

[0451] This allows users to view data in a way that is optimized for their emotional state, enabling them to make quick and efficient business decisions. For example, a user feeling tired will see a simple graph, while a relaxed user will be provided with detailed data.

[0452] This system is a powerful tool for increasing operational efficiency and reducing employee stress in logistics centers.

[0453] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0454] Step 1:

[0455] The server configures connections to market data APIs and business databases. Specifically, it saves the API endpoint, authentication key, and database connection string in a configuration file, and establishes connections to the API and database based on this information. The input is the API endpoint and authentication information, and the output is the connection establishment status.

[0456] Step 2:

[0457] The server receives a natural language prompt provided by the user. Through a terminal, the user inputs a prompt sentence such as: "I would like to perform a sales analysis using market data and business data for 2023." The input is the natural language prompt, and the output is the prompt sentence sent to the server.

[0458] Step 3:

[0459] The terminal parses the natural language prompt it receives and generates a typed prompt. It uses an NLP (Natural Language Processing) module to parse the prompt and convert it into a format that the system can understand. The input is the natural language prompt, and the output is the typed prompt.

[0460] Step 4:

[0461] The server queries the data source based on the typed prompts. It sends an HTTP request to the market data API to retrieve market data for the specified period. At the same time, it executes SQL queries against the operational database to retrieve the required data. The input is the typed prompts, and the output is the retrieved data.

[0462] Step 5:

[0463] The server cleans and preprocesses the acquired data. Specifically, it performs processes such as filling in missing values, standardizing data formats, and removing outliers. The cleaned data is saved in temporary storage. The input is the acquired data, and the output is the cleaned data.

[0464] Step 6:

[0465] The server integrates the cleaned data. It combines the market data and business data using a common key to generate integrated data. The input is the cleaned market data and business data, and the output is the integrated data.

[0466] Step 7:

[0467] The server visualizes the integrated data. It passes the data to the visualization tool in a user-specified format (e.g., graphs, charts, etc.) and displays it in a format that the user can easily understand. The input is the integrated data, and the output is the visualized data.

[0468] Step 8:

[0469] The server uses an emotion engine to recognize the user's emotions. It analyzes data obtained from the smart glasses' sensors (e.g., gaze tracking, voice, body movements, etc.) to understand the user's emotional state. The input is the sensor data, and the output is the recognized emotional state.

[0470] Step 9:

[0471] The server dynamically adjusts the visualization method based on the recognized emotional state. For example, if the user feels tired, it provides a simple graph, while if the user feels relaxed, it displays detailed data. The input is the recognized emotional state, and the output is the adjusted visualization data.

[0472] Step 10:

[0473] The terminal presents the adjusted visualized data to the user through the smart glasses. The user can check the data in real time through the smart glasses display and make business decisions. The input is the adjusted visualized data, and the output is the real-time visual information presented to the user.

[0474] In this way, efficient data visualization and emotion recognition are achieved through the processing and calculation of various input data at each processing step of the system.

[0475] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0476] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0477] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0478] [Second embodiment]

[0479] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0480] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0481] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0482] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0483] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0484] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0485] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0486] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0487] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0488] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0489] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0490] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0491] MODE FOR CARRYING OUT THE INVENTION

[0492] The present invention provides a system that allows business people to quickly and easily access market data, public information, and internal company data, and efficiently manages and visualizes the data in a unified manner. The following describes an embodiment of this system.

[0493] System Overview

[0494] The system mainly consists of a server, a terminal, and a user. The server collects, analyzes, integrates, and visualizes data, while the terminal provides an interface for users to enter prompts and check the visualization results. Users can automatically perform a series of processes from data collection to visualization simply by entering prompts.

[0495] What the program does

[0496] 1. Initial settings and data source registration

[0497] Server: The server stores API endpoints, authentication keys, database connection strings, etc. in configuration files, which establish connections to market data, public information, and internal databases.

[0498] User: The user provides the necessary API keys and database credentials to the server, which receives them and stores them securely using security features.

[0499] 2. Typing prompts

[0500] Terminal: The user types a natural language prompt into the terminal (e.g., "Sales analysis using market data and internal sales data for 2023").

[0501] Terminal: The terminal receives user input and parses the prompt using a natural language processing (NLP) module. The resulting prompt is then generated as a typed prompt and sent to the server.

[0502] 3. Data collection and processing

[0503] Server: The server receives the typed prompt and starts the data retrieval task. It sends an HTTP request to the market data API to retrieve market data for 2023. At the same time, it runs an SQL query against the internal database to retrieve sales data for 2023.

[0504] Server: The acquired data is stored in temporary storage and undergoes cleaning and pre-processing, which includes imputing missing values, standardizing data formats, and removing unnecessary data.

[0505] Server: The cleaned data is aggregated and joined using a common key (e.g., date or product ID).

[0506] 4. Data Visualization

[0507] Server: Passes the integrated data to a visualization tool and visualizes it in the format specified by the user (graph, table, dashboard, etc.). For example, it generates graphs using libraries such as Matplotlib or Tableau.

[0508] Terminal: The visualized data is sent to the terminal and displayed to the user, who can modify and customize the visualization using pull-down menus and filtering functions.

[0509] 5. Providing an Interface

[0510] Server: Provides a user-friendly interface that allows users to smoothly input data, complete prompts, view results, and filter and customize them.

[0511] Users: Using the provided interface, users can quickly analyze data and make management decisions and develop strategies. For example, they can adjust sales strategies based on the results of sales data analysis.

[0512] In this way, the system of the present invention automates the process from data collection to visualization, and provides an environment in which users can perform data analysis without needing specialized knowledge.

[0513] The processing flow will be explained below.

[0514] Step 1:

[0515] Server: The server configures the connection to the market data API and the internal database. Specifically, it stores the API endpoint, authentication key, and database connection string in a configuration file and authenticates against the API and database.

[0516] Step 2:

[0517] User: The user uses a device to send authentication information (API keys, database credentials) to the server, which receives it and stores it securely using security features.

[0518] Step 3:

[0519] Terminal: The user enters a natural language prompt into the terminal, for example, "Sales analysis using market data and internal sales data for 2023."

[0520] Step 4:

[0521] Terminal: The terminal receives the input natural language prompt and uses a natural language processing (NLP) module to parse and format the prompt. As a result of the prompt parsing, a formatted prompt is generated, including the data source to be used, the target time period, etc.

[0522] Step 5:

[0523] Terminal: Sends a typed prompt to the server.

[0524] Step 6:

[0525] Server: The server receives a typed prompt and begins the task of retrieving data by sending an HTTP request to the market data API to retrieve market data for the specified period.

[0526] Step 7:

[0527] Server: At the same time, the server runs a SQL query against the internal database to retrieve sales data for the specified period. For example, it runs the following SQL query:

[0528] sql

[0529] SELECT FROM sales WHERE year = 2023

[0530] Step 8:

[0531] Server: The acquired data is stored in temporary storage. After storage, data cleaning begins. Specifically, it performs tasks such as filling in missing values, standardizing data formats, and removing outliers.

[0532] Step 9:

[0533] Server: The cleaned data is consolidated using a common key (e.g., date or product ID), and the consolidated data is centralized and organized into a form suitable for analysis.

[0534] Step 10:

[0535] Server: Passes the consolidated data to visualization tools, for example, using visualization libraries such as Matplotlib or Tableau to generate graphs and tables.

[0536] Step 11:

[0537] Server: Sends visualized data to the device, where the data is presented in the format selected by the user (bar graph, line graph, dashboard, etc.).

[0538] Step 12:

[0539] Terminal: The terminal displays the visualization results to the user, who can further customize the displayed data using pull-down menus and filtering options.

[0540] Step 13:

[0541] Server: Ensures that data can be filtered, sorted, and prompted for additional input through a user-friendly interface, allowing users to easily explore data details and make quick business decisions based on the analysis results.

[0542] Through the above steps, this system provides an environment in which users can easily and efficiently analyze data and support rapid decision-making.

[0543] Example 1

[0544] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0545] In today's business environment, it is important to quickly and effectively integrate and analyze data from different sources. However, it is not easy to collect large amounts of market data, public information, and internal data, efficiently extract and integrate the necessary information, and visualize it in an easy-to-understand format. This task requires advanced expertise and a lot of time, placing a heavy burden on many businesspeople. In addition, the lack of intuitive ways to manipulate and customize visualized data makes it difficult to utilize analysis results in management decisions.

[0546] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0547] In this invention, the server includes: [means for collecting market data, public information, and internal company data;] [means for analyzing user-input natural language prompts and generating styled prompts using a generative AI model; and] [means for executing queries against data sources based on the styled prompts to obtain required data.] This makes it possible [to efficiently and effectively collect, integrate, analyze, and visualize required data from different data sources].

[0548] "Market data" refers to data that includes market trends, pricing information, supply and demand conditions, etc.

[0549] "Public information" is data that is publicly available and accessible to anyone, such as government statistics or news articles.

[0550] "Internal data" refers to data generated and managed within a company or organization. Examples include sales data, customer information, and inventory data.

[0551] A "natural language prompt" is an instruction or question entered by a user using natural language.

[0552] A "generative AI model" is a model that uses artificial intelligence to analyze prompts, understand their meaning, and generate structured prompts.

[0553] A "typed prompt" is a natural language prompt that has been parsed and converted into a specific format, making it easier for the system to extract the exact information needed.

[0554] A "Data Source" is an external or internal source from which data is obtained, such as an API, a database, or a CSV file.

[0555] A "query" is a query statement used to obtain information from a data source such as a database or API.

[0556] "Data cleaning" is the process of preparing data for analysis by filling in missing values, standardizing formats, and removing unnecessary data.

[0557] "Preprocessing" refers to a series of preparation steps that are performed before data analysis, including data cleaning and format conversion.

[0558] "Visualization" refers to the visual presentation of data in the form of graphs, tables, dashboards, etc.

[0559] An "interface" is a screen or function that allows a user to interact with a system, such as providing a prompt input screen or data filtering functionality.

[0560] MODE FOR CARRYING OUT THE INVENTION

[0561] This invention provides a system that allows business people to quickly and easily access market data, public information, and internal company data, and efficiently manage and visualize them in a centralized manner. In implementing this system, three parties - a server, a terminal, and a user - work together, each of which plays a specific role.

[0562] Hardware and software used

[0563] Hardware: The server uses a cloud infrastructure (e.g., Amazon Web Services, Microsoft Azure, etc.) equipped with a high-performance CPU and large amounts of memory. The terminals are PCs, tablets, smartphones, etc. operated by users.

[0564] Software: The following will be used:

[0565] Natural language processing modules (e.g., spaCy, Transformers)

[0566] API management tools (libraries that execute HTTP requests)

[0567] Database management system (e.g. PostgreSQL)

[0568] Libraries for data cleaning and preprocessing (e.g., Pandas)

[0569] Data visualization tools (e.g., Matplotlib, Tableau)

[0570] Web frameworks (e.g. Django, Flask)

[0571] What the program does

[0572] Initial Settings and Data Source Registration

[0573] The server stores API endpoints, authentication keys, and database connection strings in configuration files that establish connections to market data, public information, and internal company databases.

[0574] Users enter the necessary API keys and database authentication information through a dedicated interface and send it to the server, which then securely stores the received authentication information using its security features.

[0575] Prompt typing

[0576] The terminal provides an interface for users to enter natural language prompts, such as "Sales analysis using market data and internal sales data for 2023."

[0577] The device uses a natural language processing module (NLP) to parse the prompt and generate a typed prompt, which is then sent to the server.

[0578] Data collection and processing

[0579] The server receives the typed prompt and starts the data collection task, making an HTTP request to the market data API to retrieve market data for 2023. At the same time, it runs an SQL query against the internal database to retrieve sales data for 2023.

[0580] The server stores the acquired data in temporary storage and performs data cleaning and preprocessing, such as standardizing data formats, filling in missing values, and removing unnecessary data. The cleaned data is then integrated using a common key.

[0581] Data Visualization

[0582] The server passes the integrated data to a visualization tool, which visualizes it in the format specified by the user (e.g., graph, table, dashboard). For example, a sales progress graph can be generated using libraries such as Matplotlib or Tableau.

[0583] The terminal receives the visualized data and displays it to the user, who can access it through an interface that allows filtering and customization of the visualization results.

[0584] Providing an interface

[0585] The server provides a user-friendly interface that allows users to easily input data, enter prompts, view results, filter and customize, etc.

[0586] Users can quickly analyze data using the provided interface and use it to make business decisions and develop strategies. For example, they can adjust their sales strategies based on the results of sales data analysis.

[0587] As described above, the present invention automates the process from data collection to visualization, and provides an environment in which users can perform data analysis without needing specialized knowledge.

[0588] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0589] Step 1:

[0590] Initial Settings and Data Source Registration

[0591] The server saves the API endpoint, authentication key, and database connection string in a configuration file to establish connections to market data, public information, and internal data. The input is the API endpoint URL, authentication key, and database connection information, and the connection configuration is based on this. The output is the connection configuration information saved.

[0592] The user enters the necessary API keys and database credentials into a dedicated interface and sends them to the server. The input is the user-provided credentials sent to the server. The output is the credentials securely stored on the server.

[0593] Step 2:

[0594] Prompt typing

[0595] The terminal provides an interface for the user to input a prompt sentence in natural language. For example, the user might input "Sales analysis using market data and internal sales data for 2023." The input is the prompt sentence entered by the user in natural language.

[0596] The terminal parses the input prompt sentence using a natural language processing (NLP) module. As a result of the parsing, a typed prompt is generated and sent to the server. The output is the typed prompt sentence.

[0597] Step 3:

[0598] Data collection

[0599] The server initiates the data collection task based on the typed prompt, sending an HTTP request to the market data API to retrieve market data for the year 2023. The input is the typed prompt, and the API request is generated based on this information. The output is the retrieved market data stored in temporary storage.

[0600] The server runs SQL queries against the internal database to retrieve sales data for the year 2023. Similarly, the input is a typed prompt, and an SQL query is generated based on this information. As an output, the retrieved internal sales data is saved in temporary storage.

[0601] Step 4:

[0602] Data cleaning and preprocessing

[0603] The server temporarily stores the acquired market data and internal sales data, and then cleans and preprocesses the data. Specifically, it standardizes formats, fills in missing values, and deletes unnecessary data. The input is the acquired market data and internal sales data, and the output is cleaned, integrated data.

[0604] Step 5:

[0605] Data integration

[0606] The server aggregates the cleaned data, joining the data using a common key (e.g., date or product ID) to generate the aggregated data. The inputs are the cleaned market data and internal company data, and the output is the aggregated data.

[0607] Step 6:

[0608] Data Visualization

[0609] The server passes the integrated data to a visualization tool, which visualizes it in the format specified by the user (e.g., graph, table, dashboard, etc.). For example, a graph of sales trends is generated using a library such as Matplotlib or Tableau. The input is the integrated data, and the visualized data is generated as the output.

[0610] The terminal receives the visualized data and displays it to the user, who uses an interface to filter and customize the displayed graphs and tables. The input is the visualized data, and the output is the user-customized visualization.

[0611] Step 7:

[0612] Providing an interface

[0613] The server provides a user-friendly interface, allowing users to easily input data, enter prompts, check results, filter, customize, etc. The input is the information for the interface design, and the output is the operation screen that the user can use.

[0614] Users can use the provided interface to quickly analyze data and use it for business decisions and strategic planning. For example, they can adjust their sales strategy based on the results of sales data analysis. The input is the user's operations, and the output is specific analysis results.

[0615] (Application example 1)

[0616] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0617] In conventional factory operations, a wide variety of data is generated from various sources, but it has been difficult to efficiently collect, integrate, and visualize this data. It has been particularly difficult to monitor and analyze important performance indicators, such as the number of abnormalities and product pass rates, in real time. As a result, delays have occurred in maintenance and optimization of production lines, hindering efficient operations. To solve these issues, a system is needed that can quickly collect and analyze real-time data generated within the factory and visualize the results in an easily understandable format.

[0618] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0619] In this invention, the server includes: [means for collecting market data, public information, and internal company data;] [means for analyzing user-input natural language prompts to generate typed prompts;] [means for executing queries on data sources based on the typed prompts to obtain required data;] [means for collecting real-time data from sensors and machine operating status APIs within the factory; and [means for analyzing the collected data and visualizing production line performance indicators such as the number of abnormalities and the pass rate.] This enables efficient collection, integration, and cleaning of various data generated within the factory, and also enables real-time visualization of important performance indicators such as the number of abnormalities and the pass rate, enabling rapid response to optimize factory operations.

[0620] "Market Data" refers to various statistical information, trends, and other data related to the market.

[0621] "Public information" is information that is publicly available, including company financial reports, news articles, industry reports, etc.

[0622] "Internal data" refers to various data generated within a company, including business reports, sales data, product inspection data, and the like.

[0623] A "natural language prompt" refers to an instruction or question that a user enters in natural language.

[0624] A "typed prompt" refers to a prompt that is parsed from a natural language prompt and converted into a format that is easy for the system to understand.

[0625] "Data source" refers to the location or system from which data is obtained. Examples include APIs, databases, sensors, etc.

[0626] A "query" is an instruction to a database or API to retrieve specific data.

[0627] "Data integration" refers to the process of combining data from different sources into a single, consistent format.

[0628] "Cleaning" refers to the preprocessing of data to detect and correct incomplete, inaccurate, or unnecessary data.

[0629] "Preprocessing" refers to various preparatory tasks carried out before data analysis or visualization, including data normalization and conversion.

[0630] "Visualization" refers to the process of representing data in a visual format, such as a graph or chart, to allow users to intuitively understand the information.

[0631] "Filtering" refers to the process of specifying specific conditions when displaying or analyzing data and displaying only the necessary data.

[0632] "Customization" refers to tailoring the appearance and behavior of a system to meet the needs and requirements of the user.

[0633] "Interface" refers to the screen and input means that allow the user to operate the system.

[0634] A "sensor" refers to a device that measures physical environmental variables (such as temperature, humidity, and vibration) and outputs the data as an electronic signal.

[0635] "Machine operating status API" refers to a program interface for obtaining machine operating status and performance data.

[0636] "Real-time data" refers to the latest data available at the current time.

[0637] "Performance indicators" are standards for measuring and evaluating the efficiency and effectiveness of a system or process.

[0638] The "number of abnormal events" indicates the frequency of abnormal events occurring within a certain period of time.

[0639] The "pass rate" indicates the percentage of products that meet the standards and are judged to have passed the inspection.

[0640] The present invention provides a system for quickly collecting, analyzing, and integrating real-time data generated in a factory, and visualizing the data. Specific embodiments for carrying out the present invention will be described below.

[0641] System Overview

[0642] The system mainly consists of a server, a terminal, and a user. The server collects, analyzes, integrates, and visualizes data, while the terminal provides an interface for users to enter prompts and check the visualization results. By entering prompts, users can easily automate the entire process from data collection to visualization.

[0643] Hardware and software used

[0644] Hardware:

[0645] Sensors: Devices that measure physical environmental variables (temperature, vibration, pressure, etc.) within the factory

[0646] Server: Collects, analyzes, stores, and visualizes data

[0647] Terminal: A device where users can enter prompts and view visualization results.

[0648] Machine Operation Status API: Interface for obtaining the machine operation status

[0649] software:

[0650] Natural Language Processing (NLP) libraries: spaCy, NLTK, etc.

[0651] Data collection and cleaning: Pandas library (Python)

[0652] Database connection: SQLAlchemy (Python)

[0653] Visualization libraries: Matplotlib, Seaborn (Python)

[0654] Process Details

[0655] 1. Data collection and integration

[0656] The server collects real-time data from sensors installed in the factory and machine health APIs, such as temperature data from sensors and machine uptime information, while user-entered prompts are parsed and converted into typed prompts using natural language processing.

[0657] 2. Data cleaning and preprocessing

[0658] The collected data is stored in temporary storage and undergoes cleaning and preprocessing, such as imputing missing values, standardizing data formats, and removing inaccurate data.

[0659] 3. Data Visualization

[0660] The preprocessed data is integrated and visualized in a specified format (graph, chart, etc.). For example, data on the number of abnormalities or product pass rates can be visually displayed over time.

[0661] 4. Providing a user interface

[0662] The terminal displays the visualization results to the user, providing a filterable and customizable interface through which the user can quickly and efficiently analyze the data.

[0663] Prompt Sentence Examples

[0664] Below are some examples of prompt sentences.

[0665] "Analyze this month's product pass rate and number of machine abnormalities"

[0666] "Sales analysis using 2023 market data and internal sales data"

[0667] "What time of day this month are you seeing the most anomalies?"

[0668] The system of the present invention improves factory operations, enabling real-time data monitoring and analysis, which allows for quicker maintenance and production line adjustments and more efficient factory operations.

[0669] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0670] Step 1:

[0671] The server collects real-time data from sensors in the factory and machine operation status APIs. This includes temperature data, vibration data, and operating time information. The data sent from the sensors is accumulated on the server via HTTP requests. The server stores the acquired data in temporary storage.

[0672] Input: Real-time data from sensors and machine health APIs

[0673] Output: Raw data stored in temporary storage

[0674] Step 2:

[0675] The user inputs a natural language prompt through the terminal, for example, "Analyze this month's product pass rate and machine abnormality count." The terminal receives this natural language prompt, analyzes it using a natural language processing (NLP) module, and generates a typed prompt.

[0676] Input: User's natural language prompt

[0677] Output: Typed prompt

[0678] Step 3:

[0679] The server receives typed prompts and runs queries against the collected data, for example using a database query to get temperature data or anomaly counts for a specific time period, and the results of this query are stored in temporary storage in the form of a data frame.

[0680] Input: Typed prompts, raw data stored in temporary storage

[0681] Output: The specific data retrieved by the query

[0682] Step 4:

[0683] The server cleans and preprocesses the acquired data, including imputing missing values, removing outliers, and standardizing data formats. It uses the Pandas library to clean the data table and convert it into a format suitable for analysis.

[0684] Input: The specific data retrieved by the query

[0685] Output: Cleaned and preprocessed data

[0686] Step 5:

[0687] The server analyzes the preprocessed data and visualizes it in the specified format. For example, it can display the number of abnormalities or product pass rates as a line graph or bar graph. It generates graphs using the Matplotlib or Seaborn libraries and saves them as image files.

[0688] Input: Cleaned and preprocessed data

[0689] Output: Visualized graph or chart

[0690] Step 6:

[0691] The server then sends the generated visualization data to the terminal, which receives it and displays it graphically to the user, who can then use the provided interface to view the results and filter and customize them as needed.

[0692] Input: A visualized graph or chart

[0693] Output: A graph or chart displayed on the user's terminal

[0694] Step 7:

[0695] Based on the displayed data, users can optimize their production lines and create maintenance plans. For example, they can identify time periods when there are many abnormalities and concentrate maintenance work during those periods. The results of user operations are also saved on the server and reflected in future data analysis.

[0696] Input: User interactions and results

[0697] Output: Insights for maintenance planning and production line optimization

[0698] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0699] MODE FOR CARRYING OUT THE INVENTION

[0700] This invention provides a system that enables businesspeople to integrate market data, public information, and internal company data, and customizes data analysis by recognizing user emotions. The system is primarily composed of a server, terminals, and users, and is combined with an emotion engine. This makes it possible to dynamically adjust the data visualization method and presentation content according to the user's emotions.

[0701] System Overview

[0702] The system consists of the following main components:

[0703] 1. Server: The central processing unit responsible for data collection, analysis, integration, visualization, and emotion recognition.

[0704] 2. Terminal: A device that provides an interface for users to enter prompts and view visualization results.

[0705] 3. User: The person who uses the system to perform data analysis.

[0706] 4. Emotion Engine: A module that recognizes emotions from user input and behavior and customizes data analysis and visualization.

[0707] What the program does

[0708] 1. Initial settings and data source registration

[0709] Server: The server configures the connection to the market data API and the internal database, storing the API endpoint, authentication key, and database connection string in a configuration file, which then authenticates against the API and database.

[0710] User: The user provides credentials (API keys or database credentials) to the server, which stores them securely using security features.

[0711] 2. Typing prompts

[0712] Terminal: The user types a natural language prompt into the terminal, for example, "Sales analysis using market data and internal sales data for 2023."

[0713] Terminal: The terminal receives the natural language prompt, parses and formats the prompt using the NLP module, and sends the formatted prompt to the server.

[0714] 3. Data collection and processing

[0715] Server: The server initiates the data retrieval task based on a typed prompt. It sends an HTTP request to the market data API to retrieve market data for the specified period. At the same time, the server executes an SQL query against the internal database to retrieve sales data for the specified period.

[0716] Server: The acquired data is stored in temporary storage and undergoes cleaning and preprocessing, such as imputing missing values, standardizing data formats, and removing outliers.

[0717] Server: The cleaned data is integrated using a common key, and the integrated data is centralized and made suitable for analysis.

[0718] 4. Data Visualization

[0719] Server: Passes the integrated data to the visualization tool and visualizes it in the format specified by the user. It generates graphs and tables using visualization libraries such as Matplotlib and Tableau.

[0720] Server: Sends the visualized data to the terminal and displays it to the user, who can modify and customize the visualization using pull-down menus and filtering options.

[0721] 5. Emotion Engine

[0722] Server: Monitors user input and behavior and uses an emotion engine to recognize user emotions, for example, inferring stress or satisfaction from the user's typing speed and keystroke patterns.

[0723] Server: Dynamically adjusts the visualization and presentation of data based on the perceived emotion. If the user is nervous, it might provide a simpler graph or add a more detailed explanation.

[0724] 6. Providing an Interface

[0725] Server: Provides a user-friendly interface that allows users to smoothly input data, complete prompts, view results, filter and customize them, etc.

[0726] Users: Use a customized interface based on emotion recognition to quickly analyze data and make business decisions. For example, if a user is feeling stressed, the system will provide a simplified visualization and guide subsequent operations.

[0727] In this way, the system of the present invention automates the process from data collection to visualization, and further provides data analysis customized according to the user's emotions, thereby providing an environment in which users can efficiently utilize data without the need for specialized knowledge.

[0728] The processing flow will be explained below.

[0729] Step 1:

[0730] Server: The server configures the connection to the market data API and the internal database. Specifically, it stores the API endpoints, authentication keys, and database connection strings in configuration files and establishes the connection to the API and database.

[0731] Step 2:

[0732] User: The user uses a device to send their credentials (API key or database credentials) to the server, which receives them and stores them securely using security features.

[0733] Step 3:

[0734] Terminal: The user types a natural language prompt into the terminal (e.g., "Sales analysis using market data and internal sales data for 2023").

[0735] Step 4:

[0736] Terminal: The terminal receives the input natural language prompt and parses it using a natural language processing (NLP) module, generating a typed prompt as a result, which it then sends to the server:

[0737] {

[0738] "action": "Sales analysis",

[0739] "data_sources": ["Market Data API", "Internal Database"],

[0740] "year": 2023

[0741] }

[0742] Step 5:

[0743] Server: The server receives the typed prompt and sends an HTTP request to the market data API to retrieve market data for the specified period. It also executes an SQL query against the internal database to retrieve sales data for the specified period. For example, it executes the following SQL query:

[0744] sql

[0745] SELECT FROM sales WHERE year = 2023

[0746] Step 6:

[0747] Server: The acquired data is stored in temporary storage. After storage, data cleaning is performed. Specifically, missing values ​​are filled, the data format is standardized, and outliers are removed.

[0748] Step 7:

[0749] Server: The cleaned data is integrated using a common key (date or product ID), and the integrated data is centralized and organized into a form suitable for analysis.

[0750] Step 8:

[0751] Server: Passes the consolidated data to a visualization tool and visualizes it in the format specified by the user. For example, it generates graphs and tables using libraries such as Matplotlib or Tableau.

[0752] Step 9:

[0753] Server: Sends visualized data to the device, where it is displayed in the format selected by the user (bar graph, line graph, dashboard, etc.).

[0754] Step 10:

[0755] Terminal: The terminal displays the visualization results to the user, who can customize the displayed data using pull-down menus and filtering options.

[0756] Step 11:

[0757] Server: Uses an emotion engine to recognize emotions from user input and behavior, for example, inferring stress or satisfaction from a user's typing speed and keystroke patterns.

[0758] Step 12:

[0759] Server: Dynamically adjusts the visualization and presentation of data based on the user's emotions as recognized by the emotion engine. For example, if the user is nervous, provide a simpler graph or add a detailed explanation.

[0760] Step 13:

[0761] Server: Sends customized visualization results to the device, presenting data in a way that is easier for users to understand.

[0762] Step 14:

[0763] Users: Through a customized interface, they can quickly analyze data and make business decisions, filtering and re-prompting as needed. For example, they can adjust marketing strategies for specific products or services based on the results of analyzing sales data.

[0764] In this way, the system of the present invention automates the process from data collection to visualization, and further provides customization according to the user's emotions, thereby providing an environment in which users can efficiently utilize data without the need for specialized knowledge.

[0765] Example 2

[0766] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0767] In conventional data analysis systems, market data, public information, and internal data are collected and analyzed separately, which makes data integration time-consuming and labor-intensive, making efficient data analysis difficult. Furthermore, they lack the functionality to dynamically adjust data analysis taking into account the user's emotions and stress levels, making them less user-friendly.

[0768] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0769] In this invention, the server includes: [means for collecting market data, public information, and internal company data]; [means for analyzing user-input natural language prompts to generate typed prompts]; [means for executing queries against data sources based on the typed prompts to obtain required data]; [means for integrating, cleaning, and preprocessing the acquired data]; [means for visualizing the integrated data in a user-specified format]; [means for providing an interface for users to filter and customize data visualization results]; and [means for monitoring user input and behavior, recognizing emotions, and dynamically adjusting data analysis and visualization.] This makes it possible [to simplify data integration processing and achieve efficient data analysis]. It also makes it possible [to recognize user emotions and customize data analysis in real time, improving operability and providing a stress-free data analysis environment].

[0770] "Market Data" refers to information relating to economic activity and transactions in a particular market.

[0771] "Public information" refers to data and information that is publicly available, including, for example, government statistics, news, and reports.

[0772] "Internal data" refers to data generated within a company, including sales data, customer information, and financial data.

[0773] "User input" refers to all input operations performed by users of the system via their terminals.

[0774] A "natural language prompt" refers to a sentence of instructions or requests that a user enters in everyday language.

[0775] A "typed prompt" refers to a prompt that has been parsed from a natural language prompt and converted into a format that is easy for the system to process.

[0776] "Data source" refers to the origin or source from which data is obtained, including APIs and databases.

[0777] A "query" is a request made to a database or API to search or retrieve data.

[0778] "Cleaning" refers to the processing of data to improve its quality by correcting or removing missing or outlier values.

[0779] "Preprocessing" refers to preparatory work such as data standardization, conversion, and filtering that is carried out before data analysis.

[0780] "Visualization" refers to the process of displaying data in a visual format such as a graph, chart, or table.

[0781] "Filtering" refers to the process of selecting and displaying data based on specific conditions.

[0782] "Customization" refers to the operation of changing the system or the display format of data according to the user's needs.

[0783] "Interface" refers to the input / output means by which a user and a system interact with each other.

[0784] "Emotion recognition" refers to the process of identifying emotions from user input and behavior and having the system respond accordingly.

[0785] MODE FOR CARRYING OUT THE INVENTION

[0786] The present invention provides a system that enables businesspeople to handle market data, public information, and internal company data in an integrated manner, and further customizes data analysis by recognizing user emotions. The system is primarily composed of a server, a terminal, and a user, combined with an emotion recognition engine. Specific embodiments for implementing this system are described in detail below.

[0787] Hardware and Software Configuration

[0788] 1. Server

[0789] Hardware: Dedicated servers with powerful CPUs, large amounts of RAM and storage.

[0790] Software: Software for data collection, analysis, synthesis, visualization, and emotion recognition. Examples include databases (e.g., MySQL, PostgreSQL), programming languages ​​(e.g., Python), data visualization libraries (e.g., Matplotlib, Tableau), natural language processing engines (e.g., spaCy, NLTK), and emotion recognition models.

[0791] 2. Terminal

[0792] Hardware: The input and display devices used by users, such as PCs, tablets, and smartphones.

[0793] Software: Web browsers and applications for providing a user interface. Examples include web browsers (e.g., Google® Chrome, Mozilla Firefox) and user interface frameworks (e.g., React, Vue.js).

[0794] 3. Users

[0795] Business people using systems to analyze data.

[0796] Data processing and calculation process

[0797] The server uses a natural language processing engine to parse the natural language prompts received from the user and generate typed prompts. Based on the typed prompts, the server queries market data APIs and internal databases to retrieve the required data. The retrieved data is stored in temporary storage, after which it is cleaned and pre-processed. Cleaning includes missing value imputation, data format standardization, and outlier removal. Pre-processing includes data standardization, transformation, and filtering.

[0798] The integrated data is visualized using a data visualization library in the form of graphs, charts, tables, etc. The generated visualization results are sent to the terminal, providing an interface for users to filter and customize the data based on specific conditions.

[0799] Additionally, the server monitors user input and behavior and uses an emotion recognition engine to identify the user's emotions. Based on the emotion, the system dynamically adjusts the visualization and presentation of the data. For example, if the user is feeling stressed, the system will provide a simpler graph and add a detailed explanation.

[0800] Specific examples

[0801] Example 1: Sales analysis of market data and internal sales data

[0802] User: Enter "Sales analysis using 2023 market data and internal sales data."

[0803] Terminal: Parses the prompt and sends the typed prompt to the server.

[0804] Server: Collects data from specified data sources, cleans and integrates it, uses a data visualization library to generate a time series graph of sales, and sends it to the terminal.

[0805] Server: Monitors the user's emotions and, if they are nervous, provides a simple graph and pops up a detailed explanation.

[0806] Example 2: Customer Satisfaction Analysis

[0807] User: Type "Create Customer Satisfaction Report for Q4 2022."

[0808] Terminal: Type natural language prompts and send them to the server.

[0809] Server: Acquires data on customer satisfaction (such as survey results), cleans and integrates them, and uses Tableau to generate a heat map of customer satisfaction and send it to the device.

[0810] Server: Adjust the color and detail of the heatmap based on user sentiment.

[0811] In this way, the system of the present invention not only automates the process of collecting, processing, and visualizing the necessary data based on the user's natural language prompts, but also customizes data analysis according to the user's emotions, thereby achieving fast and efficient data analysis.

[0812] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0813] Step 1:

[0814] Initial setup and registration of authentication information

[0815] Server: Store your market data API endpoints, authentication keys, and internal database connection strings in a configuration file, which will connect and authenticate to the API and database.

[0816] User: Enters API key and authentication information on the initial setup screen of the system. The server encrypts and stores this information securely and returns a connection success message.

[0817] Input: API endpoint, authentication key, database connection information

[0818] Output: Connection successful message

[0819] Specific operation: The user enters authentication information into the input form and clicks the submit button. The server sets up the connection based on the entered information and returns a message when the connection is successful.

[0820] Step 2:

[0821] Entering and parsing user prompts

[0822] User: Uses a terminal interface to enter natural language prompts. Example: "Sales analysis using market data and internal sales data for 2023."

[0823] Terminal: Parses the input natural language prompt using an NLP module (e.g. spaCy, NLTK), generates a typed prompt, and sends it to the server.

[0824] Input: Natural language prompt (e.g., "Sales analysis using market data and internal sales data for 2023")

[0825] Output: Typed prompt

[0826] Specific operation: The user enters a prompt, and the device analyzes the natural language prompt to extract key information such as "market data," "2023," "sales data," and "sales analysis," then formats and sends it to the server.

[0827] Step 3:

[0828] Data collection

[0829] Server: Initiates data collection tasks based on typed prompts, sends HTTP requests to the market data API to retrieve market data for the specified period, and simultaneously executes SQL queries against the internal database to retrieve the required sales data.

[0830] Input: Typed prompt

[0831] Output: Collected data (market data, internal sales data)

[0832] Specific operation: The server sends an HTTP GET request to "api.marketdata.com" to retrieve market data for 2023. At the same time, it executes the query "SELECT FROM sales_data WHERE year = 2023" against the internal database to retrieve sales data.

[0833] Step 4:

[0834] Data cleaning and preprocessing

[0835] Server: Stores the acquired data in temporary storage and performs cleaning and preprocessing, such as imputing missing values, standardizing data formats, and detecting and removing outliers.

[0836] Input: Collected data

[0837] Output: Cleaned data

[0838] Specific operation: The server checks the collected data and performs missing value imputation (e.g., replacing "0" or "null" with the appropriate value), formatting standardization (e.g., standardizing date formats), and outlier removal (e.g., removing abnormally high or low values).

[0839] Step 5:

[0840] Data Integration and Transformation

[0841] Server: The cleaned data is consolidated using a common key (e.g., date) and made suitable for analysis.

[0842] Input: Cleaned data

[0843] Output: Integrated data

[0844] What it does: The server joins market data and sales data based on "date" to create a unified data set, enabling more comprehensive analysis.

[0845] Step 6:

[0846] Data Visualization

[0847] Server: Passes the integrated data to a visualization tool (e.g., Matplotlib, Tableau) and visualizes it in the format specified by the user.

[0848] Server: The visualized data is sent to the terminal and displayed to the user, who can filter and customize it.

[0849] Input: Integrated data

[0850] Output: Visualization results (graphs and charts)

[0851] What it does: The server uses Matplotlib to generate a line graph of monthly sales and sends it to the terminal where the user can view the graph. The user can select a specific month from a dropdown menu to view more details.

[0852] Step 7:

[0853] Emotion Recognition and Adaptive Presentation

[0854] Server: Monitors user input and actions and uses an emotion recognition engine to infer emotions. Example: Determine stress levels based on typing speed and keystrokes.

[0855] Server: Dynamically adjusts visualization and presentation content based on the recognized emotion. Example: If the user is nervous, present a simple graph and add a detailed explanation.

[0856] Input: User operation data (typing speed, keystroke patterns)

[0857] Output: Adaptive visualization results

[0858] Specific operation: The server analyzes input speed information and, if it determines that the user is feeling stressed, it provides a simple bar graph that makes subtraction easy to understand and displays a detailed explanation in a pop-up.

[0859] Step 8:

[0860] Providing and customizing the interface

[0861] Server: Provides a user-friendly interface that allows for easy data entry, prompt entry, results review, filtering, and customization.

[0862] Users: Use a customized interface to quickly analyze data and make decisions.

[0863] Input: None (fixed interface)

[0864] Output: User-friendly interface

[0865] What it does: Users upload data on the data dashboard screen, select the appropriate filter options from the interface, and the graphs are updated in real time.

[0866] (Application example 2)

[0867] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0868] Modern logistics centers are swept up in a wide variety of data, and require systems that can efficiently collect, analyze, and visualize it. However, when employees are confronted with vast amounts of data, they can become stressed and fatigued, and there is a lack of ways to present information accordingly. This situation reduces work efficiency and becomes a source of stress in the workplace.

[0869] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0870] In this invention, the server includes: [means for collecting market data, public information, and business data;] [means for analyzing natural language prompts entered by a user to generate typed prompts; and] [means for recognizing a user's emotions using an emotion engine and dynamically adjusting the data visualization method.] This enables [employees to efficiently process massive amounts of data, and furthermore, the emotional information presentation method improves work efficiency and reduces stress].

[0871] "Market data" is a general term for external data related to commercial activities, such as market trends, trading information, and competitive analysis.

[0872] "Public Information" means data or information that is publicly available, including websites and public reports.

[0873] "Business data" refers to data generated within a company related to its daily operations, including sales information, inventory data, customer information, and so on.

[0874] "User" refers to the person who operates the system and performs data analysis.

[0875] "Natural language prompt" refers to a natural language instruction or question that a user enters into a system.

[0876] A "typed prompt" is an instruction that has been parsed from a natural language prompt and converted into a structure that is easy for the system to understand.

[0877] "Data Source" refers to the source from which data is obtained, including market data APIs and internal databases.

[0878] A "query" is a request to retrieve specific information from a database or API.

[0879] "Cleaning" refers to the process of preparing data by removing invalid values, missing values, and outliers before analyzing the data.

[0880] "Preprocessing" refers to the preparation of data before analysis, and includes standardizing formats and scaling.

[0881] "Visualization" refers to the process of displaying acquired and integrated data in a visual format such as a graph or chart.

[0882] "Filtering" is the process of extracting parts of data that meet specific conditions and using them for display or analysis.

[0883] "Customization" refers to adjusting the display content and functions of a system according to the user's requirements and circumstances.

[0884] "Interface" refers to the operation screen and input means used by users when interacting directly with the system.

[0885] The "emotion engine" is a module that analyzes the user's emotional state and dynamically adjusts the system's output and display.

[0886] "Smart glasses" refer to wearable devices that visually display information and allow users to check data in real time by wearing them.

[0887] This invention aims to enable employees working at logistics centers to efficiently collect, analyze, and visualize market data, business data, and real-time inventory information. In addition, by utilizing an emotion engine, it provides a method for presenting information according to the employee's emotional state, thereby improving work efficiency and reducing stress.

[0888] System Configuration

[0889] Hardware Configuration

[0890] Server: The central processing unit that collects, analyzes, consolidates, and visualizes data from market data APIs and operational databases.

[0891] Terminal: A device where a user inputs data and sees results, including smart glasses.

[0892] Smart glasses: Wearable devices that visually present data to the user in real time.

[0893] Software Configuration

[0894] Market Data Collection Module: Collects market data from external APIs.

[0895] Database Query Module: Executes queries against business databases to retrieve required data.

[0896] Data Cleaning and Integration Module: Cleans, pre-processes, and integrates the collected data.

[0897] Emotion engine: Analyzes data collected from the smart glasses' sensors and recognizes the user's emotions.

[0898] Visualization module: Visualizes the integrated data according to the user's emotional state.

[0899] Interface module: Provides an interface that allows users to filter and customize the data visualization results.

[0900] Processing flow

[0901] The server collects data from market data APIs and operational databases, cleans and pre-processes it, then integrates the data and creates visualizations based on the user's natural language prompts. The emotion engine recognizes the user's emotional state from the smart glasses' sensors and adjusts how the data is presented. Through the smart glasses, users can visually view the information.

[0902] Specifically, the user types the following natural language prompt into the terminal:

[0903] I want to conduct sales analysis using market data for 2023 and internal sales data.

[0904] The server analyzes the prompts and collects and analyzes the necessary data. After cleaning and preprocessing, the data is visualized by the emotion engine, taking into account the user's emotional state, and displayed on the smart glasses.

[0905] This allows users to view data in a way that is optimized for their emotional state, enabling them to make quick and efficient business decisions. For example, a user feeling tired will see a simple graph, while a relaxed user will be provided with detailed data.

[0906] This system is a powerful tool for increasing operational efficiency and reducing employee stress in logistics centers.

[0907] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0908] Step 1:

[0909] The server configures connections to market data APIs and business databases. Specifically, it saves the API endpoint, authentication key, and database connection string in a configuration file, and establishes connections to the API and database based on this information. The input is the API endpoint and authentication information, and the output is the connection establishment status.

[0910] Step 2:

[0911] The server receives a natural language prompt provided by the user. Through a terminal, the user inputs a prompt sentence such as: "I would like to perform a sales analysis using market data and business data for 2023." The input is the natural language prompt, and the output is the prompt sentence sent to the server.

[0912] Step 3:

[0913] The terminal parses the natural language prompt it receives and generates a typed prompt. It uses an NLP (Natural Language Processing) module to parse the prompt and convert it into a format that the system can understand. The input is the natural language prompt, and the output is the typed prompt.

[0914] Step 4:

[0915] The server queries the data source based on the typed prompts. It sends an HTTP request to the market data API to retrieve market data for the specified period. At the same time, it executes SQL queries against the operational database to retrieve the required data. The input is the typed prompts, and the output is the retrieved data.

[0916] Step 5:

[0917] The server cleans and preprocesses the acquired data. Specifically, it performs processes such as filling in missing values, standardizing data formats, and removing outliers. The cleaned data is saved in temporary storage. The input is the acquired data, and the output is the cleaned data.

[0918] Step 6:

[0919] The server integrates the cleaned data. It combines the market data and business data using a common key to generate integrated data. The input is the cleaned market data and business data, and the output is the integrated data.

[0920] Step 7:

[0921] The server visualizes the integrated data. It passes the data to the visualization tool in a user-specified format (e.g., graphs, charts, etc.) and displays it in a format that the user can easily understand. The input is the integrated data, and the output is the visualized data.

[0922] Step 8:

[0923] The server uses an emotion engine to recognize the user's emotions. It analyzes data obtained from the smart glasses' sensors (e.g., gaze tracking, voice, body movements, etc.) to understand the user's emotional state. The input is the sensor data, and the output is the recognized emotional state.

[0924] Step 9:

[0925] The server dynamically adjusts the visualization method based on the recognized emotional state. For example, if the user feels tired, it provides a simple graph, while if the user feels relaxed, it displays detailed data. The input is the recognized emotional state, and the output is the adjusted visualization data.

[0926] Step 10:

[0927] The terminal presents the adjusted visualized data to the user through the smart glasses. The user can check the data in real time through the smart glasses display and make business decisions. The input is the adjusted visualized data, and the output is the real-time visual information presented to the user.

[0928] In this way, efficient data visualization and emotion recognition are achieved through the processing and calculation of various input data at each processing step of the system.

[0929] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0930] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0931] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0932] [Third embodiment]

[0933] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0934] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0935] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0936] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0937] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0938] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0939] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0940] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0941] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0942] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0943] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0944] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0945] MODE FOR CARRYING OUT THE INVENTION

[0946] The present invention provides a system that allows business people to quickly and easily access market data, public information, and internal company data, and efficiently manages and visualizes the data in a unified manner. The following describes an embodiment of this system.

[0947] System Overview

[0948] The system mainly consists of a server, a terminal, and a user. The server collects, analyzes, integrates, and visualizes data, while the terminal provides an interface for users to enter prompts and check the visualization results. Users can automatically perform a series of processes from data collection to visualization simply by entering prompts.

[0949] What the program does

[0950] 1. Initial settings and data source registration

[0951] Server: The server stores API endpoints, authentication keys, database connection strings, etc. in configuration files, which establish connections to market data, public information, and internal databases.

[0952] User: The user provides the necessary API keys and database credentials to the server, which receives them and stores them securely using security features.

[0953] 2. Typing prompts

[0954] Terminal: The user types a natural language prompt into the terminal (e.g., "Sales analysis using market data and internal sales data for 2023").

[0955] Terminal: The terminal receives user input and parses the prompt using a natural language processing (NLP) module. The resulting prompt is then generated as a typed prompt and sent to the server.

[0956] 3. Data collection and processing

[0957] Server: The server receives the typed prompt and starts the data retrieval task. It sends an HTTP request to the market data API to retrieve market data for 2023. At the same time, it runs an SQL query against the internal database to retrieve sales data for 2023.

[0958] Server: The acquired data is stored in temporary storage and undergoes cleaning and pre-processing, which includes imputing missing values, standardizing data formats, and removing unnecessary data.

[0959] Server: The cleaned data is aggregated and joined using a common key (e.g., date or product ID).

[0960] 4. Data Visualization

[0961] Server: Passes the integrated data to a visualization tool and visualizes it in the format specified by the user (graph, table, dashboard, etc.). For example, it generates graphs using libraries such as Matplotlib or Tableau.

[0962] Terminal: The visualized data is sent to the terminal and displayed to the user, who can modify and customize the visualization using pull-down menus and filtering functions.

[0963] 5. Providing an Interface

[0964] Server: Provides a user-friendly interface that allows users to smoothly input data, complete prompts, view results, and filter and customize them.

[0965] Users: Using the provided interface, users can quickly analyze data and make management decisions and develop strategies. For example, they can adjust sales strategies based on the results of sales data analysis.

[0966] In this way, the system of the present invention automates the process from data collection to visualization, and provides an environment in which users can perform data analysis without needing specialized knowledge.

[0967] The processing flow will be explained below.

[0968] Step 1:

[0969] Server: The server configures the connection to the market data API and the internal database. Specifically, it stores the API endpoint, authentication key, and database connection string in a configuration file and authenticates against the API and database.

[0970] Step 2:

[0971] User: The user uses a device to send authentication information (API keys, database credentials) to the server, which receives it and stores it securely using security features.

[0972] Step 3:

[0973] Terminal: The user enters a natural language prompt into the terminal, for example, "Sales analysis using market data and internal sales data for 2023."

[0974] Step 4:

[0975] Terminal: The terminal receives the input natural language prompt and uses a natural language processing (NLP) module to parse and format the prompt. As a result of the prompt parsing, a formatted prompt is generated, including the data source to be used, the target time period, etc.

[0976] Step 5:

[0977] Terminal: Sends a typed prompt to the server.

[0978] Step 6:

[0979] Server: The server receives a typed prompt and begins the task of retrieving data by sending an HTTP request to the market data API to retrieve market data for the specified period.

[0980] Step 7:

[0981] Server: At the same time, the server runs a SQL query against the internal database to retrieve sales data for the specified period. For example, it runs the following SQL query:

[0982] sql

[0983] SELECT FROM sales WHERE year = 2023

[0984] Step 8:

[0985] Server: The acquired data is stored in temporary storage. After storage, data cleaning begins. Specifically, it performs tasks such as filling in missing values, standardizing data formats, and removing outliers.

[0986] Step 9:

[0987] Server: The cleaned data is consolidated using a common key (e.g., date or product ID), and the consolidated data is centralized and organized into a form suitable for analysis.

[0988] Step 10:

[0989] Server: Passes the consolidated data to visualization tools, for example, using visualization libraries such as Matplotlib or Tableau to generate graphs and tables.

[0990] Step 11:

[0991] Server: Sends visualized data to the device, where the data is presented in the format selected by the user (bar graph, line graph, dashboard, etc.).

[0992] Step 12:

[0993] Terminal: The terminal displays the visualization results to the user, who can further customize the displayed data using pull-down menus and filtering options.

[0994] Step 13:

[0995] Server: Ensures that data can be filtered, sorted, and prompted for additional input through a user-friendly interface, allowing users to easily explore data details and make quick business decisions based on the analysis results.

[0996] Through the above steps, this system provides an environment in which users can easily and efficiently analyze data and support rapid decision-making.

[0997] Example 1

[0998] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0999] In today's business environment, it is important to quickly and effectively integrate and analyze data from different sources. However, it is not easy to collect large amounts of market data, public information, and internal data, efficiently extract and integrate the necessary information, and visualize it in an easy-to-understand format. This task requires advanced expertise and a lot of time, placing a heavy burden on many businesspeople. In addition, the lack of intuitive ways to manipulate and customize visualized data makes it difficult to utilize analysis results in management decisions.

[1000] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1001] In this invention, the server includes: [means for collecting market data, public information, and internal company data;] [means for analyzing user-input natural language prompts and generating styled prompts using a generative AI model; and] [means for executing queries against data sources based on the styled prompts to obtain required data.] This makes it possible [to efficiently and effectively collect, integrate, analyze, and visualize required data from different data sources].

[1002] "Market data" refers to data that includes market trends, pricing information, supply and demand conditions, etc.

[1003] "Public information" is data that is publicly available and accessible to anyone, such as government statistics or news articles.

[1004] "Internal data" refers to data generated and managed within a company or organization. Examples include sales data, customer information, and inventory data.

[1005] A "natural language prompt" is an instruction or question entered by a user using natural language.

[1006] A "generative AI model" is a model that uses artificial intelligence to analyze prompts, understand their meaning, and generate structured prompts.

[1007] A "typed prompt" is a natural language prompt that has been parsed and converted into a specific format, making it easier for the system to extract the exact information needed.

[1008] A "Data Source" is an external or internal source from which data is obtained, such as an API, a database, or a CSV file.

[1009] A "query" is a query statement used to obtain information from a data source such as a database or API.

[1010] "Data cleaning" is the process of preparing data for analysis by filling in missing values, standardizing formats, and removing unnecessary data.

[1011] "Preprocessing" refers to a series of preparation steps that are performed before data analysis, including data cleaning and format conversion.

[1012] "Visualization" refers to the visual presentation of data in the form of graphs, tables, dashboards, etc.

[1013] An "interface" is a screen or function that allows a user to interact with a system, such as providing a prompt input screen or data filtering functionality.

[1014] MODE FOR CARRYING OUT THE INVENTION

[1015] This invention provides a system that allows business people to quickly and easily access market data, public information, and internal company data, and efficiently manage and visualize them in a centralized manner. In implementing this system, three parties - a server, a terminal, and a user - work together, each of which plays a specific role.

[1016] Hardware and software used

[1017] Hardware: The server uses a cloud infrastructure (e.g., Amazon Web Services, Microsoft Azure, etc.) equipped with a high-performance CPU and large amounts of memory. The terminals are PCs, tablets, smartphones, etc. operated by users.

[1018] Software: The following will be used:

[1019] Natural language processing modules (e.g., spaCy, Transformers)

[1020] API management tools (libraries that execute HTTP requests)

[1021] Database management system (e.g. PostgreSQL)

[1022] Libraries for data cleaning and preprocessing (e.g., Pandas)

[1023] Data visualization tools (e.g., Matplotlib, Tableau)

[1024] Web frameworks (e.g. Django, Flask)

[1025] What the program does

[1026] Initial Settings and Data Source Registration

[1027] The server stores API endpoints, authentication keys, and database connection strings in configuration files that establish connections to market data, public information, and internal company databases.

[1028] Users enter the necessary API keys and database authentication information through a dedicated interface and send it to the server, which then securely stores the received authentication information using its security features.

[1029] Prompt typing

[1030] The terminal provides an interface for users to enter natural language prompts, such as "Sales analysis using market data and internal sales data for 2023."

[1031] The device uses a natural language processing module (NLP) to parse the prompt and generate a typed prompt, which is then sent to the server.

[1032] Data collection and processing

[1033] The server receives the typed prompt and starts the data collection task, making an HTTP request to the market data API to retrieve market data for 2023. At the same time, it runs an SQL query against the internal database to retrieve sales data for 2023.

[1034] The server stores the acquired data in temporary storage and performs data cleaning and preprocessing, such as standardizing data formats, filling in missing values, and removing unnecessary data. The cleaned data is then integrated using a common key.

[1035] Data Visualization

[1036] The server passes the integrated data to a visualization tool, which visualizes it in the format specified by the user (e.g., graph, table, dashboard). For example, a sales progress graph can be generated using libraries such as Matplotlib or Tableau.

[1037] The terminal receives the visualized data and displays it to the user, who can access it through an interface that allows filtering and customization of the visualization results.

[1038] Providing an interface

[1039] The server provides a user-friendly interface that allows users to easily input data, enter prompts, view results, filter and customize, etc.

[1040] Users can quickly analyze data using the provided interface and use it to make business decisions and develop strategies. For example, they can adjust their sales strategies based on the results of sales data analysis.

[1041] As described above, the present invention automates the process from data collection to visualization, and provides an environment in which users can perform data analysis without needing specialized knowledge.

[1042] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1043] Step 1:

[1044] Initial Settings and Data Source Registration

[1045] The server saves the API endpoint, authentication key, and database connection string in a configuration file to establish connections to market data, public information, and internal data. The input is the API endpoint URL, authentication key, and database connection information, and the connection configuration is based on this. The output is the connection configuration information saved.

[1046] The user enters the necessary API keys and database credentials into a dedicated interface and sends them to the server. The input is the user-provided credentials sent to the server. The output is the credentials securely stored on the server.

[1047] Step 2:

[1048] Prompt typing

[1049] The terminal provides an interface for the user to input a prompt sentence in natural language. For example, the user might input "Sales analysis using market data and internal sales data for 2023." The input is the prompt sentence entered by the user in natural language.

[1050] The terminal parses the input prompt sentence using a natural language processing (NLP) module. As a result of the parsing, a typed prompt is generated and sent to the server. The output is the typed prompt sentence.

[1051] Step 3:

[1052] Data collection

[1053] The server initiates the data collection task based on the typed prompt, sending an HTTP request to the market data API to retrieve market data for the year 2023. The input is the typed prompt, and the API request is generated based on this information. The output is the retrieved market data stored in temporary storage.

[1054] The server runs SQL queries against the internal database to retrieve sales data for the year 2023. Similarly, the input is a typed prompt, and an SQL query is generated based on this information. As an output, the retrieved internal sales data is saved in temporary storage.

[1055] Step 4:

[1056] Data cleaning and preprocessing

[1057] The server temporarily stores the acquired market data and internal sales data, and then cleans and preprocesses the data. Specifically, it standardizes formats, fills in missing values, and deletes unnecessary data. The input is the acquired market data and internal sales data, and the output is cleaned, integrated data.

[1058] Step 5:

[1059] Data integration

[1060] The server aggregates the cleaned data, joining the data using a common key (e.g., date or product ID) to generate the aggregated data. The inputs are the cleaned market data and internal company data, and the output is the aggregated data.

[1061] Step 6:

[1062] Data Visualization

[1063] The server passes the integrated data to a visualization tool, which visualizes it in the format specified by the user (e.g., graph, table, dashboard, etc.). For example, a graph of sales trends is generated using a library such as Matplotlib or Tableau. The input is the integrated data, and the visualized data is generated as the output.

[1064] The terminal receives the visualized data and displays it to the user, who uses an interface to filter and customize the displayed graphs and tables. The input is the visualized data, and the output is the user-customized visualization.

[1065] Step 7:

[1066] Providing an interface

[1067] The server provides a user-friendly interface, allowing users to easily input data, enter prompts, check results, filter, customize, etc. The input is the information for the interface design, and the output is the operation screen that the user can use.

[1068] Users can use the provided interface to quickly analyze data and use it for business decisions and strategic planning. For example, they can adjust their sales strategy based on the results of sales data analysis. The input is the user's operations, and the output is specific analysis results.

[1069] (Application example 1)

[1070] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1071] In conventional factory operations, a wide variety of data is generated from various sources, but it has been difficult to efficiently collect, integrate, and visualize this data. It has been particularly difficult to monitor and analyze important performance indicators, such as the number of abnormalities and product pass rates, in real time. As a result, delays have occurred in maintenance and optimization of production lines, hindering efficient operations. To solve these issues, a system is needed that can quickly collect and analyze real-time data generated within the factory and visualize the results in an easily understandable format.

[1072] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1073] In this invention, the server includes: [means for collecting market data, public information, and internal company data;] [means for analyzing user-input natural language prompts to generate typed prompts;] [means for executing queries on data sources based on the typed prompts to obtain required data;] [means for collecting real-time data from sensors and machine operating status APIs within the factory; and [means for analyzing the collected data and visualizing production line performance indicators such as the number of abnormalities and the pass rate.] This enables efficient collection, integration, and cleaning of various data generated within the factory, and also enables real-time visualization of important performance indicators such as the number of abnormalities and the pass rate, enabling rapid response to optimize factory operations.

[1074] "Market Data" refers to various statistical information, trends, and other data related to the market.

[1075] "Public information" is information that is publicly available, including company financial reports, news articles, industry reports, etc.

[1076] "Internal data" refers to various data generated within a company, including business reports, sales data, product inspection data, and the like.

[1077] A "natural language prompt" refers to an instruction or question that a user enters in natural language.

[1078] A "typed prompt" refers to a prompt that is parsed from a natural language prompt and converted into a format that is easy for the system to understand.

[1079] "Data source" refers to the location or system from which data is obtained. Examples include APIs, databases, sensors, etc.

[1080] A "query" is an instruction to a database or API to retrieve specific data.

[1081] "Data integration" refers to the process of combining data from different sources into a single, consistent format.

[1082] "Cleaning" refers to the preprocessing of data to detect and correct incomplete, inaccurate, or unnecessary data.

[1083] "Preprocessing" refers to various preparatory tasks carried out before data analysis or visualization, including data normalization and conversion.

[1084] "Visualization" refers to the process of representing data in a visual format, such as a graph or chart, to allow users to intuitively understand the information.

[1085] "Filtering" refers to the process of specifying specific conditions when displaying or analyzing data and displaying only the necessary data.

[1086] "Customization" refers to tailoring the appearance and behavior of a system to meet the needs and requirements of the user.

[1087] "Interface" refers to the screen and input means that allow the user to operate the system.

[1088] A "sensor" refers to a device that measures physical environmental variables (such as temperature, humidity, and vibration) and outputs the data as an electronic signal.

[1089] "Machine operating status API" refers to a program interface for obtaining machine operating status and performance data.

[1090] "Real-time data" refers to the latest data available at the current time.

[1091] "Performance indicators" are standards for measuring and evaluating the efficiency and effectiveness of a system or process.

[1092] The "number of abnormal events" indicates the frequency of abnormal events occurring within a certain period of time.

[1093] The "pass rate" indicates the percentage of products that meet the standards and are judged to have passed the inspection.

[1094] The present invention provides a system for quickly collecting, analyzing, and integrating real-time data generated in a factory, and visualizing the data. Specific embodiments for carrying out the present invention will be described below.

[1095] System Overview

[1096] The system mainly consists of a server, a terminal, and a user. The server collects, analyzes, integrates, and visualizes data, while the terminal provides an interface for users to enter prompts and check the visualization results. By entering prompts, users can easily automate the entire process from data collection to visualization.

[1097] Hardware and software used

[1098] Hardware:

[1099] Sensors: Devices that measure physical environmental variables (temperature, vibration, pressure, etc.) within the factory

[1100] Server: Collects, analyzes, stores, and visualizes data

[1101] Terminal: A device where users can enter prompts and view visualization results.

[1102] Machine Operation Status API: Interface for obtaining the machine operation status

[1103] software:

[1104] Natural Language Processing (NLP) libraries: spaCy, NLTK, etc.

[1105] Data collection and cleaning: Pandas library (Python)

[1106] Database connection: SQLAlchemy (Python)

[1107] Visualization libraries: Matplotlib, Seaborn (Python)

[1108] Process Details

[1109] 1. Data collection and integration

[1110] The server collects real-time data from sensors installed in the factory and machine health APIs, such as temperature data from sensors and machine uptime information, while user-entered prompts are parsed and converted into typed prompts using natural language processing.

[1111] 2. Data cleaning and preprocessing

[1112] The collected data is stored in temporary storage and undergoes cleaning and preprocessing, such as imputing missing values, standardizing data formats, and removing inaccurate data.

[1113] 3. Data Visualization

[1114] The preprocessed data is integrated and visualized in a specified format (graph, chart, etc.). For example, data on the number of abnormalities or product pass rates can be visually displayed over time.

[1115] 4. Providing a user interface

[1116] The terminal displays the visualization results to the user, providing a filterable and customizable interface through which the user can quickly and efficiently analyze the data.

[1117] Prompt Sentence Examples

[1118] Below are some examples of prompt sentences.

[1119] "Analyze this month's product pass rate and number of machine abnormalities"

[1120] "Sales analysis using 2023 market data and internal sales data"

[1121] "What time of day this month are you seeing the most anomalies?"

[1122] The system of the present invention improves factory operations, enabling real-time data monitoring and analysis, which allows for quicker maintenance and production line adjustments and more efficient factory operations.

[1123] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1124] Step 1:

[1125] The server collects real-time data from sensors in the factory and machine operation status APIs. This includes temperature data, vibration data, and operating time information. The data sent from the sensors is accumulated on the server via HTTP requests. The server stores the acquired data in temporary storage.

[1126] Input: Real-time data from sensors and machine health APIs

[1127] Output: Raw data stored in temporary storage

[1128] Step 2:

[1129] The user inputs a natural language prompt through the terminal, for example, "Analyze this month's product pass rate and machine abnormality count." The terminal receives this natural language prompt, analyzes it using a natural language processing (NLP) module, and generates a typed prompt.

[1130] Input: User's natural language prompt

[1131] Output: Typed prompt

[1132] Step 3:

[1133] The server receives typed prompts and runs queries against the collected data, for example using a database query to get temperature data or anomaly counts for a specific time period, and the results of this query are stored in temporary storage in the form of a data frame.

[1134] Input: Typed prompts, raw data stored in temporary storage

[1135] Output: The specific data retrieved by the query

[1136] Step 4:

[1137] The server cleans and preprocesses the acquired data, including imputing missing values, removing outliers, and standardizing data formats. It uses the Pandas library to clean the data table and convert it into a format suitable for analysis.

[1138] Input: The specific data retrieved by the query

[1139] Output: Cleaned and preprocessed data

[1140] Step 5:

[1141] The server analyzes the preprocessed data and visualizes it in the specified format. For example, it can display the number of abnormalities or product pass rates as a line graph or bar graph. It generates graphs using the Matplotlib or Seaborn libraries and saves them as image files.

[1142] Input: Cleaned and preprocessed data

[1143] Output: Visualized graph or chart

[1144] Step 6:

[1145] The server then sends the generated visualization data to the terminal, which receives it and displays it graphically to the user, who can then use the provided interface to view the results and filter and customize them as needed.

[1146] Input: A visualized graph or chart

[1147] Output: A graph or chart displayed on the user's terminal

[1148] Step 7:

[1149] Based on the displayed data, users can optimize their production lines and create maintenance plans. For example, they can identify time periods when there are many abnormalities and concentrate maintenance work during those periods. The results of user operations are also saved on the server and reflected in future data analysis.

[1150] Input: User interactions and results

[1151] Output: Insights for maintenance planning and production line optimization

[1152] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1153] MODE FOR CARRYING OUT THE INVENTION

[1154] This invention provides a system that enables businesspeople to integrate market data, public information, and internal company data, and customizes data analysis by recognizing user emotions. The system is primarily composed of a server, terminals, and users, and is combined with an emotion engine. This makes it possible to dynamically adjust the data visualization method and presentation content according to the user's emotions.

[1155] System Overview

[1156] The system consists of the following main components:

[1157] 1. Server: The central processing unit responsible for data collection, analysis, integration, visualization, and emotion recognition.

[1158] 2. Terminal: A device that provides an interface for users to enter prompts and view visualization results.

[1159] 3. User: The person who uses the system to perform data analysis.

[1160] 4. Emotion Engine: A module that recognizes emotions from user input and behavior and customizes data analysis and visualization.

[1161] What the program does

[1162] 1. Initial settings and data source registration

[1163] Server: The server configures the connection to the market data API and the internal database, storing the API endpoint, authentication key, and database connection string in a configuration file, which then authenticates against the API and database.

[1164] User: The user provides credentials (API keys or database credentials) to the server, which stores them securely using security features.

[1165] 2. Typing prompts

[1166] Terminal: The user types a natural language prompt into the terminal, for example, "Sales analysis using market data and internal sales data for 2023."

[1167] Terminal: The terminal receives the natural language prompt, parses and formats the prompt using the NLP module, and sends the formatted prompt to the server.

[1168] 3. Data collection and processing

[1169] Server: The server initiates the data retrieval task based on a typed prompt. It sends an HTTP request to the market data API to retrieve market data for the specified period. At the same time, the server executes an SQL query against the internal database to retrieve sales data for the specified period.

[1170] Server: The acquired data is stored in temporary storage and undergoes cleaning and preprocessing, such as imputing missing values, standardizing data formats, and removing outliers.

[1171] Server: The cleaned data is integrated using a common key, and the integrated data is centralized and made suitable for analysis.

[1172] 4. Data Visualization

[1173] Server: Passes the integrated data to the visualization tool and visualizes it in the format specified by the user. It generates graphs and tables using visualization libraries such as Matplotlib and Tableau.

[1174] Server: Sends the visualized data to the terminal and displays it to the user, who can modify and customize the visualization using pull-down menus and filtering options.

[1175] 5. Emotion Engine

[1176] Server: Monitors user input and behavior and uses an emotion engine to recognize user emotions, for example, inferring stress or satisfaction from the user's typing speed and keystroke patterns.

[1177] Server: Dynamically adjusts the visualization and presentation of data based on the perceived emotion. If the user is nervous, it might provide a simpler graph or add a more detailed explanation.

[1178] 6. Providing an Interface

[1179] Server: Provides a user-friendly interface that allows users to smoothly input data, complete prompts, view results, filter and customize them, etc.

[1180] Users: Use a customized interface based on emotion recognition to quickly analyze data and make business decisions. For example, if a user is feeling stressed, the system will provide a simplified visualization and guide subsequent operations.

[1181] In this way, the system of the present invention automates the process from data collection to visualization, and further provides data analysis customized according to the user's emotions, thereby providing an environment in which users can efficiently utilize data without the need for specialized knowledge.

[1182] The processing flow will be explained below.

[1183] Step 1:

[1184] Server: The server configures the connection to the market data API and the internal database. Specifically, it stores the API endpoints, authentication keys, and database connection strings in configuration files and establishes the connection to the API and database.

[1185] Step 2:

[1186] User: The user uses a device to send their credentials (API key or database credentials) to the server, which receives them and stores them securely using security features.

[1187] Step 3:

[1188] Terminal: The user types a natural language prompt into the terminal (e.g., "Sales analysis using market data and internal sales data for 2023").

[1189] Step 4:

[1190] Terminal: The terminal receives the input natural language prompt and parses it using a natural language processing (NLP) module, generating a typed prompt as a result, which it then sends to the server:

[1191] {

[1192] "action": "Sales analysis",

[1193] "data_sources": ["Market Data API", "Internal Database"],

[1194] "year": 2023

[1195] }

[1196] Step 5:

[1197] Server: The server receives the typed prompt and sends an HTTP request to the market data API to retrieve market data for the specified period. It also executes an SQL query against the internal database to retrieve sales data for the specified period. For example, it executes the following SQL query:

[1198] sql

[1199] SELECT FROM sales WHERE year = 2023

[1200] Step 6:

[1201] Server: The acquired data is stored in temporary storage. After storage, data cleaning is performed. Specifically, missing values ​​are filled, the data format is standardized, and outliers are removed.

[1202] Step 7:

[1203] Server: The cleaned data is integrated using a common key (date or product ID), and the integrated data is centralized and organized into a form suitable for analysis.

[1204] Step 8:

[1205] Server: Passes the consolidated data to a visualization tool and visualizes it in the format specified by the user. For example, it generates graphs and tables using libraries such as Matplotlib or Tableau.

[1206] Step 9:

[1207] Server: Sends visualized data to the device, where it is displayed in the format selected by the user (bar graph, line graph, dashboard, etc.).

[1208] Step 10:

[1209] Terminal: The terminal displays the visualization results to the user, who can customize the displayed data using pull-down menus and filtering options.

[1210] Step 11:

[1211] Server: Uses an emotion engine to recognize emotions from user input and behavior, for example, inferring stress or satisfaction from a user's typing speed and keystroke patterns.

[1212] Step 12:

[1213] Server: Dynamically adjusts the visualization and presentation of data based on the user's emotions as recognized by the emotion engine. For example, if the user is nervous, provide a simpler graph or add a detailed explanation.

[1214] Step 13:

[1215] Server: Sends customized visualization results to the device, presenting data in a way that is easier for users to understand.

[1216] Step 14:

[1217] Users: Through a customized interface, they can quickly analyze data and make business decisions, filtering and re-prompting as needed. For example, they can adjust marketing strategies for specific products or services based on the results of analyzing sales data.

[1218] In this way, the system of the present invention automates the process from data collection to visualization, and further provides customization according to the user's emotions, thereby providing an environment in which users can efficiently utilize data without the need for specialized knowledge.

[1219] Example 2

[1220] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1221] In conventional data analysis systems, market data, public information, and internal data are collected and analyzed separately, which makes data integration time-consuming and labor-intensive, making efficient data analysis difficult. Furthermore, they lack the functionality to dynamically adjust data analysis taking into account the user's emotions and stress levels, making them less user-friendly.

[1222] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1223] In this invention, the server includes: [means for collecting market data, public information, and internal company data]; [means for analyzing user-input natural language prompts to generate typed prompts]; [means for executing queries against data sources based on the typed prompts to obtain required data]; [means for integrating, cleaning, and preprocessing the acquired data]; [means for visualizing the integrated data in a user-specified format]; [means for providing an interface for users to filter and customize data visualization results]; and [means for monitoring user input and behavior, recognizing emotions, and dynamically adjusting data analysis and visualization.] This makes it possible [to simplify data integration processing and achieve efficient data analysis]. It also makes it possible [to recognize user emotions and customize data analysis in real time, improving operability and providing a stress-free data analysis environment].

[1224] "Market Data" refers to information relating to economic activity and transactions in a particular market.

[1225] "Public information" refers to data and information that is publicly available, including, for example, government statistics, news, and reports.

[1226] "Internal data" refers to data generated within a company, including sales data, customer information, and financial data.

[1227] "User input" refers to all input operations performed by users of the system via their terminals.

[1228] A "natural language prompt" refers to a sentence of instructions or requests that a user enters in everyday language.

[1229] A "typed prompt" refers to a prompt that has been parsed from a natural language prompt and converted into a format that is easy for the system to process.

[1230] "Data source" refers to the origin or source from which data is obtained, including APIs and databases.

[1231] A "query" is a request made to a database or API to search or retrieve data.

[1232] "Cleaning" refers to the processing of data to improve its quality by correcting or removing missing or outlier values.

[1233] "Preprocessing" refers to preparatory work such as data standardization, conversion, and filtering that is carried out before data analysis.

[1234] "Visualization" refers to the process of displaying data in a visual format such as a graph, chart, or table.

[1235] "Filtering" refers to the process of selecting and displaying data based on specific conditions.

[1236] "Customization" refers to the operation of changing the system or the display format of data according to the user's needs.

[1237] "Interface" refers to the input / output means by which a user and a system interact with each other.

[1238] "Emotion recognition" refers to the process of identifying emotions from user input and behavior and having the system respond accordingly.

[1239] MODE FOR CARRYING OUT THE INVENTION

[1240] The present invention provides a system that enables businesspeople to handle market data, public information, and internal company data in an integrated manner, and further customizes data analysis by recognizing user emotions. The system is primarily composed of a server, a terminal, and a user, combined with an emotion recognition engine. Specific embodiments for implementing this system are described in detail below.

[1241] Hardware and Software Configuration

[1242] 1. Server

[1243] Hardware: Dedicated servers with powerful CPUs, large amounts of RAM and storage.

[1244] Software: Software for data collection, analysis, synthesis, visualization, and emotion recognition. Examples include databases (e.g., MySQL, PostgreSQL), programming languages ​​(e.g., Python), data visualization libraries (e.g., Matplotlib, Tableau), natural language processing engines (e.g., spaCy, NLTK), and emotion recognition models.

[1245] 2. Terminal

[1246] Hardware: The input and display devices used by users, such as PCs, tablets, and smartphones.

[1247] Software: Web browsers and applications used to provide a user interface. Examples include web browsers (e.g., Google Chrome, Mozilla Firefox) and user interface frameworks (e.g., React, Vue.js).

[1248] 3. Users

[1249] Business people using systems to analyze data.

[1250] Data processing and calculation process

[1251] The server uses a natural language processing engine to parse the natural language prompts received from the user and generate typed prompts. Based on the typed prompts, the server queries market data APIs and internal databases to retrieve the required data. The retrieved data is stored in temporary storage, after which it is cleaned and pre-processed. Cleaning includes missing value imputation, data format standardization, and outlier removal. Pre-processing includes data standardization, transformation, and filtering.

[1252] The integrated data is visualized using a data visualization library in the form of graphs, charts, tables, etc. The generated visualization results are sent to the terminal, providing an interface for users to filter and customize the data based on specific conditions.

[1253] Additionally, the server monitors user input and behavior and uses an emotion recognition engine to identify the user's emotions. Based on the emotion, the system dynamically adjusts the visualization and presentation of the data. For example, if the user is feeling stressed, the system will provide a simpler graph and add a detailed explanation.

[1254] Specific examples

[1255] Example 1: Sales analysis of market data and internal sales data

[1256] User: Enter "Sales analysis using 2023 market data and internal sales data."

[1257] Terminal: Parses the prompt and sends the typed prompt to the server.

[1258] Server: Collects data from specified data sources, cleans and integrates it, uses a data visualization library to generate a time series graph of sales, and sends it to the terminal.

[1259] Server: Monitors the user's emotions and, if they are nervous, provides a simple graph and pops up a detailed explanation.

[1260] Example 2: Customer Satisfaction Analysis

[1261] User: Type "Create Customer Satisfaction Report for Q4 2022."

[1262] Terminal: Type natural language prompts and send them to the server.

[1263] Server: Acquires data on customer satisfaction (such as survey results), cleans and integrates them, and uses Tableau to generate a heat map of customer satisfaction and send it to the device.

[1264] Server: Adjust the color and detail of the heatmap based on user sentiment.

[1265] In this way, the system of the present invention not only automates the process of collecting, processing, and visualizing the necessary data based on the user's natural language prompts, but also customizes data analysis according to the user's emotions, thereby achieving fast and efficient data analysis.

[1266] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1267] Step 1:

[1268] Initial setup and registration of authentication information

[1269] Server: Store your market data API endpoints, authentication keys, and internal database connection strings in a configuration file, which will connect and authenticate to the API and database.

[1270] User: Enters API key and authentication information on the initial setup screen of the system. The server encrypts and stores this information securely and returns a connection success message.

[1271] Input: API endpoint, authentication key, database connection information

[1272] Output: Connection successful message

[1273] Specific operation: The user enters authentication information into the input form and clicks the submit button. The server sets up the connection based on the entered information and returns a message when the connection is successful.

[1274] Step 2:

[1275] Entering and parsing user prompts

[1276] User: Uses a terminal interface to enter natural language prompts. Example: "Sales analysis using market data and internal sales data for 2023."

[1277] Terminal: Parses the input natural language prompt using an NLP module (e.g. spaCy, NLTK), generates a typed prompt, and sends it to the server.

[1278] Input: Natural language prompt (e.g., "Sales analysis using market data and internal sales data for 2023")

[1279] Output: Typed prompt

[1280] Specific operation: The user enters a prompt, and the device analyzes the natural language prompt to extract key information such as "market data," "2023," "sales data," and "sales analysis," then formats and sends it to the server.

[1281] Step 3:

[1282] Data collection

[1283] Server: Initiates data collection tasks based on typed prompts, sends HTTP requests to the market data API to retrieve market data for the specified period, and simultaneously executes SQL queries against the internal database to retrieve the required sales data.

[1284] Input: Typed prompt

[1285] Output: Collected data (market data, internal sales data)

[1286] Specific operation: The server sends an HTTP GET request to "api.marketdata.com" to retrieve market data for 2023. At the same time, it executes the query "SELECT FROM sales_data WHERE year = 2023" against the internal database to retrieve sales data.

[1287] Step 4:

[1288] Data cleaning and preprocessing

[1289] Server: Stores the acquired data in temporary storage and performs cleaning and preprocessing, such as imputing missing values, standardizing data formats, and detecting and removing outliers.

[1290] Input: Collected data

[1291] Output: Cleaned data

[1292] Specific operation: The server checks the collected data and performs missing value imputation (e.g., replacing "0" or "null" with the appropriate value), formatting standardization (e.g., standardizing date formats), and outlier removal (e.g., removing abnormally high or low values).

[1293] Step 5:

[1294] Data Integration and Transformation

[1295] Server: The cleaned data is consolidated using a common key (e.g., date) and made suitable for analysis.

[1296] Input: Cleaned data

[1297] Output: Integrated data

[1298] What it does: The server joins market data and sales data based on "date" to create a unified data set, enabling more comprehensive analysis.

[1299] Step 6:

[1300] Data Visualization

[1301] Server: Passes the integrated data to a visualization tool (e.g., Matplotlib, Tableau) and visualizes it in the format specified by the user.

[1302] Server: The visualized data is sent to the terminal and displayed to the user, who can filter and customize it.

[1303] Input: Integrated data

[1304] Output: Visualization results (graphs and charts)

[1305] What it does: The server uses Matplotlib to generate a line graph of monthly sales and sends it to the terminal where the user can view the graph. The user can select a specific month from a dropdown menu to view more details.

[1306] Step 7:

[1307] Emotion Recognition and Adaptive Presentation

[1308] Server: Monitors user input and actions and uses an emotion recognition engine to infer emotions. Example: Determine stress levels based on typing speed and keystrokes.

[1309] Server: Dynamically adjusts visualization and presentation content based on the recognized emotion. Example: If the user is nervous, present a simple graph and add a detailed explanation.

[1310] Input: User operation data (typing speed, keystroke patterns)

[1311] Output: Adaptive visualization results

[1312] Specific operation: The server analyzes input speed information and, if it determines that the user is feeling stressed, it provides a simple bar graph that makes subtraction easy to understand and displays a detailed explanation in a pop-up.

[1313] Step 8:

[1314] Providing and customizing the interface

[1315] Server: Provides a user-friendly interface that allows for easy data entry, prompt entry, results review, filtering, and customization.

[1316] Users: Use a customized interface to quickly analyze data and make decisions.

[1317] Input: None (fixed interface)

[1318] Output: User-friendly interface

[1319] What it does: Users upload data on the data dashboard screen, select the appropriate filter options from the interface, and the graphs are updated in real time.

[1320] (Application example 2)

[1321] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1322] Modern logistics centers are swept up in a wide variety of data, and require systems that can efficiently collect, analyze, and visualize it. However, when employees are confronted with vast amounts of data, they can become stressed and fatigued, and there is a lack of ways to present information accordingly. This situation reduces work efficiency and becomes a source of stress in the workplace.

[1323] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1324] In this invention, the server includes: [means for collecting market data, public information, and business data;] [means for analyzing natural language prompts entered by a user to generate typed prompts; and] [means for recognizing a user's emotions using an emotion engine and dynamically adjusting the data visualization method.] This enables [employees to efficiently process massive amounts of data, and furthermore, the emotional information presentation method improves work efficiency and reduces stress].

[1325] "Market data" is a general term for external data related to commercial activities, such as market trends, trading information, and competitive analysis.

[1326] "Public Information" means data or information that is publicly available, including websites and public reports.

[1327] "Business data" refers to data generated within a company related to its daily operations, including sales information, inventory data, customer information, and so on.

[1328] "User" refers to the person who operates the system and performs data analysis.

[1329] "Natural language prompt" refers to a natural language instruction or question that a user enters into a system.

[1330] A "typed prompt" is an instruction that has been parsed from a natural language prompt and converted into a structure that is easy for the system to understand.

[1331] "Data Source" refers to the source from which data is obtained, including market data APIs and internal databases.

[1332] A "query" is a request to retrieve specific information from a database or API.

[1333] "Cleaning" refers to the process of preparing data by removing invalid values, missing values, and outliers before analyzing the data.

[1334] "Preprocessing" refers to the preparation of data before analysis, and includes standardizing formats and scaling.

[1335] "Visualization" refers to the process of displaying acquired and integrated data in a visual format such as a graph or chart.

[1336] "Filtering" is the process of extracting parts of data that meet specific conditions and using them for display or analysis.

[1337] "Customization" refers to adjusting the display content and functions of a system according to the user's requirements and circumstances.

[1338] "Interface" refers to the operation screen and input means used by users when interacting directly with the system.

[1339] The "emotion engine" is a module that analyzes the user's emotional state and dynamically adjusts the system's output and display.

[1340] "Smart glasses" refer to wearable devices that visually display information and allow users to check data in real time by wearing them.

[1341] This invention aims to enable employees working at logistics centers to efficiently collect, analyze, and visualize market data, business data, and real-time inventory information. In addition, by utilizing an emotion engine, it provides a method for presenting information according to the employee's emotional state, thereby improving work efficiency and reducing stress.

[1342] System Configuration

[1343] Hardware Configuration

[1344] Server: The central processing unit that collects, analyzes, consolidates, and visualizes data from market data APIs and operational databases.

[1345] Terminal: A device where a user inputs data and sees results, including smart glasses.

[1346] Smart glasses: Wearable devices that visually present data to the user in real time.

[1347] Software Configuration

[1348] Market Data Collection Module: Collects market data from external APIs.

[1349] Database Query Module: Executes queries against business databases to retrieve required data.

[1350] Data Cleaning and Integration Module: Cleans, pre-processes, and integrates the collected data.

[1351] Emotion engine: Analyzes data collected from the smart glasses' sensors and recognizes the user's emotions.

[1352] Visualization module: Visualizes the integrated data according to the user's emotional state.

[1353] Interface module: Provides an interface that allows users to filter and customize the data visualization results.

[1354] Processing flow

[1355] The server collects data from market data APIs and operational databases, cleans and pre-processes it, then integrates the data and creates visualizations based on the user's natural language prompts. The emotion engine recognizes the user's emotional state from the smart glasses' sensors and adjusts how the data is presented. Through the smart glasses, users can visually view the information.

[1356] Specifically, the user types the following natural language prompt into the terminal:

[1357] I want to conduct sales analysis using market data for 2023 and internal sales data.

[1358] The server analyzes the prompts and collects and analyzes the necessary data. After cleaning and preprocessing, the data is visualized by the emotion engine, taking into account the user's emotional state, and displayed on the smart glasses.

[1359] This allows users to view data in a way that is optimized for their emotional state, enabling them to make quick and efficient business decisions. For example, a user feeling tired will see a simple graph, while a relaxed user will be provided with detailed data.

[1360] This system is a powerful tool for increasing operational efficiency and reducing employee stress in logistics centers.

[1361] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1362] Step 1:

[1363] The server configures connections to market data APIs and business databases. Specifically, it saves the API endpoint, authentication key, and database connection string in a configuration file, and establishes connections to the API and database based on this information. The input is the API endpoint and authentication information, and the output is the connection establishment status.

[1364] Step 2:

[1365] The server receives a natural language prompt provided by the user. Through a terminal, the user inputs a prompt sentence such as: "I would like to perform a sales analysis using market data and business data for 2023." The input is the natural language prompt, and the output is the prompt sentence sent to the server.

[1366] Step 3:

[1367] The terminal parses the natural language prompt it receives and generates a typed prompt. It uses an NLP (Natural Language Processing) module to parse the prompt and convert it into a format that the system can understand. The input is the natural language prompt, and the output is the typed prompt.

[1368] Step 4:

[1369] The server queries the data source based on the typed prompts. It sends an HTTP request to the market data API to retrieve market data for the specified period. At the same time, it executes SQL queries against the operational database to retrieve the required data. The input is the typed prompts, and the output is the retrieved data.

[1370] Step 5:

[1371] The server cleans and preprocesses the acquired data. Specifically, it performs processes such as filling in missing values, standardizing data formats, and removing outliers. The cleaned data is saved in temporary storage. The input is the acquired data, and the output is the cleaned data.

[1372] Step 6:

[1373] The server integrates the cleaned data. It combines the market data and business data using a common key to generate integrated data. The input is the cleaned market data and business data, and the output is the integrated data.

[1374] Step 7:

[1375] The server visualizes the integrated data. It passes the data to the visualization tool in a user-specified format (e.g., graphs, charts, etc.) and displays it in a format that the user can easily understand. The input is the integrated data, and the output is the visualized data.

[1376] Step 8:

[1377] The server uses an emotion engine to recognize the user's emotions. It analyzes data obtained from the smart glasses' sensors (e.g., gaze tracking, voice, body movements, etc.) to understand the user's emotional state. The input is the sensor data, and the output is the recognized emotional state.

[1378] Step 9:

[1379] The server dynamically adjusts the visualization method based on the recognized emotional state. For example, if the user feels tired, it provides a simple graph, while if the user feels relaxed, it displays detailed data. The input is the recognized emotional state, and the output is the adjusted visualization data.

[1380] Step 10:

[1381] The terminal presents the adjusted visualized data to the user through the smart glasses. The user can check the data in real time through the smart glasses display and make business decisions. The input is the adjusted visualized data, and the output is the real-time visual information presented to the user.

[1382] In this way, efficient data visualization and emotion recognition are achieved through the processing and calculation of various input data at each processing step of the system.

[1383] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1384] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1385] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1386] [Fourth embodiment]

[1387] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1388] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1389] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1390] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1391] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1392] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1393] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1394] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1395] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1396] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1397] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1398] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1399] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1400] MODE FOR CARRYING OUT THE INVENTION

[1401] The present invention provides a system that allows business people to quickly and easily access market data, public information, and internal company data, and efficiently manages and visualizes the data in a unified manner. The following describes an embodiment of this system.

[1402] System Overview

[1403] The system mainly consists of a server, a terminal, and a user. The server collects, analyzes, integrates, and visualizes data, while the terminal provides an interface for users to enter prompts and check the visualization results. Users can automatically perform a series of processes from data collection to visualization simply by entering prompts.

[1404] What the program does

[1405] 1. Initial settings and data source registration

[1406] Server: The server stores API endpoints, authentication keys, database connection strings, etc. in configuration files, which establish connections to market data, public information, and internal databases.

[1407] User: The user provides the necessary API keys and database credentials to the server, which receives them and stores them securely using security features.

[1408] 2. Typing prompts

[1409] Terminal: The user types a natural language prompt into the terminal (e.g., "Sales analysis using market data and internal sales data for 2023").

[1410] Terminal: The terminal receives user input and parses the prompt using a natural language processing (NLP) module. The resulting prompt is then generated as a typed prompt and sent to the server.

[1411] 3. Data collection and processing

[1412] Server: The server receives the typed prompt and starts the data retrieval task. It sends an HTTP request to the market data API to retrieve market data for 2023. At the same time, it runs an SQL query against the internal database to retrieve sales data for 2023.

[1413] Server: The acquired data is stored in temporary storage and undergoes cleaning and pre-processing, which includes imputing missing values, standardizing data formats, and removing unnecessary data.

[1414] Server: The cleaned data is aggregated and joined using a common key (e.g., date or product ID).

[1415] 4. Data Visualization

[1416] Server: Passes the integrated data to a visualization tool and visualizes it in the format specified by the user (graph, table, dashboard, etc.). For example, it generates graphs using libraries such as Matplotlib or Tableau.

[1417] Terminal: The visualized data is sent to the terminal and displayed to the user, who can modify and customize the visualization using pull-down menus and filtering functions.

[1418] 5. Providing an Interface

[1419] Server: Provides a user-friendly interface that allows users to smoothly input data, complete prompts, view results, and filter and customize them.

[1420] Users: Using the provided interface, users can quickly analyze data and make management decisions and develop strategies. For example, they can adjust sales strategies based on the results of sales data analysis.

[1421] In this way, the system of the present invention automates the process from data collection to visualization, and provides an environment in which users can perform data analysis without needing specialized knowledge.

[1422] The processing flow will be explained below.

[1423] Step 1:

[1424] Server: The server configures the connection to the market data API and the internal database. Specifically, it stores the API endpoint, authentication key, and database connection string in a configuration file and authenticates against the API and database.

[1425] Step 2:

[1426] User: The user uses a device to send authentication information (API keys, database credentials) to the server, which receives it and stores it securely using security features.

[1427] Step 3:

[1428] Terminal: The user enters a natural language prompt into the terminal, for example, "Sales analysis using market data and internal sales data for 2023."

[1429] Step 4:

[1430] Terminal: The terminal receives the input natural language prompt and uses a natural language processing (NLP) module to parse and format the prompt. As a result of the prompt parsing, a formatted prompt is generated, including the data source to be used, the target time period, etc.

[1431] Step 5:

[1432] Terminal: Sends a typed prompt to the server.

[1433] Step 6:

[1434] Server: The server receives a typed prompt and begins the task of retrieving data by sending an HTTP request to the market data API to retrieve market data for the specified period.

[1435] Step 7:

[1436] Server: At the same time, the server runs a SQL query against the internal database to retrieve sales data for the specified period. For example, it runs the following SQL query:

[1437] sql

[1438] SELECT FROM sales WHERE year = 2023

[1439] Step 8:

[1440] Server: The acquired data is stored in temporary storage. After storage, data cleaning begins. Specifically, it performs tasks such as filling in missing values, standardizing data formats, and removing outliers.

[1441] Step 9:

[1442] Server: The cleaned data is consolidated using a common key (e.g., date or product ID), and the consolidated data is centralized and organized into a form suitable for analysis.

[1443] Step 10:

[1444] Server: Passes the consolidated data to visualization tools, for example, using visualization libraries such as Matplotlib or Tableau to generate graphs and tables.

[1445] Step 11:

[1446] Server: Sends visualized data to the device, where the data is presented in the format selected by the user (bar graph, line graph, dashboard, etc.).

[1447] Step 12:

[1448] Terminal: The terminal displays the visualization results to the user, who can further customize the displayed data using pull-down menus and filtering options.

[1449] Step 13:

[1450] Server: Ensures that data can be filtered, sorted, and prompted for additional input through a user-friendly interface, allowing users to easily explore data details and make quick business decisions based on the analysis results.

[1451] Through the above steps, this system provides an environment in which users can easily and efficiently analyze data and support rapid decision-making.

[1452] Example 1

[1453] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1454] In today's business environment, it is important to quickly and effectively integrate and analyze data from different sources. However, it is not easy to collect large amounts of market data, public information, and internal data, efficiently extract and integrate the necessary information, and visualize it in an easy-to-understand format. This task requires advanced expertise and a lot of time, placing a heavy burden on many businesspeople. In addition, the lack of intuitive ways to manipulate and customize visualized data makes it difficult to utilize analysis results in management decisions.

[1455] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1456] In this invention, the server includes: [means for collecting market data, public information, and internal company data;] [means for analyzing user-input natural language prompts and generating styled prompts using a generative AI model; and] [means for executing queries against data sources based on the styled prompts to obtain required data.] This makes it possible [to efficiently and effectively collect, integrate, analyze, and visualize required data from different data sources].

[1457] "Market data" refers to data that includes market trends, pricing information, supply and demand conditions, etc.

[1458] "Public information" is data that is publicly available and accessible to anyone, such as government statistics or news articles.

[1459] "Internal data" refers to data generated and managed within a company or organization. Examples include sales data, customer information, and inventory data.

[1460] A "natural language prompt" is an instruction or question entered by a user using natural language.

[1461] A "generative AI model" is a model that uses artificial intelligence to analyze prompts, understand their meaning, and generate structured prompts.

[1462] A "typed prompt" is a natural language prompt that has been parsed and converted into a specific format, making it easier for the system to extract the exact information needed.

[1463] A "Data Source" is an external or internal source from which data is obtained, such as an API, a database, or a CSV file.

[1464] A "query" is a query statement used to obtain information from a data source such as a database or API.

[1465] "Data cleaning" is the process of preparing data for analysis by filling in missing values, standardizing formats, and removing unnecessary data.

[1466] "Preprocessing" refers to a series of preparation steps that are performed before data analysis, including data cleaning and format conversion.

[1467] "Visualization" refers to the visual presentation of data in the form of graphs, tables, dashboards, etc.

[1468] An "interface" is a screen or function that allows a user to interact with a system, such as providing a prompt input screen or data filtering functionality.

[1469] MODE FOR CARRYING OUT THE INVENTION

[1470] This invention provides a system that allows business people to quickly and easily access market data, public information, and internal company data, and efficiently manage and visualize them in a centralized manner. In implementing this system, three parties - a server, a terminal, and a user - work together, each of which plays a specific role.

[1471] Hardware and software used

[1472] Hardware: The server uses a cloud infrastructure (e.g., Amazon Web Services, Microsoft Azure, etc.) equipped with a high-performance CPU and large amounts of memory. The terminals are PCs, tablets, smartphones, etc. operated by users.

[1473] Software: The following will be used:

[1474] Natural language processing modules (e.g., spaCy, Transformers)

[1475] API management tools (libraries that execute HTTP requests)

[1476] Database management system (e.g. PostgreSQL)

[1477] Libraries for data cleaning and preprocessing (e.g., Pandas)

[1478] Data visualization tools (e.g., Matplotlib, Tableau)

[1479] Web frameworks (e.g. Django, Flask)

[1480] What the program does

[1481] Initial Settings and Data Source Registration

[1482] The server stores API endpoints, authentication keys, and database connection strings in configuration files that establish connections to market data, public information, and internal company databases.

[1483] Users enter the necessary API keys and database authentication information through a dedicated interface and send it to the server, which then securely stores the received authentication information using its security features.

[1484] Prompt typing

[1485] The terminal provides an interface for users to enter natural language prompts, such as "Sales analysis using market data and internal sales data for 2023."

[1486] The device uses a natural language processing module (NLP) to parse the prompt and generate a typed prompt, which is then sent to the server.

[1487] Data collection and processing

[1488] The server receives the typed prompt and starts the data collection task, making an HTTP request to the market data API to retrieve market data for 2023. At the same time, it runs an SQL query against the internal database to retrieve sales data for 2023.

[1489] The server stores the acquired data in temporary storage and performs data cleaning and preprocessing, such as standardizing data formats, filling in missing values, and removing unnecessary data. The cleaned data is then integrated using a common key.

[1490] Data Visualization

[1491] The server passes the integrated data to a visualization tool, which visualizes it in the format specified by the user (e.g., graph, table, dashboard). For example, a sales progress graph can be generated using libraries such as Matplotlib or Tableau.

[1492] The terminal receives the visualized data and displays it to the user, who can access it through an interface that allows filtering and customization of the visualization results.

[1493] Providing an interface

[1494] The server provides a user-friendly interface that allows users to easily input data, enter prompts, view results, filter and customize, etc.

[1495] Users can quickly analyze data using the provided interface and use it to make business decisions and develop strategies. For example, they can adjust their sales strategies based on the results of sales data analysis.

[1496] As described above, the present invention automates the process from data collection to visualization, and provides an environment in which users can perform data analysis without needing specialized knowledge.

[1497] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1498] Step 1:

[1499] Initial Settings and Data Source Registration

[1500] The server saves the API endpoint, authentication key, and database connection string in a configuration file to establish connections to market data, public information, and internal data. The input is the API endpoint URL, authentication key, and database connection information, and the connection configuration is based on this. The output is the connection configuration information saved.

[1501] The user enters the necessary API keys and database credentials into a dedicated interface and sends them to the server. The input is the user-provided credentials sent to the server. The output is the credentials securely stored on the server.

[1502] Step 2:

[1503] Prompt typing

[1504] The terminal provides an interface for the user to input a prompt sentence in natural language. For example, the user might input "Sales analysis using market data and internal sales data for 2023." The input is the prompt sentence entered by the user in natural language.

[1505] The terminal parses the input prompt sentence using a natural language processing (NLP) module. As a result of the parsing, a typed prompt is generated and sent to the server. The output is the typed prompt sentence.

[1506] Step 3:

[1507] Data collection

[1508] The server initiates the data collection task based on the typed prompt, sending an HTTP request to the market data API to retrieve market data for the year 2023. The input is the typed prompt, and the API request is generated based on this information. The output is the retrieved market data stored in temporary storage.

[1509] The server runs SQL queries against the internal database to retrieve sales data for the year 2023. Similarly, the input is a typed prompt, and an SQL query is generated based on this information. As an output, the retrieved internal sales data is saved in temporary storage.

[1510] Step 4:

[1511] Data cleaning and preprocessing

[1512] The server temporarily stores the acquired market data and internal sales data, and then cleans and preprocesses the data. Specifically, it standardizes formats, fills in missing values, and deletes unnecessary data. The input is the acquired market data and internal sales data, and the output is cleaned, integrated data.

[1513] Step 5:

[1514] Data integration

[1515] The server aggregates the cleaned data, joining the data using a common key (e.g., date or product ID) to generate the aggregated data. The inputs are the cleaned market data and internal company data, and the output is the aggregated data.

[1516] Step 6:

[1517] Data Visualization

[1518] The server passes the integrated data to a visualization tool, which visualizes it in the format specified by the user (e.g., graph, table, dashboard, etc.). For example, a graph of sales trends is generated using a library such as Matplotlib or Tableau. The input is the integrated data, and the visualized data is generated as the output.

[1519] The terminal receives the visualized data and displays it to the user, who uses an interface to filter and customize the displayed graphs and tables. The input is the visualized data, and the output is the user-customized visualization.

[1520] Step 7:

[1521] Providing an interface

[1522] The server provides a user-friendly interface, allowing users to easily input data, enter prompts, check results, filter, customize, etc. The input is the information for the interface design, and the output is the operation screen that the user can use.

[1523] Users can use the provided interface to quickly analyze data and use it for business decisions and strategic planning. For example, they can adjust their sales strategy based on the results of sales data analysis. The input is the user's operations, and the output is specific analysis results.

[1524] (Application example 1)

[1525] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1526] In conventional factory operations, a wide variety of data is generated from various sources, but it has been difficult to efficiently collect, integrate, and visualize this data. It has been particularly difficult to monitor and analyze important performance indicators, such as the number of abnormalities and product pass rates, in real time. As a result, delays have occurred in maintenance and optimization of production lines, hindering efficient operations. To solve these issues, a system is needed that can quickly collect and analyze real-time data generated within the factory and visualize the results in an easily understandable format.

[1527] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1528] In this invention, the server includes: [means for collecting market data, public information, and internal company data;] [means for analyzing user-input natural language prompts to generate typed prompts;] [means for executing queries on data sources based on the typed prompts to obtain required data;] [means for collecting real-time data from sensors and machine operating status APIs within the factory; and [means for analyzing the collected data and visualizing production line performance indicators such as the number of abnormalities and the pass rate.] This enables efficient collection, integration, and cleaning of various data generated within the factory, and also enables real-time visualization of important performance indicators such as the number of abnormalities and the pass rate, enabling rapid response to optimize factory operations.

[1529] "Market Data" refers to various statistical information, trends, and other data related to the market.

[1530] "Public information" is information that is publicly available, including company financial reports, news articles, industry reports, etc.

[1531] "Internal data" refers to various data generated within a company, including business reports, sales data, product inspection data, and the like.

[1532] A "natural language prompt" refers to an instruction or question that a user enters in natural language.

[1533] A "typed prompt" refers to a prompt that is parsed from a natural language prompt and converted into a format that is easy for the system to understand.

[1534] "Data source" refers to the location or system from which data is obtained. Examples include APIs, databases, sensors, etc.

[1535] A "query" is an instruction to a database or API to retrieve specific data.

[1536] "Data integration" refers to the process of combining data from different sources into a single, consistent format.

[1537] "Cleaning" refers to the preprocessing of data to detect and correct incomplete, inaccurate, or unnecessary data.

[1538] "Preprocessing" refers to various preparatory tasks carried out before data analysis or visualization, including data normalization and conversion.

[1539] "Visualization" refers to the process of representing data in a visual format, such as a graph or chart, to allow users to intuitively understand the information.

[1540] "Filtering" refers to the process of specifying specific conditions when displaying or analyzing data and displaying only the necessary data.

[1541] "Customization" refers to tailoring the appearance and behavior of a system to meet the needs and requirements of the user.

[1542] "Interface" refers to the screen and input means that allow the user to operate the system.

[1543] A "sensor" refers to a device that measures physical environmental variables (such as temperature, humidity, and vibration) and outputs the data as an electronic signal.

[1544] "Machine operating status API" refers to a program interface for obtaining machine operating status and performance data.

[1545] "Real-time data" refers to the latest data available at the current time.

[1546] "Performance indicators" are standards for measuring and evaluating the efficiency and effectiveness of a system or process.

[1547] The "number of abnormal events" indicates the frequency of abnormal events occurring within a certain period of time.

[1548] The "pass rate" indicates the percentage of products that meet the standards and are judged to have passed the inspection.

[1549] The present invention provides a system for quickly collecting, analyzing, and integrating real-time data generated in a factory, and visualizing the data. Specific embodiments for carrying out the present invention will be described below.

[1550] System Overview

[1551] The system mainly consists of a server, a terminal, and a user. The server collects, analyzes, integrates, and visualizes data, while the terminal provides an interface for users to enter prompts and check the visualization results. By entering prompts, users can easily automate the entire process from data collection to visualization.

[1552] Hardware and software used

[1553] Hardware:

[1554] Sensors: Devices that measure physical environmental variables (temperature, vibration, pressure, etc.) within the factory

[1555] Server: Collects, analyzes, stores, and visualizes data

[1556] Terminal: A device where users can enter prompts and view visualization results.

[1557] Machine Operation Status API: Interface for obtaining the machine operation status

[1558] software:

[1559] Natural Language Processing (NLP) libraries: spaCy, NLTK, etc.

[1560] Data collection and cleaning: Pandas library (Python)

[1561] Database connection: SQLAlchemy (Python)

[1562] Visualization libraries: Matplotlib, Seaborn (Python)

[1563] Process Details

[1564] 1. Data collection and integration

[1565] The server collects real-time data from sensors installed in the factory and machine health APIs, such as temperature data from sensors and machine uptime information, while user-entered prompts are parsed and converted into typed prompts using natural language processing.

[1566] 2. Data cleaning and preprocessing

[1567] The collected data is stored in temporary storage and undergoes cleaning and preprocessing, such as imputing missing values, standardizing data formats, and removing inaccurate data.

[1568] 3. Data Visualization

[1569] The preprocessed data is integrated and visualized in a specified format (graph, chart, etc.). For example, data on the number of abnormalities or product pass rates can be visually displayed over time.

[1570] 4. Providing a user interface

[1571] The terminal displays the visualization results to the user, providing a filterable and customizable interface through which the user can quickly and efficiently analyze the data.

[1572] Prompt Sentence Examples

[1573] Below are some examples of prompt sentences.

[1574] "Analyze this month's product pass rate and number of machine abnormalities"

[1575] "Sales analysis using 2023 market data and internal sales data"

[1576] "What time of day this month are you seeing the most anomalies?"

[1577] The system of the present invention improves factory operations, enabling real-time data monitoring and analysis, which allows for quicker maintenance and production line adjustments and more efficient factory operations.

[1578] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1579] Step 1:

[1580] The server collects real-time data from sensors in the factory and machine operation status APIs. This includes temperature data, vibration data, and operating time information. The data sent from the sensors is accumulated on the server via HTTP requests. The server stores the acquired data in temporary storage.

[1581] Input: Real-time data from sensors and machine health APIs

[1582] Output: Raw data stored in temporary storage

[1583] Step 2:

[1584] The user inputs a natural language prompt through the terminal, for example, "Analyze this month's product pass rate and machine abnormality count." The terminal receives this natural language prompt, analyzes it using a natural language processing (NLP) module, and generates a typed prompt.

[1585] Input: User's natural language prompt

[1586] Output: Typed prompt

[1587] Step 3:

[1588] The server receives typed prompts and runs queries against the collected data, for example using a database query to get temperature data or anomaly counts for a specific time period, and the results of this query are stored in temporary storage in the form of a data frame.

[1589] Input: Typed prompts, raw data stored in temporary storage

[1590] Output: The specific data retrieved by the query

[1591] Step 4:

[1592] The server cleans and preprocesses the acquired data, including imputing missing values, removing outliers, and standardizing data formats. It uses the Pandas library to clean the data table and convert it into a format suitable for analysis.

[1593] Input: The specific data retrieved by the query

[1594] Output: Cleaned and preprocessed data

[1595] Step 5:

[1596] The server analyzes the preprocessed data and visualizes it in the specified format. For example, it can display the number of abnormalities or product pass rates as a line graph or bar graph. It generates graphs using the Matplotlib or Seaborn libraries and saves them as image files.

[1597] Input: Cleaned and preprocessed data

[1598] Output: Visualized graph or chart

[1599] Step 6:

[1600] The server then sends the generated visualization data to the terminal, which receives it and displays it graphically to the user, who can then use the provided interface to view the results and filter and customize them as needed.

[1601] Input: A visualized graph or chart

[1602] Output: A graph or chart displayed on the user's terminal

[1603] Step 7:

[1604] Based on the displayed data, users can optimize their production lines and create maintenance plans. For example, they can identify time periods when there are many abnormalities and concentrate maintenance work during those periods. The results of user operations are also saved on the server and reflected in future data analysis.

[1605] Input: User interactions and results

[1606] Output: Insights for maintenance planning and production line optimization

[1607] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1608] MODE FOR CARRYING OUT THE INVENTION

[1609] This invention provides a system that enables businesspeople to integrate market data, public information, and internal company data, and customizes data analysis by recognizing user emotions. The system is primarily composed of a server, terminals, and users, and is combined with an emotion engine. This makes it possible to dynamically adjust the data visualization method and presentation content according to the user's emotions.

[1610] System Overview

[1611] The system consists of the following main components:

[1612] 1. Server: The central processing unit responsible for data collection, analysis, integration, visualization, and emotion recognition.

[1613] 2. Terminal: A device that provides an interface for users to enter prompts and view visualization results.

[1614] 3. User: The person who uses the system to perform data analysis.

[1615] 4. Emotion Engine: A module that recognizes emotions from user input and behavior and customizes data analysis and visualization.

[1616] What the program does

[1617] 1. Initial settings and data source registration

[1618] Server: The server configures the connection to the market data API and the internal database, storing the API endpoint, authentication key, and database connection string in a configuration file, which then authenticates against the API and database.

[1619] User: The user provides credentials (API keys or database credentials) to the server, which stores them securely using security features.

[1620] 2. Typing prompts

[1621] Terminal: The user types a natural language prompt into the terminal, for example, "Sales analysis using market data and internal sales data for 2023."

[1622] Terminal: The terminal receives the natural language prompt, parses and formats the prompt using the NLP module, and sends the formatted prompt to the server.

[1623] 3. Data collection and processing

[1624] Server: The server initiates the data retrieval task based on a typed prompt. It sends an HTTP request to the market data API to retrieve market data for the specified period. At the same time, the server executes an SQL query against the internal database to retrieve sales data for the specified period.

[1625] Server: The acquired data is stored in temporary storage and undergoes cleaning and preprocessing, such as imputing missing values, standardizing data formats, and removing outliers.

[1626] Server: The cleaned data is integrated using a common key, and the integrated data is centralized and made suitable for analysis.

[1627] 4. Data Visualization

[1628] Server: Passes the integrated data to the visualization tool and visualizes it in the format specified by the user. It generates graphs and tables using visualization libraries such as Matplotlib and Tableau.

[1629] Server: Sends the visualized data to the terminal and displays it to the user, who can modify and customize the visualization using pull-down menus and filtering options.

[1630] 5. Emotion Engine

[1631] Server: Monitors user input and behavior and uses an emotion engine to recognize user emotions, for example, inferring stress or satisfaction from the user's typing speed and keystroke patterns.

[1632] Server: Dynamically adjusts the visualization and presentation of data based on the perceived emotion. If the user is nervous, it might provide a simpler graph or add a more detailed explanation.

[1633] 6. Providing an Interface

[1634] Server: Provides a user-friendly interface that allows users to smoothly input data, complete prompts, view results, filter and customize them, etc.

[1635] Users: Use a customized interface based on emotion recognition to quickly analyze data and make business decisions. For example, if a user is feeling stressed, the system will provide a simplified visualization and guide subsequent operations.

[1636] In this way, the system of the present invention automates the process from data collection to visualization, and further provides data analysis customized according to the user's emotions, thereby providing an environment in which users can efficiently utilize data without the need for specialized knowledge.

[1637] The processing flow will be explained below.

[1638] Step 1:

[1639] Server: The server configures the connection to the market data API and the internal database. Specifically, it stores the API endpoints, authentication keys, and database connection strings in configuration files and establishes the connection to the API and database.

[1640] Step 2:

[1641] User: The user uses a device to send their credentials (API key or database credentials) to the server, which receives them and stores them securely using security features.

[1642] Step 3:

[1643] Terminal: The user types a natural language prompt into the terminal (e.g., "Sales analysis using market data and internal sales data for 2023").

[1644] Step 4:

[1645] Terminal: The terminal receives the input natural language prompt and parses it using a natural language processing (NLP) module, generating a typed prompt as a result, which it then sends to the server:

[1646] {

[1647] "action": "Sales analysis",

[1648] "data_sources": ["Market Data API", "Internal Database"],

[1649] "year": 2023

[1650] }

[1651] Step 5:

[1652] Server: The server receives the typed prompt and sends an HTTP request to the market data API to retrieve market data for the specified period. It also executes an SQL query against the internal database to retrieve sales data for the specified period. For example, it executes the following SQL query:

[1653] sql

[1654] SELECT FROM sales WHERE year = 2023

[1655] Step 6:

[1656] Server: The acquired data is stored in temporary storage. After storage, data cleaning is performed. Specifically, missing values ​​are filled, the data format is standardized, and outliers are removed.

[1657] Step 7:

[1658] Server: The cleaned data is integrated using a common key (date or product ID), and the integrated data is centralized and organized into a form suitable for analysis.

[1659] Step 8:

[1660] Server: Passes the consolidated data to a visualization tool and visualizes it in the format specified by the user. For example, it generates graphs and tables using libraries such as Matplotlib or Tableau.

[1661] Step 9:

[1662] Server: Sends visualized data to the device, where it is displayed in the format selected by the user (bar graph, line graph, dashboard, etc.).

[1663] Step 10:

[1664] Terminal: The terminal displays the visualization results to the user, who can customize the displayed data using pull-down menus and filtering options.

[1665] Step 11:

[1666] Server: Uses an emotion engine to recognize emotions from user input and behavior, for example, inferring stress or satisfaction from a user's typing speed and keystroke patterns.

[1667] Step 12:

[1668] Server: Dynamically adjusts the visualization and presentation of data based on the user's emotions as recognized by the emotion engine. For example, if the user is nervous, provide a simpler graph or add a detailed explanation.

[1669] Step 13:

[1670] Server: Sends customized visualization results to the device, presenting data in a way that is easier for users to understand.

[1671] Step 14:

[1672] Users: Through a customized interface, they can quickly analyze data and make business decisions, filtering and re-prompting as needed. For example, they can adjust marketing strategies for specific products or services based on the results of analyzing sales data.

[1673] In this way, the system of the present invention automates the process from data collection to visualization, and further provides customization according to the user's emotions, thereby providing an environment in which users can efficiently utilize data without the need for specialized knowledge.

[1674] Example 2

[1675] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1676] In conventional data analysis systems, market data, public information, and internal data are collected and analyzed separately, which makes data integration time-consuming and labor-intensive, making efficient data analysis difficult. Furthermore, they lack the functionality to dynamically adjust data analysis taking into account the user's emotions and stress levels, making them less user-friendly.

[1677] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1678] In this invention, the server includes: [means for collecting market data, public information, and internal company data]; [means for analyzing user-input natural language prompts to generate typed prompts]; [means for executing queries against data sources based on the typed prompts to obtain required data]; [means for integrating, cleaning, and preprocessing the acquired data]; [means for visualizing the integrated data in a user-specified format]; [means for providing an interface for users to filter and customize data visualization results]; and [means for monitoring user input and behavior, recognizing emotions, and dynamically adjusting data analysis and visualization.] This makes it possible [to simplify data integration processing and achieve efficient data analysis]. It also makes it possible [to recognize user emotions and customize data analysis in real time, improving operability and providing a stress-free data analysis environment].

[1679] "Market Data" refers to information relating to economic activity and transactions in a particular market.

[1680] "Public information" refers to data and information that is publicly available, including, for example, government statistics, news, and reports.

[1681] "Internal data" refers to data generated within a company, including sales data, customer information, and financial data.

[1682] "User input" refers to all input operations performed by users of the system via their terminals.

[1683] A "natural language prompt" refers to a sentence of instructions or requests that a user enters in everyday language.

[1684] A "typed prompt" refers to a prompt that has been parsed from a natural language prompt and converted into a format that is easy for the system to process.

[1685] "Data source" refers to the origin or source from which data is obtained, including APIs and databases.

[1686] A "query" is a request made to a database or API to search or retrieve data.

[1687] "Cleaning" refers to the processing of data to improve its quality by correcting or removing missing or outlier values.

[1688] "Preprocessing" refers to preparatory work such as data standardization, conversion, and filtering that is carried out before data analysis.

[1689] "Visualization" refers to the process of displaying data in a visual format such as a graph, chart, or table.

[1690] "Filtering" refers to the process of selecting and displaying data based on specific conditions.

[1691] "Customization" refers to the operation of changing the system or the display format of data according to the user's needs.

[1692] "Interface" refers to the input / output means by which a user and a system interact with each other.

[1693] "Emotion recognition" refers to the process of identifying emotions from user input and behavior and having the system respond accordingly.

[1694] MODE FOR CARRYING OUT THE INVENTION

[1695] The present invention provides a system that enables businesspeople to handle market data, public information, and internal company data in an integrated manner, and further customizes data analysis by recognizing user emotions. The system is primarily composed of a server, a terminal, and a user, combined with an emotion recognition engine. Specific embodiments for implementing this system are described in detail below.

[1696] Hardware and Software Configuration

[1697] 1. Server

[1698] Hardware: Dedicated servers with powerful CPUs, large amounts of RAM and storage.

[1699] Software: Software for data collection, analysis, synthesis, visualization, and emotion recognition. Examples include databases (e.g., MySQL, PostgreSQL), programming languages ​​(e.g., Python), data visualization libraries (e.g., Matplotlib, Tableau), natural language processing engines (e.g., spaCy, NLTK), and emotion recognition models.

[1700] 2. Terminal

[1701] Hardware: The input and display devices used by users, such as PCs, tablets, and smartphones.

[1702] Software: Web browsers and applications used to provide a user interface. Examples include web browsers (e.g., Google Chrome, Mozilla Firefox) and user interface frameworks (e.g., React, Vue.js).

[1703] 3. Users

[1704] Business people using systems to analyze data.

[1705] Data processing and calculation process

[1706] The server uses a natural language processing engine to parse the natural language prompts received from the user and generate typed prompts. Based on the typed prompts, the server queries market data APIs and internal databases to retrieve the required data. The retrieved data is stored in temporary storage, after which it is cleaned and pre-processed. Cleaning includes missing value imputation, data format standardization, and outlier removal. Pre-processing includes data standardization, transformation, and filtering.

[1707] The integrated data is visualized using a data visualization library in the form of graphs, charts, tables, etc. The generated visualization results are sent to the terminal, providing an interface for users to filter and customize the data based on specific conditions.

[1708] Additionally, the server monitors user input and behavior and uses an emotion recognition engine to identify the user's emotions. Based on the emotion, the system dynamically adjusts the visualization and presentation of the data. For example, if the user is feeling stressed, the system will provide a simpler graph and add a detailed explanation.

[1709] Specific examples

[1710] Example 1: Sales analysis of market data and internal sales data

[1711] User: Enter "Sales analysis using 2023 market data and internal sales data."

[1712] Terminal: Parses the prompt and sends the typed prompt to the server.

[1713] Server: Collects data from specified data sources, cleans and integrates it, uses a data visualization library to generate a time series graph of sales, and sends it to the terminal.

[1714] Server: Monitors the user's emotions and, if they are nervous, provides a simple graph and pops up a detailed explanation.

[1715] Example 2: Customer Satisfaction Analysis

[1716] User: Type "Create Customer Satisfaction Report for Q4 2022."

[1717] Terminal: Type natural language prompts and send them to the server.

[1718] Server: Acquires data on customer satisfaction (such as survey results), cleans and integrates them, and uses Tableau to generate a heat map of customer satisfaction and send it to the device.

[1719] Server: Adjust the color and detail of the heatmap based on user sentiment.

[1720] In this way, the system of the present invention not only automates the process of collecting, processing, and visualizing the necessary data based on the user's natural language prompts, but also customizes data analysis according to the user's emotions, thereby achieving fast and efficient data analysis.

[1721] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1722] Step 1:

[1723] Initial setup and registration of authentication information

[1724] Server: Store your market data API endpoints, authentication keys, and internal database connection strings in a configuration file, which will connect and authenticate to the API and database.

[1725] User: Enters API key and authentication information on the initial setup screen of the system. The server encrypts and stores this information securely and returns a connection success message.

[1726] Input: API endpoint, authentication key, database connection information

[1727] Output: Connection successful message

[1728] Specific operation: The user enters authentication information into the input form and clicks the submit button. The server sets up the connection based on the entered information and returns a message when the connection is successful.

[1729] Step 2:

[1730] Entering and parsing user prompts

[1731] User: Uses a terminal interface to enter natural language prompts. Example: "Sales analysis using market data and internal sales data for 2023."

[1732] Terminal: Parses the input natural language prompt using an NLP module (e.g. spaCy, NLTK), generates a typed prompt, and sends it to the server.

[1733] Input: Natural language prompt (e.g., "Sales analysis using market data and internal sales data for 2023")

[1734] Output: Typed prompt

[1735] Specific operation: The user enters a prompt, and the device analyzes the natural language prompt to extract key information such as "market data," "2023," "sales data," and "sales analysis," then formats and sends it to the server.

[1736] Step 3:

[1737] Data collection

[1738] Server: Initiates data collection tasks based on typed prompts, sends HTTP requests to the market data API to retrieve market data for the specified period, and simultaneously executes SQL queries against the internal database to retrieve the required sales data.

[1739] Input: Typed prompt

[1740] Output: Collected data (market data, internal sales data)

[1741] Specific operation: The server sends an HTTP GET request to "api.marketdata.com" to retrieve market data for 2023. At the same time, it executes the query "SELECT FROM sales_data WHERE year = 2023" against the internal database to retrieve sales data.

[1742] Step 4:

[1743] Data cleaning and preprocessing

[1744] Server: Stores the acquired data in temporary storage and performs cleaning and preprocessing, such as imputing missing values, standardizing data formats, and detecting and removing outliers.

[1745] Input: Collected data

[1746] Output: Cleaned data

[1747] Specific operation: The server checks the collected data and performs missing value imputation (e.g., replacing "0" or "null" with the appropriate value), formatting standardization (e.g., standardizing date formats), and outlier removal (e.g., removing abnormally high or low values).

[1748] Step 5:

[1749] Data Integration and Transformation

[1750] Server: The cleaned data is consolidated using a common key (e.g., date) and made suitable for analysis.

[1751] Input: Cleaned data

[1752] Output: Integrated data

[1753] What it does: The server joins market data and sales data based on "date" to create a unified data set, enabling more comprehensive analysis.

[1754] Step 6:

[1755] Data Visualization

[1756] Server: Passes the integrated data to a visualization tool (e.g., Matplotlib, Tableau) and visualizes it in the format specified by the user.

[1757] Server: The visualized data is sent to the terminal and displayed to the user, who can filter and customize it.

[1758] Input: Integrated data

[1759] Output: Visualization results (graphs and charts)

[1760] What it does: The server uses Matplotlib to generate a line graph of monthly sales and sends it to the terminal where the user can view the graph. The user can select a specific month from a dropdown menu to view more details.

[1761] Step 7:

[1762] Emotion Recognition and Adaptive Presentation

[1763] Server: Monitors user input and actions and uses an emotion recognition engine to infer emotions. Example: Determine stress levels based on typing speed and keystrokes.

[1764] Server: Dynamically adjusts visualization and presentation content based on the recognized emotion. Example: If the user is nervous, present a simple graph and add a detailed explanation.

[1765] Input: User operation data (typing speed, keystroke patterns)

[1766] Output: Adaptive visualization results

[1767] Specific operation: The server analyzes input speed information and, if it determines that the user is feeling stressed, it provides a simple bar graph that makes subtraction easy to understand and displays a detailed explanation in a pop-up.

[1768] Step 8:

[1769] Providing and customizing the interface

[1770] Server: Provides a user-friendly interface that allows for easy data entry, prompt entry, results review, filtering, and customization.

[1771] Users: Use a customized interface to quickly analyze data and make decisions.

[1772] Input: None (fixed interface)

[1773] Output: User-friendly interface

[1774] What it does: Users upload data on the data dashboard screen, select the appropriate filter options from the interface, and the graphs are updated in real time.

[1775] (Application example 2)

[1776] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1777] Modern logistics centers are swept up in a wide variety of data, and require systems that can efficiently collect, analyze, and visualize it. However, when employees are confronted with vast amounts of data, they can become stressed and fatigued, and there is a lack of ways to present information accordingly. This situation reduces work efficiency and becomes a source of stress in the workplace.

[1778] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1779] In this invention, the server includes: [means for collecting market data, public information, and business data;] [means for analyzing natural language prompts entered by a user to generate typed prompts; and] [means for recognizing a user's emotions using an emotion engine and dynamically adjusting the data visualization method.] This enables [employees to efficiently process massive amounts of data, and furthermore, the emotional information presentation method improves work efficiency and reduces stress].

[1780] "Market data" is a general term for external data related to commercial activities, such as market trends, trading information, and competitive analysis.

[1781] "Public Information" means data or information that is publicly available, including websites and public reports.

[1782] "Business data" refers to data generated within a company related to its daily operations, including sales information, inventory data, customer information, and so on.

[1783] "User" refers to the person who operates the system and performs data analysis.

[1784] "Natural language prompt" refers to a natural language instruction or question that a user enters into a system.

[1785] A "typed prompt" is an instruction that has been parsed from a natural language prompt and converted into a structure that is easy for the system to understand.

[1786] "Data Source" refers to the source from which data is obtained, including market data APIs and internal databases.

[1787] A "query" is a request to retrieve specific information from a database or API.

[1788] "Cleaning" refers to the process of preparing data by removing invalid values, missing values, and outliers before analyzing the data.

[1789] "Preprocessing" refers to the preparation of data before analysis, and includes standardizing formats and scaling.

[1790] "Visualization" refers to the process of displaying acquired and integrated data in a visual format such as a graph or chart.

[1791] "Filtering" is the process of extracting parts of data that meet specific conditions and using them for display or analysis.

[1792] "Customization" refers to adjusting the display content and functions of a system according to the user's requirements and circumstances.

[1793] "Interface" refers to the operation screen and input means used by users when interacting directly with the system.

[1794] The "emotion engine" is a module that analyzes the user's emotional state and dynamically adjusts the system's output and display.

[1795] "Smart glasses" refer to wearable devices that visually display information and allow users to check data in real time by wearing them.

[1796] This invention aims to enable employees working at logistics centers to efficiently collect, analyze, and visualize market data, business data, and real-time inventory information. In addition, by utilizing an emotion engine, it provides a method for presenting information according to the employee's emotional state, thereby improving work efficiency and reducing stress.

[1797] System Configuration

[1798] Hardware Configuration

[1799] Server: The central processing unit that collects, analyzes, consolidates, and visualizes data from market data APIs and operational databases.

[1800] Terminal: A device where a user inputs data and sees results, including smart glasses.

[1801] Smart glasses: Wearable devices that visually present data to the user in real time.

[1802] Software Configuration

[1803] Market Data Collection Module: Collects market data from external APIs.

[1804] Database Query Module: Executes queries against business databases to retrieve required data.

[1805] Data Cleaning and Integration Module: Cleans, pre-processes, and integrates the collected data.

[1806] Emotion engine: Analyzes data collected from the smart glasses' sensors and recognizes the user's emotions.

[1807] Visualization module: Visualizes the integrated data according to the user's emotional state.

[1808] Interface module: Provides an interface that allows users to filter and customize the data visualization results.

[1809] Processing flow

[1810] The server collects data from market data APIs and operational databases, cleans and pre-processes it, then integrates the data and creates visualizations based on the user's natural language prompts. The emotion engine recognizes the user's emotional state from the smart glasses' sensors and adjusts how the data is presented. Through the smart glasses, users can visually view the information.

[1811] Specifically, the user types the following natural language prompt into the terminal:

[1812] I want to conduct sales analysis using market data for 2023 and internal sales data.

[1813] The server analyzes the prompts and collects and analyzes the necessary data. After cleaning and preprocessing, the data is visualized by the emotion engine, taking into account the user's emotional state, and displayed on the smart glasses.

[1814] This allows users to view data in a way that is optimized for their emotional state, enabling them to make quick and efficient business decisions. For example, a user feeling tired will see a simple graph, while a relaxed user will be provided with detailed data.

[1815] This system is a powerful tool for increasing operational efficiency and reducing employee stress in logistics centers.

[1816] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1817] Step 1:

[1818] The server configures connections to market data APIs and business databases. Specifically, it saves the API endpoint, authentication key, and database connection string in a configuration file, and establishes connections to the API and database based on this information. The input is the API endpoint and authentication information, and the output is the connection establishment status.

[1819] Step 2:

[1820] The server receives a natural language prompt provided by the user. Through a terminal, the user inputs a prompt sentence such as: "I would like to perform a sales analysis using market data and business data for 2023." The input is the natural language prompt, and the output is the prompt sentence sent to the server.

[1821] Step 3:

[1822] The terminal parses the natural language prompt it receives and generates a typed prompt. It uses an NLP (Natural Language Processing) module to parse the prompt and convert it into a format that the system can understand. The input is the natural language prompt, and the output is the typed prompt.

[1823] Step 4:

[1824] The server queries the data source based on the typed prompts. It sends an HTTP request to the market data API to retrieve market data for the specified period. At the same time, it executes SQL queries against the operational database to retrieve the required data. The input is the typed prompts, and the output is the retrieved data.

[1825] Step 5:

[1826] The server cleans and preprocesses the acquired data. Specifically, it performs processes such as filling in missing values, standardizing data formats, and removing outliers. The cleaned data is saved in temporary storage. The input is the acquired data, and the output is the cleaned data.

[1827] Step 6:

[1828] The server integrates the cleaned data. It combines the market data and business data using a common key to generate integrated data. The input is the cleaned market data and business data, and the output is the integrated data.

[1829] Step 7:

[1830] The server visualizes the integrated data. It passes the data to the visualization tool in a user-specified format (e.g., graphs, charts, etc.) and displays it in a format that the user can easily understand. The input is the integrated data, and the output is the visualized data.

[1831] Step 8:

[1832] The server uses an emotion engine to recognize the user's emotions. It analyzes data obtained from the smart glasses' sensors (e.g., gaze tracking, voice, body movements, etc.) to understand the user's emotional state. The input is the sensor data, and the output is the recognized emotional state.

[1833] Step 9:

[1834] The server dynamically adjusts the visualization method based on the recognized emotional state. For example, if the user feels tired, it provides a simple graph, while if the user feels relaxed, it displays detailed data. The input is the recognized emotional state, and the output is the adjusted visualization data.

[1835] Step 10:

[1836] The terminal presents the adjusted visualized data to the user through the smart glasses. The user can check the data in real time through the smart glasses display and make business decisions. The input is the adjusted visualized data, and the output is the real-time visual information presented to the user.

[1837] In this way, efficient data visualization and emotion recognition are achieved through the processing and calculation of various input data at each processing step of the system.

[1838] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1839] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1840] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1841] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1842] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1843] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1844] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1845] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1846] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1847] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1848] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1849] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1850] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1851] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1852] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1853] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1854] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1855] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1856] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1857] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1858] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1859] The following is further disclosed regarding the above embodiment.

[1860] (Claim 1)

[1861] [Means of collecting market data, public information, and internal company data;

[1862] [means for parsing a user-input natural language prompt to generate a typed prompt;

[1863] [A means to query the data source based on typed prompts to retrieve the required data; and

[1864] [Methods for integrating, cleaning and pre-processing the acquired data;

[1865] [Means for visualizing the integrated data in a user-specified format; and

[1866] [A means for providing an interface for users to filter and customize data visualization results; and

[1867] A system including:

[1868] (Claim 2)

[1869] [The system of claim 1, wherein natural language processing techniques are used for prompt analysis.

[1870] (Claim 3)

[1871] [The system of claim 1, wherein the system simultaneously queries a market data API and an internal database.

[1872] "Example 1"

[1873] (Claim 1)

[1874] [Means of collecting market data, public information, and internal company data;

[1875] [Means for analyzing user-input natural language prompts and generating typed prompts using a generative AI model; and

[1876] [A means to query the data source based on typed prompts to retrieve the required data; and

[1877] [Methods for integrating, cleaning and pre-processing the acquired data;

[1878] [Means for visualizing the integrated data in a user-specified format; and

[1879] [Means of sending visualization results to a terminal and providing an interface for users to view, filter, and customize the results;

[1880] A system including:

[1881] (Claim 2)

[1882] [The system of claim 1 uses natural language processing techniques for prompt analysis and generates typed prompts using a generative AI model.

[1883] (Claim 3)

[1884] [The system of claim 1 performs simultaneous queries against a market data API and an internal database, and performs data cleaning and pre-processing as the data is retrieved.

[1885] "Application Example 1"

[1886] (Claim 1)

[1887] [Means of collecting market data, public information, and internal company data;

[1888] [means for parsing a user-input natural language prompt to generate a typed prompt;

[1889] [A means to query the data source based on typed prompts to retrieve the required data; and

[1890] [Methods for integrating, cleaning and pre-processing the acquired data;

[1891] [Means for visualizing the integrated data in a user-specified format; and

[1892] [A means for providing an interface for users to filter and customize data visualization results; and

[1893] [Means of collecting real-time data from sensors and machine status APIs in the factory;

[1894] [Means for analyzing collected data and visualizing production line performance indicators such as the number of abnormalities and pass rate,

[1895] A system including:

[1896] (Claim 2)

[1897] [The system of claim 1, wherein natural language processing techniques are used for prompt analysis.

[1898] (Claim 3)

[1899] [The system of claim 1, wherein the system simultaneously queries a market data API and an internal database.

[1900] "Example 2: Combining Emotion Engines"

[1901] (Claim 1)

[1902] [Means of collecting market data, public information, and internal company data;

[1903] [means for parsing a user-input natural language prompt to generate a typed prompt;

[1904] [A means to query the data source based on typed prompts to retrieve the required data; and

[1905] [Methods for integrating, cleaning and pre-processing the acquired data;

[1906] [Means for visualizing the integrated data in a user-specified format; and

[1907] [A means for providing an interface for users to filter and customize data visualization results; and

[1908] [Means of monitoring user input and behavior, recognizing emotions, and dynamically adjusting data analysis and visualization;

[1909] A system including:

[1910] (Claim 2)

[1911] [The system of claim 1, wherein natural language processing techniques are used for prompt analysis.

[1912] (Claim 3)

[1913] [The system of claim 1, wherein the system simultaneously queries a market data API and an internal database.

[1914] (Claim 4)

[1915] [The system of claim 1, which adjusts the method and content of data visualization based on emotion recognition.

[1916] "Application example 2 when combining emotion engines"

[1917] (Claim 1)

[1918] [Means of collecting market data, public information, and business data; and

[1919] [means for parsing a user-input natural language prompt to generate a typed prompt;

[1920] [A means to query the data source based on typed prompts to retrieve the required data; and

[1921] [Methods for integrating, cleaning and pre-processing the acquired data;

[1922] [Means for visualizing the integrated data in a user-specified format; and

[1923] [A means for providing an interface for users to filter and customize data visualization results; and

[1924] [Means for recognizing user emotions using an emotion engine and dynamically adjusting data visualization methods;

[1925] [Means for visually presenting information to a user using smart glasses;

[1926] A system including:

[1927] (Claim 2)

[1928] [The system of claim 1, wherein natural language processing techniques are used for prompt analysis.

[1929] (Claim 3)

[1930] [The system of claim 1, wherein the system executes queries against the market data API and the business database simultaneously. [Explanation of symbols]

[1931] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means of collecting market data, public information, and internal company data; means for parsing a user-input natural language prompt to generate a typed prompt; A means to query the data source based on typed prompts to retrieve the required data; A means of integrating, cleaning and pre-processing the acquired data; a means for visualizing the integrated data in a user-specified format; a means for providing an interface for a user to filter and customize the data visualization results; A system including:

2. The system of claim 1 , wherein the prompt analysis uses natural language processing techniques.

3. The system of claim 1 , wherein the system simultaneously queries the market data API and the in-house database.

Images