system

Abstract

Provide a system. A data input means; A means for dividing input data into features and target variables; means for splitting the split data into training data and test data; A means of selecting the appropriate machine learning model depending on the type of problem; means for training the selected model with training data; means for generating prediction results and evaluation scores for the model using the test data; means for outputting the prediction result and the evaluation score; A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In recent years, data analysis and machine learning have become increasingly important, but utilizing them requires specialized knowledge. When ordinary companies or individuals want to perform data analysis, they incur the cost of hiring experts such as data scientists and the burden of learning to use complex tools. However, to easily make data-driven decisions, tools that can easily perform predictions and classifications without requiring specialized knowledge are needed. To address this issue, the present invention provides a no-code tool that can perform data predictions and classifications with simple operations, even without the knowledge of a data scientist. [Means for solving the problem]

[0005] The present invention solves the above-mentioned problems by providing a system including a data input means, a means for dividing the input data into features and a response variable, a means for dividing the divided data into training data and test data, a means for selecting an appropriate machine learning model according to the type of problem, a means for training the selected model with the training data, a means for generating prediction results and evaluation scores for the model using the test data, and a means for outputting the prediction results and evaluation scores. Furthermore, by selecting a random forest classifier when the type of problem is classification and a random forest regressor when the type of problem is regression, the system allows a user to select an appropriate model without specialized knowledge. Furthermore, the data input means accepts input from the user and displays the prediction results and evaluation scores to the user, thereby improving user convenience.

[0006] The "data input means" is a means for a user to input data that he or she wishes to analyze into the system.

[0007] A "feature" is an aspect or attribute of data that is used as an independent variable in a predictive model.

[0008] A "target variable" is a dependent variable that is the target of prediction in a prediction model.

[0009] "Training data" refers to the data set used to train a machine learning model.

[0010] "Test data" refers to a dataset used to evaluate the performance of a machine learning model.

[0011] A "machine learning model" is an algorithm that learns patterns from data and performs prediction or classification tasks.

[0012] A "random forest classifier" is a type of machine learning algorithm that builds multiple decision trees and aggregates their predictions to perform classification.

[0013] A "random forest regressor" is a type of machine learning algorithm that uses multiple decision trees to perform regression analysis and predict continuous values.

[0014] "Prediction results" refer to the predicted values ​​or classification results generated by a machine learning model based on input data.

[0015] An "evaluation score" is an index used to quantitatively evaluate the performance of a machine learning model. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] This invention is a no-code tool that allows users without the specialized knowledge of data scientists to easily predict and classify data.

[0038] System Overview

[0039] When a user inputs their data and request, the server automatically selects an appropriate machine learning model based on that data, trains it, and evaluates it, returning the results to the user via their device.

[0040] Server-side processing

[0041] The server processes the data as follows:

[0042] 1. Receive data entered by the user.

[0043] 2. Convert the received data into a Pandas DataFrame.

[0044] 3. Split the target variable and features.

[0045] 4. Split the data into training and test data.

[0046] 5. Select the appropriate model depending on the type of problem (classification or regression).

[0047] 6. Train the selected model on the training data.

[0048] 7. Use the test data to generate model predictions and evaluation scores.

[0049] 8. Output the generated prediction results and evaluation scores in JSON format.

[0050] Operation on the terminal side

[0051] The terminal performs the following actions:

[0052] 1. Accepts data and prediction or classification settings from the user as input.

[0053] 2. Convert the input data into a specified format and send it to the server as a POST request.

[0054] 3. Receive the prediction results and evaluation scores returned from the server.

[0055] 4. Display the received results to the user.

[0056] User operations

[0057] The user does the following:

[0058] 1. Enter data into the system (e.g., upload a CSV file or enter data manually).

[0059] 2. Specify the target variable for prediction or classification and the type of problem (classification or regression).

[0060] 3. The system processes the request on the server side and waits for the results to be returned.

[0061] 4. The results will be displayed on your device, and you can check the prediction results and evaluation score.

[0062] Specific examples

[0063] For example, suppose a user prepares the following dataset and performs a classification task:

[0064] Example dataset:

[0065] Feature 1: Age

[0066] Feature 2: Income

[0067] target: purchase history (0 or 1)

[0068] Once the user enters this data into the system and selects a classification task, the following happens automatically on the server side:

[0069] 1. Data is received and converted into a DataFrame.

[0070] 2. Age and income are set as features, and purchase history is set as the target variable.

[0071] 3. Split the dataset into training and test data.

[0072] 4. The server selects a random forest classifier for the classification problem and trains the model on the training data.

[0073] 5. The trained model is used to make predictions on the test data, generating results and evaluation scores.

[0074] 6. The generated prediction results and evaluation scores are sent back to the user's device and displayed on the screen.

[0075] This allows users to predict and classify data without specialized knowledge, making it easy to make data-based decisions.

[0076] The processing flow will be explained below.

[0077] Step 1:

[0078] The user inputs data into the system. The data can be uploaded as a CSV file or entered directly into the system's input form. This input specifies the target variable for prediction or classification and the type of problem (classification or regression).

[0079] Step 2:

[0080] The terminal receives the data entered by the user and converts it into JSON format, which includes features, target variables, and problem types.

[0081] Step 3:

[0082] The device sends JSON-formatted data to the server as a POST request to an API endpoint hosted by the system (e.g., http: / / localhost:5000 / predict).

[0083] Step 4:

[0084] The server receives the request and converts the JSON-formatted data into a Pandas DataFrame, which makes it easier to manipulate the data in your program.

[0085] Step 5:

[0086] The server splits the data frame into target variables and features. The target variables are the objects that the user wants to predict or classify.

[0087] Step 6:

[0088] The server splits the data into training and testing data, typically assigning 80% of the data to training data and 20% to testing data.

[0089] Step 7:

[0090] The server selects the appropriate machine learning model depending on the type of problem: a random forest classifier for classification problems, and a random forest regressor for regression problems.

[0091] Step 8:

[0092] The server trains the selected model using the training data, using the features and the target variable.

[0093] Step 9:

[0094] The server uses the trained model to make predictions on the test data, which produces the prediction results.

[0095] Step 10:

[0096] The server evaluates the model's performance using the test data and calculates a score that indicates the model's accuracy.

[0097] Step 11:

[0098] The server returns the generated prediction results and evaluation scores as a response in JSON format. The response includes a list of prediction results and evaluation scores.

[0099] Step 12:

[0100] The terminal receives the response from the server and displays the prediction results and evaluation scores to the user, allowing the user to check the results of the data analysis.

[0101] Example 1

[0102] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0103] Conventional data analysis tools are difficult to use for users without specialized knowledge and are cumbersome to operate. In addition, difficulties in data preprocessing and selecting appropriate machine learning models have led to problems with reduced accuracy in prediction results and evaluation scores.

[0104] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0105] In this invention, the server includes a data input means, a means for converting the input data into a data frame, a means for dividing the converted data into features and a target variable, a means for dividing the divided data into training data and test data, a means for selecting an appropriate model depending on the type of problem, a means for training the selected model with the training data, a means for generating prediction results and evaluation scores of the model using the test data, and a means for outputting the prediction results and evaluation scores. This enables users to easily predict and classify data without specialized knowledge.

[0106] The "data input means" is an interface for accepting data from a user.

[0107] The "means for converting into a data frame" is a processing mechanism for converting received data into a tabular data structure.

[0108] "Means for splitting features and objective variables" is a function for extracting and splitting the input data (features) required for analysis and prediction and the target output data (objective variables) from a data frame.

[0109] The "means for splitting into training data and test data" is a processing function for splitting the original data into a data set for model training and a data set for evaluation.

[0110] A "model selection method" is an algorithm for selecting an appropriate machine learning model based on the characteristics of the data and the type of problem (classification or regression).

[0111] "Means for training a model" refers to a process for training a selected machine learning model using training data to improve its predictive capabilities.

[0112] The "means for generating prediction results and evaluation scores" is a function that uses a trained model to make predictions on test data and calculates the prediction results and evaluation indicators such as accuracy.

[0113] The "means for outputting prediction results and evaluation scores" is an interface for providing the generated prediction results and evaluation scores to the user.

[0114] System Overview

[0115] The system of this invention is a no-code tool that allows users without data scientist expertise to easily perform data prediction and classification. It is mainly composed of three elements: a server, a terminal, and a user.

[0116] Server-side processing

[0117] The server receives the data provided by the user and converts it into a data frame format. The specific processing is done using the Python Pandas library. From this data frame, the features and target variables are extracted separately.

[0118] Next, the server splits the data into training and test data using the train_test_split function from scikit-learn. Depending on the type of problem (classification or regression), it selects an appropriate machine learning model: for classification, it selects a random forest classifier, and for regression, it selects a random forest regressor.

[0119] The selected model is trained on the training data. Then, predictions are made using the model on the test data, and prediction results and evaluation scores (e.g., metrics such as precision and recall) are generated. The generated prediction results and evaluation scores are converted into JSON format and sent to the user's device.

[0120] Operation on the terminal side

[0121] The terminal receives data and configuration information (such as classification and prediction requests) from the user, converts it into a specific format, and sends it to the server as a POST request. A simple user interface is provided, allowing the user to operate it intuitively.

[0122] After receiving the prediction results and evaluation scores from the server, the device displays this information to the user, who can then check the results through the input interface or dashboard.

[0123] User operations

[0124] Users input data into the system by uploading a CSV file or manually entering the data. They also specify the target variable for prediction or classification and select whether the problem is classification or regression.

[0125] For example, suppose a user prepares a dataset containing age and income information and performs a classification task on purchase history (0 or 1). Then, the user inputs the dataset into the system and specifies the classification task. In this case, an appropriate model is automatically selected and trained on the server side. As a result, the user can check the prediction results and evaluation scores based on the input data.

[0126] Prompt Sentence Examples

[0127] Below are some examples of prompts to input to the generative AI model.

[0128] "Using the following dataset, create a prompt to classify purchase history (0 or 1). The dataset contains information on age and income."

[0129] This prompt allows the user to ask the generative AI for assistance with the necessary data processing and application of machine learning models.

[0130] This invention allows users to easily perform advanced data analysis without specialized knowledge.

[0131] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0132] Step 1:

[0133] The server receives data entered by the user. The input is sent in the form of a POST request from the terminal. Specifically, this is done by the user uploading a CSV file or entering data directly into a form. The received data is temporarily stored so that it can be passed on to the next processing step as is.

[0134] Step 2:

[0135] The server converts the received data into a data frame using the Pandas library. At this time, the received data (e.g., in CSV format) is read using the Pandas read_csv function and converted into a data frame. The input is the contents of the CSV file, and the output is a Pandas data frame object.

[0136] Step 3:

[0137] The server separates the features and the response variable from the data frame. Specifically, it uses the column names of the data frame to extract the columns designated as response variables and separates the other columns as features. This allows the feature (X) and response variable (y) to be obtained from the input data (all data frames).

[0138] Step 4:

[0139] The server splits the data into training data and test data using the scikit-learn train_test_split function. The input is the features and the objective variable, and the output is the training dataset (X_train, y_train) and the test dataset (X_test, y_test). The split ratio is generally set to 80:20 by default.

[0140] Step 5:

[0141] The server determines the problem type and selects the appropriate machine learning model: the input is the problem type (classification or regression) specified by the user, and the output is the corresponding model object (e.g., a random forest classifier for classification, or a random forest regressor for regression).

[0142] Step 6:

[0143] The server trains the selected model using the training data. Specifically, it uses the model's fit method to input the training data (X_train, y_train) and output the trained model. In this step, the model parameters are optimized.

[0144] Step 7:

[0145] The server uses the trained model to make predictions on the test data. The input is the trained model and test data (X_test), and the output is the prediction result. Specifically, the server makes predictions using the model's predict method.

[0146] Step 8:

[0147] The server generates prediction results and evaluation scores. The inputs are the prediction results and the actual values ​​of the test data (y_test), and the output is the evaluation score (e.g., precision or recall). Specifically, the score is calculated using the evaluation function of scikit-learn.

[0148] Step 9:

[0149] The server converts the generated prediction results and evaluation scores into JSON format and sends them to the terminal. The terminal parses the received JSON data and displays it to the user. The input is the prediction results and evaluation scores, and the output is the information displayed in the user interface.

[0150] Step 10:

[0151] The user checks the prediction results and evaluation scores displayed on the device, and decides on the next action based on the results. Specifically, the results are displayed in the form of a dashboard or report, so the user only needs to refer to this.

[0152] (Application example 1)

[0153] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0154] Predicting failures in factory equipment and robots and efficiently performing maintenance work has a significant impact on reducing downtime and improving productivity. However, conventional predictive maintenance systems require advanced data science knowledge and experts, making them difficult for ordinary workers to use. As a result, there were issues such as not being able to perform maintenance at the appropriate time and making it difficult to prevent failures before they occur.

[0155] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0156] In this invention, the server includes a data input means, a means for dividing the input data into features and a target variable, a means for dividing the divided data into training data and test data, a means for selecting an appropriate machine learning model depending on the type of problem, a means for training the selected model with the training data, a means for generating prediction results and evaluation scores of the model using the test data, a means for outputting the prediction results and evaluation scores, and a means for receiving sensor data and performing pattern recognition and predictive maintenance. This enables even users without specialized knowledge to easily predict failures in equipment and robots in factories and perform maintenance work at appropriate times.

[0157] "Data input means" refers to a device or interface that accepts data input by a user.

[0158] A "feature" is an individual attribute or variable extracted from data for the purpose of making a prediction or classification.

[0159] The "dependent variable" refers to the data output as a result of prediction or classification, and is often called the "target."

[0160] "Training data" refers to the dataset used to train a machine learning model.

[0161] "Test data" refers to a dataset used to evaluate the performance of a trained machine learning model.

[0162] A "machine learning model" refers to an algorithm or mathematical model that makes predictions or classifications based on data.

[0163] "Prediction result" refers to the result derived by the machine learning model based on the input data.

[0164] An "evaluation score" is an index used to quantitatively evaluate the predictive performance of a machine learning model.

[0165] "Sensor Data" means physical or environmental data collected from a sensor device.

[0166] "Pattern recognition" refers to the technique of identifying specific schematic patterns or trends in data.

[0167] "Predictive maintenance" refers to the use of data analysis to make predictions and take appropriate maintenance actions to prevent equipment and system failures and malfunctions.

[0168] This invention is a no-code tool that enables users without the specialized knowledge of data scientists to easily predict and classify data. As an application example, we will explain how to realize a predictive maintenance system installed in a factory robot.

[0169] System Overview

[0170] We provide a system in which a server, a terminal, and a user work together to perform the following operations:

[0171] Hardware and Software

[0172] Hardware

[0173] Various sensors inside the robot (e.g. temperature, vibration, pressure, etc.)

[0174] Centralized control server in the factory

[0175] Factory manager's device (smartphone or PC)

[0176] software

[0177] Python

[0178] Pandas, Scikit-learn (machine learning library)

[0179] Flask (web server framework)

[0180] MQTT (sensor data communication protocol)

[0181] JSON (data format)

[0182] Server-side processing

[0183] The server processes the data as follows:

[0184] 1. Data reception: Receive data sent from each sensor using the MQTT protocol and convert it into a Pandas DataFrame.

[0185] 2. Data preprocessing: Completing missing and outlier values ​​and normalizing data.

[0186] 3. Data splitting: Split the data into features and target variables, and then split it into training data and test data.

[0187] 4. Model selection and training: Select an appropriate machine learning model (e.g., random forest, support vector machine, etc.) depending on the type of problem and train it on the training data.

[0188] 5. Prediction and Evaluation: The test data is used to generate predictions and evaluation scores for the model.

[0189] 6. Result output: The generated prediction results and evaluation scores are generated in JSON format and sent to the terminal.

[0190] Operation on the terminal side

[0191] The terminal performs the following actions:

[0192] 1. User input acceptance: Accepts data (e.g., CSV file upload) and prediction and classification settings as input from the factory manager.

[0193] 2. Send request: The entered data and settings are converted into a specified format and sent to the server as a POST request.

[0194] 3. Receiving and displaying results: The prediction results and evaluation scores returned from the server are received and displayed to the user.

[0195] User operations

[0196] The specific operation steps are as follows:

[0197] 1. Data input: Data acquired from sensors in the factory is input into the system. For example, prepare the following data set:

[0198] Temperature: 35.4, 35.6, 36.0, 36.2, 35.9

[0199] Vibration: 0.05, 0.07, 0.08, 0.09, 0.07

[0200] Pressure: 1.1, 1.2, 1.3, 1.4, 1.2

[0201] Fault history: 0, 0, 1, 0, 0

[0202] 2. Example prompt:

[0203] "Based on the following data, please predict whether or not there will be a breakdown within a week.

[0204] Temperature: 35.4, 35.6, 36.0, 36.2, 35.9

[0205] Vibration: 0.05, 0.07, 0.08, 0.09, 0.07

[0206] Pressure: 1.1, 1.2, 1.3, 1.4, 1.2

[0207] Fault history: 0, 0, 1, 0, 0"

[0208] 3. Prediction and maintenance: The server selects a machine learning model based on the input data, performs failure prediction, and returns the results to the user. The user then performs appropriate maintenance work based on the prediction results.

[0209] This allows even users without specialized knowledge to easily predict failures in equipment and robots within a factory, enabling maintenance work to be carried out at the appropriate time.

[0210] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0211] Step 1:

[0212] Data reception

[0213] The server receives data sent from sensors in the factory using the MQTT protocol. The received data is converted into a Pandas DataFrame. Here, the input is sensor data, and the output is data in Pandas DataFrame format. This conversion makes it easier to handle the data.

[0214] Step 2:

[0215] Data Preprocessing

[0216] The server receives a Pandas DataFrame and performs missing value imputation, outlier processing, and data normalization. The input is raw data in Pandas DataFrame format, and the output is preprocessed data. This preprocessing improves the quality of the data and increases the accuracy of the machine learning model.

[0217] Step 3:

[0218] Data partitioning

[0219] The server splits the preprocessed data into features and targets, and then splits it into training data and test data. The input is the preprocessed data, and the output is a training dataset and a test dataset. This splitting enables training and evaluation of machine learning models.

[0220] Step 4:

[0221] Model Selection and Training

[0222] The server selects an appropriate machine learning model depending on the type of problem (classification or regression) and trains the model using the training data. The input is the training dataset, and the output is the trained machine learning model. This step enables prediction and classification based on the data.

[0223] Step 5:

[0224] Prediction and Evaluation

[0225] The server uses the trained model to make predictions on the test data and generates prediction results and evaluation scores. The inputs are the trained model and the test dataset, and the outputs are the prediction results and evaluation scores. This allows the performance of the model to be quantitatively evaluated.

[0226] Step 6:

[0227] Result output

[0228] The server generates the generated prediction results and evaluation scores in JSON format and sends them to the terminal. The input is the prediction results and evaluation scores, and the output is JSON format data. This step allows the user to easily check the results.

[0229] Step 7:

[0230] Accepting User Input

[0231] The terminal accepts data (e.g., uploading a CSV file) and prediction and classification settings as input from the factory manager. The input is data and settings from the user, and the output is a request to the server. At this step, the user can operate the system and start executing prediction and classification tasks.

[0232] Step 8:

[0233] Send request

[0234] The terminal converts the input data and settings into a specified format and sends it to the server as a POST request. The input is data and settings from the user, and the output is a request to the server. In this step, the terminal requests the server to process the data.

[0235] Step 9:

[0236] Receive and display results

[0237] The terminal receives the prediction results and evaluation scores returned from the server and displays them to the user. The input is JSON-formatted data from the server, and the output is the results displayed to the user. This allows the user to check the prediction results and take appropriate maintenance action.

[0238] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0239] This invention is a system that combines a no-code tool that enables users without specialized data scientist knowledge to easily predict and classify data with an emotion engine that recognizes user emotions. The incorporation of the emotion engine enables data processing and display of results based on the user's emotions, providing a more user-friendly experience.

[0240] System Overview

[0241] When a user inputs data and requests, the server automatically selects an appropriate machine learning model based on the data, and then trains and evaluates it. In addition, an emotion engine recognizes the user's emotions and adjusts data preprocessing and result display based on the emotions. The results are returned to the user via the device.

[0242] Server-side processing

[0243] The server processes the data as follows:

[0244] 1. Receive data entered by the user and convert it to a Pandas DataFrame.

[0245] 2. The emotion engine recognizes the user's emotion and acquires the emotion data.

[0246] 3. Split the target variable and features from the data frame.

[0247] 4. Split the data into training and test data, which may be adjusted based on user sentiment.

[0248] 5. Select the appropriate machine learning model for the type of problem. The sentiment engine adjusts model selection as needed.

[0249] 6. Train the selected model on the training data.

[0250] 7. Use the test data to generate model predictions and evaluation scores.

[0251] 8. The generated prediction results and evaluation scores are adjusted according to the user's sentiment and output in an appropriate format.

[0252] Emotion Engine Operation

[0253] The emotion engine does the following:

[0254] 1. Recognize the user's emotions in real time while they are typing, for example by reading facial expressions through a camera or analyzing the tone of voice through voice input.

[0255] 2. Optimize data preprocessing based on the recognized emotion. For example, if a negative emotion is recognized, the system will perform more careful error detection.

[0256] 3. Adjust the display of the final prediction result and evaluation score depending on the emotion. If a positive emotion is recognized, the result display can be simplified, while if a negative emotion is recognized, a detailed explanation can be added.

[0257] Operation on the terminal side

[0258] The terminal performs the following actions:

[0259] 1. Accepts data and prediction or classification settings from the user as input.

[0260] 2. Convert the input data into JSON format and send it to the server as a POST request.

[0261] 3. The emotion engine is used to obtain the user's emotions in real time, and this information is also sent to the server.

[0262] 4. Receive the prediction results and evaluation scores returned from the server.

[0263] 5. Display the received results in an appropriate format based on the user's sentiment.

[0264] User operations

[0265] The user does the following:

[0266] 1. Input data into the system. Emotion recognition is automatic.

[0267] 2. Specify the target variable and problem type for prediction or classification.

[0268] 3. The system processes the data on the server side and waits for the results to be returned. During this time, emotion recognition is also performed in parallel.

[0269] 4. The results are displayed on the device, where you can check the prediction results and evaluation score, and receive emotional feedback.

[0270] Specific examples

[0271] For example, suppose a user prepares the following dataset and performs a classification task:

[0272] Example dataset:

[0273] Feature 1: Age

[0274] Feature 2: Income

[0275] target: purchase history (0 or 1)

[0276] When a user enters this data into the system and selects a classification task, the following occurs:

[0277] 1. Data is received and converted into a DataFrame.

[0278] 2. The emotion engine recognizes the user's emotions and acquires the data.

[0279] 3. Age and income are set as features, and purchase history is set as the objective variable.

[0280] 4. Split the dataset into training and test data, possibly with adjustments based on emotion recognition.

[0281] 5. The server selects a random forest classifier for the classification problem and trains the model on the training data.

[0282] 6. Use the trained model to make predictions on the test data and generate results and evaluation scores.

[0283] 7. The generated prediction results and evaluation scores are adjusted according to the user's sentiment and sent back to the device.

[0284] 8. The device displays the prediction results and evaluation scores to the user. Positive emotions are displayed in a simple format, while negative emotions are displayed with a detailed explanation.

[0285] This allows users to predict and classify data without specialized knowledge, and also receive appropriate feedback based on their emotions.

[0286] The processing flow will be explained below.

[0287] Step 1:

[0288] The user inputs data into the system. The data can be uploaded as a CSV file or entered directly into the system's input form. In addition, the target variable for prediction or classification and the type of problem (classification or regression) are specified.

[0289] Step 2:

[0290] The terminal receives the data entered by the user and converts it into JSON format, which includes features, target variables, and problem types.

[0291] Step 3:

[0292] The device sends JSON format data to the server as a POST request, which also includes emotion data for real-time recognition of the user's emotions.

[0293] Step 4:

[0294] The server receives the request and converts the JSON-formatted data into a Pandas DataFrame, which makes it easier to manipulate the data in your program.

[0295] Step 5:

[0296] The server splits the data frame into target variables and features. The target variables are the objects that the user wants to predict or classify.

[0297] Step 6:

[0298] The server splits the data into training and testing data, typically assigning 80% of the data to training data and 20% to testing data.

[0299] Step 7:

[0300] The emotion engine recognizes the user's emotions in real time as they are being input, and sends the emotion data to the server using techniques such as facial expression recognition and voice analysis.

[0301] Step 8:

[0302] The server selects an appropriate machine learning model depending on the type of problem: a random forest classifier for classification problems, a random forest regressor for regression problems, and model selection and hyperparameter tuning depending on the emotion data.

[0303] Step 9:

[0304] The server trains the selected model using the training data, using the features and the target variable.

[0305] Step 10:

[0306] The server uses the trained model to make predictions on the test data, which produces the prediction results.

[0307] Step 11:

[0308] The server evaluates the model's performance using the test data and calculates a score that indicates the model's accuracy.

[0309] Step 12:

[0310] The emotion engine analyzes the user's emotion data and adjusts the way predictions and rating scores are displayed: simple for positive emotions, and detailed explanations for negative emotions.

[0311] Step 13:

[0312] The server returns the generated prediction results, evaluation scores, and emotion-based adjustment information as a response in JSON format. The response includes a list of prediction results and evaluation scores.

[0313] Step 14:

[0314] The device receives the response from the server and displays the prediction results and evaluation scores to the user, providing tailored feedback based on the user's emotions.

[0315] As a concrete example, let's say a user prepares the following dataset and performs a classification task:

[0316] Example dataset:

[0317] Feature 1: Age

[0318] Feature 2: Income

[0319] target: purchase history (0 or 1)

[0320] When a user enters this data into the system and selects a classification task, the following occurs:

[0321] Steps 1-3:

[0322] The user inputs data, the emotion engine recognizes emotions in real time, and the device sends this data and emotion information to the server.

[0323] Steps 4-6:

[0324] The server receives the data, converts it into a DataFrame, splits it into target variables and features, and then splits the dataset into training data and test data.

[0325] Steps 7-12:

[0326] The server continues to process the data in cooperation with the emotion engine, selecting a random forest classifier for the classification problem and training the model on the training data, then making predictions using the test data to generate results and evaluation scores.

[0327] Steps 13-14:

[0328] The emotion engine adjusts according to the user's emotions and returns the prediction results and evaluation score to the device, which receives and displays them to the user. In this case, a simple display is provided for positive emotions, and a detailed explanation is provided for negative emotions.

[0329] This allows users to predict and classify data without specialized knowledge, and also receive appropriate feedback based on their emotions.

[0330] Example 2

[0331] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0332] Conventional data analysis systems require the specialized knowledge of data scientists, making them difficult for general users to use. Furthermore, they do not provide a user-friendly experience because they do not take into account the user's feelings when processing data or displaying results. This can lead to stress and frustration when users use the system.

[0333] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0334] In this invention, the server includes a data input means, a means for dividing the input data into features and a response variable, a means for dividing the divided data into training data and test data, a means for selecting an appropriate machine learning model according to the type of problem, a means for training the selected model with the training data, a means for generating prediction results and an evaluation score of the model using the test data, a means for outputting the prediction results and the evaluation score, a means for recognizing user emotions in real time, and a means for adjusting data preprocessing and result display based on the user emotions. This enables general users without the specialized knowledge of a data scientist to easily perform data analysis and receive feedback according to the user's emotions.

[0335] "Data input means" refers to a device or function for capturing data input by a user into the system.

[0336] A "feature" is an element of data that a machine learning model uses to make a prediction or classification.

[0337] A "target variable" is an element of data that a machine learning model generates as a prediction or classification result.

[0338] "Training data" is a data set used to train a machine learning model.

[0339] "Test data" is a dataset used to evaluate the performance of a trained machine learning model.

[0340] A "machine learning model" is a computational method or algorithm that learns patterns and regularities from data and makes predictions and classifications for new data.

[0341] The "emotion recognition means" is a device or function for recognizing the user's emotions in real time.

[0342] "Preprocessing means" refers to a device or function that formats and transforms data to make it easier to analyze data or train machine learning models.

[0343] "Feedback" refers to information provided to the user by the system regarding prediction results and evaluation scores, in order to promote the user's understanding and next action.

[0344] This invention is a system that incorporates an emotion engine that recognizes user emotions into a no-code tool that enables general users without the specialized knowledge of a data scientist to easily perform data analysis. This system allows users to receive feedback according to their emotions, providing a more user-friendly experience.

[0345] Server-side hardware and software

[0346] The server of this system uses the following hardware and software:

[0347] Hardware: A server machine with a powerful CPU and sufficient memory

[0348] software:

[0349] Pandas: Used to receive and transform data

[0350] scikit-learn: Used for data splitting, model selection, and model training

[0351] Emotion Engine: Recognize user emotions in real time

[0352] Process Overview

[0353] When a user inputs data and prediction or classification settings, the server automatically selects an appropriate machine learning model based on the data, and then trains and evaluates it. In addition, an emotion engine recognizes the user's emotions and adjusts data preprocessing and result display based on the emotions. The results are returned to the user via the device.

[0354] Terminal hardware and software

[0355] The terminals of this system use the following hardware and software:

[0356] Hardware: Devices with a camera and microphone (e.g., PC, tablet, smartphone)

[0357] software:

[0358] Front-end application: data entry and result display

[0359] Emotion Recognition API: Acquire emotion data from camera and voice input

[0360] System Operation

[0361] 1. Data Entry

[0362] A user uses a terminal to input data, for example, values ​​such as "age" and "income," and specifies a classification task.

[0363] 2. Emotion recognition

[0364] The device uses a built-in camera and microphone to recognize the user's emotions in real time and transmits the emotion data to a server.

[0365] 3. Data Transmission

[0366] The terminal converts the input data into JSON format and sends it to the server.

[0367] 4. Model Selection and Training

[0368] The server converts the received data into a Pandas DataFrame and uses scikit-learn to select and train an appropriate machine learning model. At the same time, the emotion engine retrieves the emotion data and adjusts data preprocessing as needed.

[0369] 5. Prediction and Evaluation

[0370] The server makes predictions on the test data and generates evaluation scores, which are adjusted based on the emotion data.

[0371] 6. Displaying the results

[0372] The device displays the results received from the server in an appropriate format depending on the user's emotions. For example, if the user has a positive emotion, the results are displayed simply, and if the user has a negative emotion, the results are displayed with a detailed explanation.

[0373] Specific examples

[0374] For example, suppose a user has the following dataset and is tasked with classification:

[0375] Example dataset:

[0376] Feature 1: Age

[0377] Feature 2: Income

[0378] target: purchase history (0 or 1)

[0379] Once the user enters this data into the system and selects a classification task, the system operates as follows:

[0380] 1. Data is received and converted into a DataFrame

[0381] The server receives the age and income data and converts it into a Pandas DataFrame.

[0382] 2. Emotion Recognition and Data Segmentation

[0383] The emotion engine recognizes the user's smile and collects the data. The target variable is purchase history, and age and income are divided as features.

[0384] 3. Model training and evaluation

[0385] The server selects and trains a random forest, generating predictions and evaluation scores for the test data.

[0386] 4. Adjusting and displaying the results

[0387] The server adjusts the results according to the user's emotions and sends them back to the device. The results are displayed simply with positive emotions.

[0388] Examples of prompt statements

[0389] "I want to classify data based on age, income, and purchase history. Please let me know the prediction results and evaluation scores."

[0390] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0391] Processing Steps

[0392] Step 1:

[0393] Data Entry

[0394] The user inputs data from the terminal. Specifically, the user inputs feature quantities such as "age" and "income" and objective variables such as "purchase history."

[0395] Input: Age (e.g., 30), Income (e.g., 50000), Purchase History (e.g., 1)

[0396] Output: Input data on the terminal

[0397] Step 2:

[0398] Sending input data

[0399] The terminal converts the input data into JSON format and sends it to the server as a POST request.

[0400] Input: Data entered by the user

[0401] Output: JSON data sent to the server ({"age": 30, "income": 50000, "purchase_history": 1})

[0402] Step 3:

[0403] Acquiring and sending emotion data

[0404] The device uses a camera and microphone to capture the user's emotional data in real time and transmits it to a server.

[0405] Input: Emotion data obtained through a camera or microphone

[0406] Output: Emotion data sent to the server (e.g., {"emotion": "positive"})

[0407] Step 4:

[0408] Receiving and Converting Data

[0409] The server receives the JSON data sent from the terminal and converts it into a DataFrame using the Pandas read_json method.

[0410] Input: JSON data sent from the terminal

[0411] Output: Pandas DataFrame format data

[0412] Step 5:

[0413] emotion recognition

[0414] The emotion engine analyzes the emotion data sent from the server and recognizes the user's emotions in real time.

[0415] Input: Emotion data sent from the server

[0416] Output: Recognized emotion data (e.g., {"emotion": "positive"})

[0417] Step 6:

[0418] Splitting data into features and target variables

[0419] The server splits the features (age, income) and the target variable (purchase history) from the DataFrame.

[0420] Input: Pandas DataFrame format data

[0421] Output: Data split into features and target variables (e.g., features: age and income, target variable: purchase history)

[0422] Step 7:

[0423] Splitting the data into training and test

[0424] The server splits the data into training and test data using the train_test_split function, adjusting the shuffling of the data based on the sentiment data if necessary.

[0425] Input: Data split into features and target variables

[0426] Output: training data and test data (e.g., training data: 80%, test data: 20%)

[0427] Step 8:

[0428] Model selection and adjustment

[0429] The server selects the appropriate machine learning model depending on the type of problem, and may adjust the model selection based on emotion data. For example, random forest is selected for classification problems.

[0430] Input: training data, problem type, emotion data

[0431] Output: The selected machine learning model (e.g., Random Forest)

[0432] Step 9:

[0433] Training the model

[0434] The server uses the selected model to train the training data. Run model.fit(X_train, y_train).

[0435] Input: Training data, selected machine learning model

[0436] Output: A trained machine learning model

[0437] Step 10:

[0438] Prediction and Evaluation

[0439] The server makes predictions using the test data and generates evaluation scores using accuracy_score, confusion_matrix, etc. Run model.predict(X_test).

[0440] Input: trained machine learning model, test data

[0441] Output: Prediction results and evaluation scores (e.g. accuracy, confusion matrix)

[0442] Step 11:

[0443] Adjusting and outputting prediction results

[0444] The server adjusts the prediction results and evaluation scores according to the user's emotions and returns them to the device in an appropriate format, for example, adding a more detailed explanation if the emotion is negative.

[0445] Input: Prediction results, rating scores, sentiment data

[0446] Output: JSON data of adjusted prediction results and evaluation scores

[0447] Step 12:

[0448] Displaying prediction results

[0449] The device displays the prediction results and evaluation scores received from the server in an appropriate format depending on the user's emotions. For example, it displays them simply if the emotion is positive, and in detail if the emotion is negative.

[0450] Input: Adjusted prediction results and evaluation scores sent from the server

[0451] Output: Prediction results and evaluation scores displayed on the device

[0452] (Application example 2)

[0453] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0454] Conventional data prediction and classification systems lack user experience because they do not take user emotions into account when processing or displaying results. Additionally, users without the expertise of a data scientist find it difficult to use the systems.

[0455] The specification processing by the specification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a data input means, a means for dividing the input data into features and a response variable, a means for dividing the divided data into training data and test data, a means for selecting an appropriate machine learning model according to the type of problem, a means for training the selected model with the training data, a means for generating prediction results and evaluation scores of the model using the test data, a means for outputting the prediction results and evaluation scores, an emotion recognition means for recognizing a user's emotions in real time, and a means for adjusting data preprocessing and result display using emotion data acquired by the emotion recognition means. This enables data processing and result display according to the user's emotions, providing a more intuitive and user-friendly system.

[0456] "Data input means" refers to a device or interface that accepts data input by a user.

[0457] "Features" refer to important attributes extracted from data that are used for analysis such as prediction and classification.

[0458] A "target variable" refers to a specific value or label of data that is the target of prediction or classification.

[0459] "Splitting means" refers to a device or algorithm that performs processing to split input data into features and target variables.

[0460] "Training data" refers to the data set used to train a machine learning model.

[0461] "Test data" refers to a dataset that has not been used for training but is used to evaluate the performance of a trained machine learning model.

[0462] "Machine learning model selection means" refers to a device or algorithm for selecting an appropriate machine learning model based on the type of problem given.

[0463] "Model training means" refers to a device or algorithm that trains a selected machine learning model with training data.

[0464] "Prediction result generation means" refers to a device or algorithm that uses test data to generate prediction results and evaluation scores for a trained machine learning model.

[0465] "Output means" refers to a device or interface for presenting the prediction results and evaluation scores to the user in an appropriate format.

[0466] "Emotion recognition means" refers to a device or algorithm for recognizing a user's emotions in real time.

[0467] "Emotion data utilization means" refers to a device or algorithm that utilizes the emotion data acquired by the emotion recognition means to adjust data preprocessing and result display.

[0468] The present invention combines a data prediction and classification system with a user emotion recognition function, allowing users to experience data processing and result display according to their emotions, providing a more intuitive and user-friendly interface.

[0469] System Overview

[0470] The system consists of a data input means, a means for dividing features and target variables, a means for dividing training data and test data, a means for selecting a machine learning model, a means for training the model, a means for generating prediction results and evaluation scores, an output means, a means for recognizing emotions, and a means for using emotion data.

[0471] Hardware and software used

[0472] The system uses the following hardware and software:

[0473] 1. Hardware:

[0474] Webcam: Captures your facial expressions in real time.

[0475] Computer: Handles all processing.

[0476] 2. Software:

[0477] Python: An entire programming language.

[0478] OpenCV: A library for processing video from a webcam.

[0479] FER (Facial Expression Recognition) Library: A library for recognizing user emotions.

[0480] Pandas: A library for data preprocessing.

[0481] Scikit-learn: A library for machine learning models.

[0482] Requests: A library that communicates with the API.

[0483] Processing flow

[0484] server

[0485] The server does the following:

[0486] 1. Receive data entered by the user and convert it into a data frame using Pandas.

[0487] 2. Emotion data of the user is acquired by the emotion recognition means.

[0488] 3. Split the features and objective variables from the data frame and divide it into training data and test data.

[0489] 4. Select an appropriate machine learning model and train the selected model using the training data.

[0490] 5. Use the trained model to make predictions on the test data and generate an evaluation score.

[0491] 6. Use sentiment data to refine prediction results and evaluation scores into appropriate formats.

[0492] 7. Send the results back to the device.

[0493] Terminal

[0494] The terminal does the following:

[0495] 1. Accepts data and prediction / classification settings from the user, converts the input data into JSON format, and sends it to the server.

[0496] 2. Capture user emotion data in real time using a webcam and send it to the server.

[0497] 3. Receive the prediction results and evaluation scores returned from the server and display them in an appropriate format based on the user's sentiment.

[0498] User

[0499] The user does the following:

[0500] 1. Data is input into the system and emotion recognition is done automatically.

[0501] 2. Specify the target variable and problem type for prediction or classification.

[0502] 3. Wait for the server to process the data, and the results will be displayed on the device.

[0503] 4. Check your prediction results and evaluation scores and get feedback.

[0504] Specific examples

[0505] For example, if a user prepares purchase history data and performs a classification task, age and income are the features, and purchase history is the target variable. The system processes it as follows:

[0506] 1. Data is received and converted into a Pandas dataframe.

[0507] 2. The user's emotion data is acquired by the FER library.

[0508] 3. The data is split into features and target variables, and then split into training data and test data.

[0509] 4. A random forest classifier is selected and trained on the training data.

[0510] 5. The trained model is used to make predictions on the test data and generate an evaluation score.

[0511] 6. Taking into account the emotion data, the prediction results and evaluation scores are adjusted and sent back to the device.

[0512] 7. The prediction results and evaluation scores are displayed to the user.

[0513] Example prompt sentence:

[0514] Create a system like this: An application for an online shopping site that uses a webcam to recognize user emotions in real time and recommends appropriate products based on those emotions. You will use OpenCV and FER library to recognize user emotions, and the recommender system will use a machine learning model to recommend products based on the user and emotion data.

[0515] The above is a specific embodiment of the present invention. This system enables emotion-based data preprocessing and result display, allowing users to enjoy a more intuitive interface.

[0516] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0517] Step 1:

[0518] The terminal accepts data and prediction or classification settings from the user. The terminal converts the input data into JSON format and sends it to the server as a POST request. The input in this step is user-entered data, and the output is JSON-formatted data.

[0519] Step 2:

[0520] The device captures the user's facial expressions in real time using a webcam. The FER library is used to recognize the user's emotions from the captured video. The recognized emotion data is also sent to the server. The input in this step is the real-time video, and the output is the user's emotion data.

[0521] Step 3:

[0522] The server receives JSON-formatted data sent from the terminal and converts it into a data frame using Pandas. It then extracts features and target variables from the data frame. The input in this step is JSON-formatted data, and the output is a data frame split into features and target variables.

[0523] Step 4:

[0524] The server splits the data divided into features and objective variables into training data and test data. At the same time, it also performs preprocessing based on emotion data. The input in this step is the data frame divided into features and objective variables and emotion data, and the output is training data and test data.

[0525] Step 5:

[0526] The server selects an appropriate machine learning model depending on the problem type, for example, a random forest classifier for a classification problem. The input in this step is the problem type of the data, and the output is the selected machine learning model.

[0527] Step 6:

[0528] The server uses the selected machine learning model to train the model on the training data. The inputs in this step are the training data and the machine learning model, and the output is the trained model.

[0529] Step 7:

[0530] The server uses the trained model to predict the test data and generate an evaluation score. The inputs in this step are the test data and the trained model, and the outputs are the prediction results and the evaluation score.

[0531] Step 8:

[0532] The server uses the emotion data to adjust the prediction result and evaluation score according to the emotion. For example, if the user expresses a negative emotion, it adds a detailed explanation. The inputs in this step are the prediction result, the evaluation score, and the emotion data, and the output is the adjusted prediction result and evaluation score.

[0533] Step 9:

[0534] The device receives the prediction results and evaluation scores returned from the server and displays them in an appropriate format depending on the user's emotions. For example, a simple display for positive emotions and a detailed explanation for negative emotions. The input in this step is the adjusted prediction results and evaluation scores, and the output is the result displayed to the user.

[0535] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0536] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0537] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0538] [Second embodiment]

[0539] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0540] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0541] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0542] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0543] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0544] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0545] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0546] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0547] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0548] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0549] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0550] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0551] This invention is a no-code tool that allows users without the specialized knowledge of data scientists to easily predict and classify data.

[0552] System Overview

[0553] When a user inputs their data and request, the server automatically selects an appropriate machine learning model based on that data, trains it, and evaluates it, returning the results to the user via their device.

[0554] Server-side processing

[0555] The server processes the data as follows:

[0556] 1. Receive data entered by the user.

[0557] 2. Convert the received data into a Pandas DataFrame.

[0558] 3. Split the target variable and features.

[0559] 4. Split the data into training and test data.

[0560] 5. Select the appropriate model depending on the type of problem (classification or regression).

[0561] 6. Train the selected model on the training data.

[0562] 7. Use the test data to generate model predictions and evaluation scores.

[0563] 8. Output the generated prediction results and evaluation scores in JSON format.

[0564] Operation on the terminal side

[0565] The terminal performs the following actions:

[0566] 1. Accepts data and prediction or classification settings from the user as input.

[0567] 2. Convert the input data into a specified format and send it to the server as a POST request.

[0568] 3. Receive the prediction results and evaluation scores returned from the server.

[0569] 4. Display the received results to the user.

[0570] User operations

[0571] The user does the following:

[0572] 1. Enter data into the system (e.g., upload a CSV file or enter data manually).

[0573] 2. Specify the target variable for prediction or classification and the type of problem (classification or regression).

[0574] 3. The system processes the request on the server side and waits for the results to be returned.

[0575] 4. The results will be displayed on your device, and you can check the prediction results and evaluation score.

[0576] Specific examples

[0577] For example, suppose a user prepares the following dataset and performs a classification task:

[0578] Example dataset:

[0579] Feature 1: Age

[0580] Feature 2: Income

[0581] target: purchase history (0 or 1)

[0582] Once the user enters this data into the system and selects a classification task, the following happens automatically on the server side:

[0583] 1. Data is received and converted into a DataFrame.

[0584] 2. Age and income are set as features, and purchase history is set as the target variable.

[0585] 3. Split the dataset into training and test data.

[0586] 4. The server selects a random forest classifier for the classification problem and trains the model on the training data.

[0587] 5. The trained model is used to make predictions on the test data, generating results and evaluation scores.

[0588] 6. The generated prediction results and evaluation scores are sent back to the user's device and displayed on the screen.

[0589] This allows users to predict and classify data without specialized knowledge, making it easy to make data-based decisions.

[0590] The processing flow will be explained below.

[0591] Step 1:

[0592] The user inputs data into the system. The data can be uploaded as a CSV file or entered directly into the system's input form. This input specifies the target variable for prediction or classification and the type of problem (classification or regression).

[0593] Step 2:

[0594] The terminal receives the data entered by the user and converts it into JSON format, which includes features, target variables, and problem types.

[0595] Step 3:

[0596] The device sends JSON-formatted data to the server as a POST request to an API endpoint hosted by the system (e.g., http: / / localhost:5000 / predict).

[0597] Step 4:

[0598] The server receives the request and converts the JSON-formatted data into a Pandas DataFrame, which makes it easier to manipulate the data in your program.

[0599] Step 5:

[0600] The server splits the data frame into target variables and features. The target variables are the objects that the user wants to predict or classify.

[0601] Step 6:

[0602] The server splits the data into training and testing data, typically assigning 80% of the data to training data and 20% to testing data.

[0603] Step 7:

[0604] The server selects the appropriate machine learning model depending on the type of problem: a random forest classifier for classification problems, and a random forest regressor for regression problems.

[0605] Step 8:

[0606] The server trains the selected model using the training data, using the features and the target variable.

[0607] Step 9:

[0608] The server uses the trained model to make predictions on the test data, which produces the prediction results.

[0609] Step 10:

[0610] The server evaluates the model's performance using the test data and calculates a score that indicates the model's accuracy.

[0611] Step 11:

[0612] The server returns the generated prediction results and evaluation scores as a response in JSON format. The response includes a list of prediction results and evaluation scores.

[0613] Step 12:

[0614] The terminal receives the response from the server and displays the prediction results and evaluation scores to the user, allowing the user to check the results of the data analysis.

[0615] Example 1

[0616] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0617] Conventional data analysis tools are difficult to use for users without specialized knowledge and are cumbersome to operate. In addition, difficulties in data preprocessing and selecting appropriate machine learning models have led to problems with reduced accuracy in prediction results and evaluation scores.

[0618] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0619] In this invention, the server includes a data input means, a means for converting the input data into a data frame, a means for dividing the converted data into features and a target variable, a means for dividing the divided data into training data and test data, a means for selecting an appropriate model depending on the type of problem, a means for training the selected model with the training data, a means for generating prediction results and evaluation scores of the model using the test data, and a means for outputting the prediction results and evaluation scores. This enables users to easily predict and classify data without specialized knowledge.

[0620] The "data input means" is an interface for accepting data from a user.

[0621] The "means for converting into a data frame" is a processing mechanism for converting received data into a tabular data structure.

[0622] "Means for splitting features and objective variables" is a function for extracting and splitting the input data (features) required for analysis and prediction and the target output data (objective variables) from a data frame.

[0623] The "means for splitting into training data and test data" is a processing function for splitting the original data into a data set for model training and a data set for evaluation.

[0624] A "model selection method" is an algorithm for selecting an appropriate machine learning model based on the characteristics of the data and the type of problem (classification or regression).

[0625] "Means for training a model" refers to a process for training a selected machine learning model using training data to improve its predictive capabilities.

[0626] The "means for generating prediction results and evaluation scores" is a function that uses a trained model to make predictions on test data and calculates the prediction results and evaluation indicators such as accuracy.

[0627] The "means for outputting prediction results and evaluation scores" is an interface for providing the generated prediction results and evaluation scores to the user.

[0628] System Overview

[0629] The system of this invention is a no-code tool that allows users without data scientist expertise to easily perform data prediction and classification. It is mainly composed of three elements: a server, a terminal, and a user.

[0630] Server-side processing

[0631] The server receives the data provided by the user and converts it into a data frame format. The specific processing is done using the Python Pandas library. From this data frame, the features and target variables are extracted separately.

[0632] Next, the server splits the data into training and test data using the train_test_split function from scikit-learn. Depending on the type of problem (classification or regression), it selects an appropriate machine learning model: for classification, it selects a random forest classifier, and for regression, it selects a random forest regressor.

[0633] The selected model is trained on the training data. Then, predictions are made using the model on the test data, and prediction results and evaluation scores (e.g., metrics such as precision and recall) are generated. The generated prediction results and evaluation scores are converted into JSON format and sent to the user's device.

[0634] Operation on the terminal side

[0635] The terminal receives data and configuration information (such as classification and prediction requests) from the user, converts it into a specific format, and sends it to the server as a POST request. A simple user interface is provided, allowing the user to operate it intuitively.

[0636] After receiving the prediction results and evaluation scores from the server, the device displays this information to the user, who can then check the results through the input interface or dashboard.

[0637] User operations

[0638] Users input data into the system by uploading a CSV file or manually entering the data. They also specify the target variable for prediction or classification and select whether the problem is classification or regression.

[0639] For example, suppose a user prepares a dataset containing age and income information and performs a classification task on purchase history (0 or 1). Then, the user inputs the dataset into the system and specifies the classification task. In this case, an appropriate model is automatically selected and trained on the server side. As a result, the user can check the prediction results and evaluation scores based on the input data.

[0640] Prompt Sentence Examples

[0641] Below are some examples of prompts to input to the generative AI model.

[0642] "Using the following dataset, create a prompt to classify purchase history (0 or 1). The dataset contains information on age and income."

[0643] This prompt allows the user to ask the generative AI for assistance with the necessary data processing and application of machine learning models.

[0644] This invention allows users to easily perform advanced data analysis without specialized knowledge.

[0645] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0646] Step 1:

[0647] The server receives data entered by the user. The input is sent in the form of a POST request from the terminal. Specifically, this is done by the user uploading a CSV file or entering data directly into a form. The received data is temporarily stored so that it can be passed on to the next processing step as is.

[0648] Step 2:

[0649] The server converts the received data into a data frame using the Pandas library. At this time, the received data (e.g., in CSV format) is read using the Pandas read_csv function and converted into a data frame. The input is the contents of the CSV file, and the output is a Pandas data frame object.

[0650] Step 3:

[0651] The server separates the features and the response variable from the data frame. Specifically, it uses the column names of the data frame to extract the columns designated as response variables and separates the other columns as features. This allows the feature (X) and response variable (y) to be obtained from the input data (all data frames).

[0652] Step 4:

[0653] The server splits the data into training data and test data using the scikit-learn train_test_split function. The input is the features and the objective variable, and the output is the training dataset (X_train, y_train) and the test dataset (X_test, y_test). The split ratio is generally set to 80:20 by default.

[0654] Step 5:

[0655] The server determines the problem type and selects the appropriate machine learning model: the input is the problem type (classification or regression) specified by the user, and the output is the corresponding model object (e.g., a random forest classifier for classification, or a random forest regressor for regression).

[0656] Step 6:

[0657] The server trains the selected model using the training data. Specifically, it uses the model's fit method to input the training data (X_train, y_train) and output the trained model. In this step, the model parameters are optimized.

[0658] Step 7:

[0659] The server uses the trained model to make predictions on the test data. The input is the trained model and test data (X_test), and the output is the prediction result. Specifically, the server makes predictions using the model's predict method.

[0660] Step 8:

[0661] The server generates prediction results and evaluation scores. The inputs are the prediction results and the actual values ​​of the test data (y_test), and the output is the evaluation score (e.g., precision or recall). Specifically, the score is calculated using the evaluation function of scikit-learn.

[0662] Step 9:

[0663] The server converts the generated prediction results and evaluation scores into JSON format and sends them to the terminal. The terminal parses the received JSON data and displays it to the user. The input is the prediction results and evaluation scores, and the output is the information displayed in the user interface.

[0664] Step 10:

[0665] The user checks the prediction results and evaluation scores displayed on the device, and decides on the next action based on the results. Specifically, the results are displayed in the form of a dashboard or report, so the user only needs to refer to this.

[0666] (Application example 1)

[0667] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0668] Predicting failures in factory equipment and robots and efficiently performing maintenance work has a significant impact on reducing downtime and improving productivity. However, conventional predictive maintenance systems require advanced data science knowledge and experts, making them difficult for ordinary workers to use. As a result, there were issues such as not being able to perform maintenance at the appropriate time and making it difficult to prevent failures before they occur.

[0669] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0670] In this invention, the server includes a data input means, a means for dividing the input data into features and a target variable, a means for dividing the divided data into training data and test data, a means for selecting an appropriate machine learning model depending on the type of problem, a means for training the selected model with the training data, a means for generating prediction results and evaluation scores of the model using the test data, a means for outputting the prediction results and evaluation scores, and a means for receiving sensor data and performing pattern recognition and predictive maintenance. This enables even users without specialized knowledge to easily predict failures in equipment and robots in factories and perform maintenance work at appropriate times.

[0671] "Data input means" refers to a device or interface that accepts data input by a user.

[0672] A "feature" is an individual attribute or variable extracted from data for the purpose of making a prediction or classification.

[0673] The "dependent variable" refers to the data output as a result of prediction or classification, and is often called the "target."

[0674] "Training data" refers to the dataset used to train a machine learning model.

[0675] "Test data" refers to a dataset used to evaluate the performance of a trained machine learning model.

[0676] A "machine learning model" refers to an algorithm or mathematical model that makes predictions or classifications based on data.

[0677] "Prediction result" refers to the result derived by the machine learning model based on the input data.

[0678] An "evaluation score" is an index used to quantitatively evaluate the predictive performance of a machine learning model.

[0679] "Sensor Data" means physical or environmental data collected from a sensor device.

[0680] "Pattern recognition" refers to the technique of identifying specific schematic patterns or trends in data.

[0681] "Predictive maintenance" refers to the use of data analysis to make predictions and take appropriate maintenance actions to prevent equipment and system failures and malfunctions.

[0682] This invention is a no-code tool that enables users without the specialized knowledge of data scientists to easily predict and classify data. As an application example, we will explain how to realize a predictive maintenance system installed in a factory robot.

[0683] System Overview

[0684] We provide a system in which a server, a terminal, and a user work together to perform the following operations:

[0685] Hardware and Software

[0686] Hardware

[0687] Various sensors inside the robot (e.g. temperature, vibration, pressure, etc.)

[0688] Centralized control server in the factory

[0689] Factory manager's device (smartphone or PC)

[0690] software

[0691] Python

[0692] Pandas, Scikit-learn (machine learning library)

[0693] Flask (web server framework)

[0694] MQTT (sensor data communication protocol)

[0695] JSON (data format)

[0696] Server-side processing

[0697] The server processes the data as follows:

[0698] 1. Data reception: Receive data sent from each sensor using the MQTT protocol and convert it into a Pandas DataFrame.

[0699] 2. Data preprocessing: Completing missing and outlier values ​​and normalizing data.

[0700] 3. Data splitting: Split the data into features and target variables, and then split it into training data and test data.

[0701] 4. Model selection and training: Select an appropriate machine learning model (e.g., random forest, support vector machine, etc.) depending on the type of problem and train it on the training data.

[0702] 5. Prediction and Evaluation: The test data is used to generate predictions and evaluation scores for the model.

[0703] 6. Result output: The generated prediction results and evaluation scores are generated in JSON format and sent to the terminal.

[0704] Operation on the terminal side

[0705] The terminal performs the following actions:

[0706] 1. User input acceptance: Accepts data (e.g., CSV file upload) and prediction and classification settings as input from the factory manager.

[0707] 2. Send request: The entered data and settings are converted into a specified format and sent to the server as a POST request.

[0708] 3. Receiving and displaying results: The prediction results and evaluation scores returned from the server are received and displayed to the user.

[0709] User operations

[0710] The specific operation steps are as follows:

[0711] 1. Data input: Data acquired from sensors in the factory is input into the system. For example, prepare the following data set:

[0712] Temperature: 35.4, 35.6, 36.0, 36.2, 35.9

[0713] Vibration: 0.05, 0.07, 0.08, 0.09, 0.07

[0714] Pressure: 1.1, 1.2, 1.3, 1.4, 1.2

[0715] Fault history: 0, 0, 1, 0, 0

[0716] 2. Example prompt:

[0717] "Based on the following data, please predict whether or not there will be a breakdown within a week.

[0718] Temperature: 35.4, 35.6, 36.0, 36.2, 35.9

[0719] Vibration: 0.05, 0.07, 0.08, 0.09, 0.07

[0720] Pressure: 1.1, 1.2, 1.3, 1.4, 1.2

[0721] Fault history: 0, 0, 1, 0, 0"

[0722] 3. Prediction and maintenance: The server selects a machine learning model based on the input data, performs failure prediction, and returns the results to the user. The user then performs appropriate maintenance work based on the prediction results.

[0723] This allows even users without specialized knowledge to easily predict failures in equipment and robots within a factory, enabling maintenance work to be carried out at the appropriate time.

[0724] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0725] Step 1:

[0726] Data reception

[0727] The server receives data sent from sensors in the factory using the MQTT protocol. The received data is converted into a Pandas DataFrame. Here, the input is sensor data, and the output is data in Pandas DataFrame format. This conversion makes it easier to handle the data.

[0728] Step 2:

[0729] Data Preprocessing

[0730] The server receives a Pandas DataFrame and performs missing value imputation, outlier processing, and data normalization. The input is raw data in Pandas DataFrame format, and the output is preprocessed data. This preprocessing improves the quality of the data and increases the accuracy of the machine learning model.

[0731] Step 3:

[0732] Data partitioning

[0733] The server splits the preprocessed data into features and targets, and then splits it into training data and test data. The input is the preprocessed data, and the output is a training dataset and a test dataset. This splitting enables training and evaluation of machine learning models.

[0734] Step 4:

[0735] Model Selection and Training

[0736] The server selects an appropriate machine learning model depending on the type of problem (classification or regression) and trains the model using the training data. The input is the training dataset, and the output is the trained machine learning model. This step enables prediction and classification based on the data.

[0737] Step 5:

[0738] Prediction and Evaluation

[0739] The server uses the trained model to make predictions on the test data and generates prediction results and evaluation scores. The inputs are the trained model and the test dataset, and the outputs are the prediction results and evaluation scores. This allows the performance of the model to be quantitatively evaluated.

[0740] Step 6:

[0741] Result output

[0742] The server generates the generated prediction results and evaluation scores in JSON format and sends them to the terminal. The input is the prediction results and evaluation scores, and the output is JSON format data. This step allows the user to easily check the results.

[0743] Step 7:

[0744] Accepting User Input

[0745] The terminal accepts data (e.g., uploading a CSV file) and prediction and classification settings as input from the factory manager. The input is data and settings from the user, and the output is a request to the server. At this step, the user can operate the system and start executing prediction and classification tasks.

[0746] Step 8:

[0747] Send request

[0748] The terminal converts the input data and settings into a specified format and sends it to the server as a POST request. The input is data and settings from the user, and the output is a request to the server. In this step, the terminal requests the server to process the data.

[0749] Step 9:

[0750] Receive and display results

[0751] The terminal receives the prediction results and evaluation scores returned from the server and displays them to the user. The input is JSON-formatted data from the server, and the output is the results displayed to the user. This allows the user to check the prediction results and take appropriate maintenance action.

[0752] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0753] This invention is a system that combines a no-code tool that enables users without specialized data scientist knowledge to easily predict and classify data with an emotion engine that recognizes user emotions. The incorporation of the emotion engine enables data processing and display of results based on the user's emotions, providing a more user-friendly experience.

[0754] System Overview

[0755] When a user inputs data and requests, the server automatically selects an appropriate machine learning model based on the data, and then trains and evaluates it. In addition, an emotion engine recognizes the user's emotions and adjusts data preprocessing and result display based on the emotions. The results are returned to the user via the device.

[0756] Server-side processing

[0757] The server processes the data as follows:

[0758] 1. Receive data entered by the user and convert it to a Pandas DataFrame.

[0759] 2. The emotion engine recognizes the user's emotion and acquires the emotion data.

[0760] 3. Split the target variable and features from the data frame.

[0761] 4. Split the data into training and test data, which may be adjusted based on user sentiment.

[0762] 5. Select the appropriate machine learning model for the type of problem. The sentiment engine adjusts model selection as needed.

[0763] 6. Train the selected model on the training data.

[0764] 7. Use the test data to generate model predictions and evaluation scores.

[0765] 8. The generated prediction results and evaluation scores are adjusted according to the user's sentiment and output in an appropriate format.

[0766] Emotion Engine Operation

[0767] The emotion engine does the following:

[0768] 1. Recognize the user's emotions in real time while they are typing, for example by reading facial expressions through a camera or analyzing the tone of voice through voice input.

[0769] 2. Optimize data preprocessing based on the recognized emotion. For example, if a negative emotion is recognized, the system will perform more careful error detection.

[0770] 3. Adjust the display of the final prediction result and evaluation score depending on the emotion. If a positive emotion is recognized, the result display can be simplified, while if a negative emotion is recognized, a detailed explanation can be added.

[0771] Operation on the terminal side

[0772] The terminal performs the following actions:

[0773] 1. Accepts data and prediction or classification settings from the user as input.

[0774] 2. Convert the input data into JSON format and send it to the server as a POST request.

[0775] 3. The emotion engine is used to obtain the user's emotions in real time, and this information is also sent to the server.

[0776] 4. Receive the prediction results and evaluation scores returned from the server.

[0777] 5. Display the received results in an appropriate format based on the user's sentiment.

[0778] User operations

[0779] The user does the following:

[0780] 1. Input data into the system. Emotion recognition is automatic.

[0781] 2. Specify the target variable and problem type for prediction or classification.

[0782] 3. The system processes the data on the server side and waits for the results to be returned. During this time, emotion recognition is also performed in parallel.

[0783] 4. The results are displayed on the device, where you can check the prediction results and evaluation score, and receive emotional feedback.

[0784] Specific examples

[0785] For example, suppose a user prepares the following dataset and performs a classification task:

[0786] Example dataset:

[0787] Feature 1: Age

[0788] Feature 2: Income

[0789] target: purchase history (0 or 1)

[0790] When a user enters this data into the system and selects a classification task, the following occurs:

[0791] 1. Data is received and converted into a DataFrame.

[0792] 2. The emotion engine recognizes the user's emotions and acquires the data.

[0793] 3. Age and income are set as features, and purchase history is set as the objective variable.

[0794] 4. Split the dataset into training and test data, possibly with adjustments based on emotion recognition.

[0795] 5. The server selects a random forest classifier for the classification problem and trains the model on the training data.

[0796] 6. Use the trained model to make predictions on the test data and generate results and evaluation scores.

[0797] 7. The generated prediction results and evaluation scores are adjusted according to the user's sentiment and sent back to the device.

[0798] 8. The device displays the prediction results and evaluation scores to the user. Positive emotions are displayed in a simple format, while negative emotions are displayed with a detailed explanation.

[0799] This allows users to predict and classify data without specialized knowledge, and also receive appropriate feedback based on their emotions.

[0800] The processing flow will be explained below.

[0801] Step 1:

[0802] The user inputs data into the system. The data can be uploaded as a CSV file or entered directly into the system's input form. In addition, the target variable for prediction or classification and the type of problem (classification or regression) are specified.

[0803] Step 2:

[0804] The terminal receives the data entered by the user and converts it into JSON format, which includes features, target variables, and problem types.

[0805] Step 3:

[0806] The device sends JSON format data to the server as a POST request, which also includes emotion data for real-time recognition of the user's emotions.

[0807] Step 4:

[0808] The server receives the request and converts the JSON-formatted data into a Pandas DataFrame, which makes it easier to manipulate the data in your program.

[0809] Step 5:

[0810] The server splits the data frame into target variables and features. The target variables are the objects that the user wants to predict or classify.

[0811] Step 6:

[0812] The server splits the data into training and testing data, typically assigning 80% of the data to training data and 20% to testing data.

[0813] Step 7:

[0814] The emotion engine recognizes the user's emotions in real time as they are being input, and sends the emotion data to the server using techniques such as facial expression recognition and voice analysis.

[0815] Step 8:

[0816] The server selects an appropriate machine learning model depending on the type of problem: a random forest classifier for classification problems, a random forest regressor for regression problems, and model selection and hyperparameter tuning depending on the emotion data.

[0817] Step 9:

[0818] The server trains the selected model using the training data, using the features and the target variable.

[0819] Step 10:

[0820] The server uses the trained model to make predictions on the test data, which produces the prediction results.

[0821] Step 11:

[0822] The server evaluates the model's performance using the test data and calculates a score that indicates the model's accuracy.

[0823] Step 12:

[0824] The emotion engine analyzes the user's emotion data and adjusts the way predictions and rating scores are displayed: simple for positive emotions, and detailed explanations for negative emotions.

[0825] Step 13:

[0826] The server returns the generated prediction results, evaluation scores, and emotion-based adjustment information as a response in JSON format. The response includes a list of prediction results and evaluation scores.

[0827] Step 14:

[0828] The device receives the response from the server and displays the prediction results and evaluation scores to the user, providing tailored feedback based on the user's emotions.

[0829] As a concrete example, let's say a user prepares the following dataset and performs a classification task:

[0830] Example dataset:

[0831] Feature 1: Age

[0832] Feature 2: Income

[0833] target: purchase history (0 or 1)

[0834] When a user enters this data into the system and selects a classification task, the following occurs:

[0835] Steps 1-3:

[0836] The user inputs data, the emotion engine recognizes emotions in real time, and the device sends this data and emotion information to the server.

[0837] Steps 4-6:

[0838] The server receives the data, converts it into a DataFrame, splits it into target variables and features, and then splits the dataset into training data and test data.

[0839] Steps 7-12:

[0840] The server continues to process the data in cooperation with the emotion engine, selecting a random forest classifier for the classification problem and training the model on the training data, then making predictions using the test data to generate results and evaluation scores.

[0841] Steps 13-14:

[0842] The emotion engine adjusts according to the user's emotions and returns the prediction results and evaluation score to the device, which receives and displays them to the user. In this case, a simple display is provided for positive emotions, and a detailed explanation is provided for negative emotions.

[0843] This allows users to predict and classify data without specialized knowledge, and also receive appropriate feedback based on their emotions.

[0844] Example 2

[0845] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0846] Conventional data analysis systems require the specialized knowledge of data scientists, making them difficult for general users to use. Furthermore, they do not provide a user-friendly experience because they do not take into account the user's feelings when processing data or displaying results. This can lead to stress and frustration when users use the system.

[0847] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0848] In this invention, the server includes a data input means, a means for dividing the input data into features and a response variable, a means for dividing the divided data into training data and test data, a means for selecting an appropriate machine learning model according to the type of problem, a means for training the selected model with the training data, a means for generating prediction results and an evaluation score of the model using the test data, a means for outputting the prediction results and the evaluation score, a means for recognizing user emotions in real time, and a means for adjusting data preprocessing and result display based on the user emotions. This enables general users without the specialized knowledge of a data scientist to easily perform data analysis and receive feedback according to the user's emotions.

[0849] "Data input means" refers to a device or function for capturing data input by a user into the system.

[0850] A "feature" is an element of data that a machine learning model uses to make a prediction or classification.

[0851] A "target variable" is an element of data that a machine learning model generates as a prediction or classification result.

[0852] "Training data" is a data set used to train a machine learning model.

[0853] "Test data" is a dataset used to evaluate the performance of a trained machine learning model.

[0854] A "machine learning model" is a computational method or algorithm that learns patterns and regularities from data and makes predictions and classifications for new data.

[0855] The "emotion recognition means" is a device or function for recognizing the user's emotions in real time.

[0856] "Preprocessing means" refers to a device or function that formats and transforms data to make it easier to analyze data or train machine learning models.

[0857] "Feedback" refers to information provided to the user by the system regarding prediction results and evaluation scores, in order to promote the user's understanding and next action.

[0858] This invention is a system that incorporates an emotion engine that recognizes user emotions into a no-code tool that enables general users without the specialized knowledge of a data scientist to easily perform data analysis. This system allows users to receive feedback according to their emotions, providing a more user-friendly experience.

[0859] Server-side hardware and software

[0860] The server of this system uses the following hardware and software:

[0861] Hardware: A server machine with a powerful CPU and sufficient memory

[0862] software:

[0863] Pandas: Used to receive and transform data

[0864] scikit-learn: Used for data splitting, model selection, and model training

[0865] Emotion Engine: Recognize user emotions in real time

[0866] Process Overview

[0867] When a user inputs data and prediction or classification settings, the server automatically selects an appropriate machine learning model based on the data, and then trains and evaluates it. In addition, an emotion engine recognizes the user's emotions and adjusts data preprocessing and result display based on the emotions. The results are returned to the user via the device.

[0868] Terminal hardware and software

[0869] The terminals of this system use the following hardware and software:

[0870] Hardware: Devices with a camera and microphone (e.g., PC, tablet, smartphone)

[0871] software:

[0872] Front-end application: data entry and result display

[0873] Emotion Recognition API: Acquire emotion data from camera and voice input

[0874] System Operation

[0875] 1. Data Entry

[0876] A user uses a terminal to input data, for example, values ​​such as "age" and "income," and specifies a classification task.

[0877] 2. Emotion recognition

[0878] The device uses a built-in camera and microphone to recognize the user's emotions in real time and transmits the emotion data to a server.

[0879] 3. Data Transmission

[0880] The terminal converts the input data into JSON format and sends it to the server.

[0881] 4. Model Selection and Training

[0882] The server converts the received data into a Pandas DataFrame and uses scikit-learn to select and train an appropriate machine learning model. At the same time, the emotion engine retrieves the emotion data and adjusts data preprocessing as needed.

[0883] 5. Prediction and Evaluation

[0884] The server makes predictions on the test data and generates evaluation scores, which are adjusted based on the emotion data.

[0885] 6. Displaying the results

[0886] The device displays the results received from the server in an appropriate format depending on the user's emotions. For example, if the user has a positive emotion, the results are displayed simply, and if the user has a negative emotion, the results are displayed with a detailed explanation.

[0887] Specific examples

[0888] For example, suppose a user has the following dataset and is tasked with classification:

[0889] Example dataset:

[0890] Feature 1: Age

[0891] Feature 2: Income

[0892] target: purchase history (0 or 1)

[0893] Once the user enters this data into the system and selects a classification task, the system operates as follows:

[0894] 1. Data is received and converted into a DataFrame

[0895] The server receives the age and income data and converts it into a Pandas DataFrame.

[0896] 2. Emotion Recognition and Data Segmentation

[0897] The emotion engine recognizes the user's smile and collects the data. The target variable is purchase history, and age and income are divided as features.

[0898] 3. Model training and evaluation

[0899] The server selects and trains a random forest, generating predictions and evaluation scores for the test data.

[0900] 4. Adjusting and displaying the results

[0901] The server adjusts the results according to the user's emotions and sends them back to the device. The results are displayed simply with positive emotions.

[0902] Examples of prompt statements

[0903] "I want to classify data based on age, income, and purchase history. Please let me know the prediction results and evaluation scores."

[0904] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0905] Processing Steps

[0906] Step 1:

[0907] Data Entry

[0908] The user inputs data from the terminal. Specifically, the user inputs feature quantities such as "age" and "income" and objective variables such as "purchase history."

[0909] Input: Age (e.g., 30), Income (e.g., 50000), Purchase History (e.g., 1)

[0910] Output: Input data on the terminal

[0911] Step 2:

[0912] Sending input data

[0913] The terminal converts the input data into JSON format and sends it to the server as a POST request.

[0914] Input: Data entered by the user

[0915] Output: JSON data sent to the server ({"age": 30, "income": 50000, "purchase_history": 1})

[0916] Step 3:

[0917] Acquiring and sending emotion data

[0918] The device uses a camera and microphone to capture the user's emotional data in real time and transmits it to a server.

[0919] Input: Emotion data obtained through a camera or microphone

[0920] Output: Emotion data sent to the server (e.g., {"emotion": "positive"})

[0921] Step 4:

[0922] Receiving and Converting Data

[0923] The server receives the JSON data sent from the terminal and converts it into a DataFrame using the Pandas read_json method.

[0924] Input: JSON data sent from the terminal

[0925] Output: Pandas DataFrame format data

[0926] Step 5:

[0927] emotion recognition

[0928] The emotion engine analyzes the emotion data sent from the server and recognizes the user's emotions in real time.

[0929] Input: Emotion data sent from the server

[0930] Output: Recognized emotion data (e.g., {"emotion": "positive"})

[0931] Step 6:

[0932] Splitting data into features and target variables

[0933] The server splits the features (age, income) and the target variable (purchase history) from the DataFrame.

[0934] Input: Pandas DataFrame format data

[0935] Output: Data split into features and target variables (e.g., features: age and income, target variable: purchase history)

[0936] Step 7:

[0937] Splitting the data into training and test

[0938] The server splits the data into training and test data using the train_test_split function, adjusting the shuffling of the data based on the sentiment data if necessary.

[0939] Input: Data split into features and target variables

[0940] Output: training data and test data (e.g., training data: 80%, test data: 20%)

[0941] Step 8:

[0942] Model selection and adjustment

[0943] The server selects the appropriate machine learning model depending on the type of problem, and may adjust the model selection based on emotion data. For example, random forest is selected for classification problems.

[0944] Input: training data, problem type, emotion data

[0945] Output: The selected machine learning model (e.g., Random Forest)

[0946] Step 9:

[0947] Training the model

[0948] The server uses the selected model to train the training data. Run model.fit(X_train, y_train).

[0949] Input: Training data, selected machine learning model

[0950] Output: A trained machine learning model

[0951] Step 10:

[0952] Prediction and Evaluation

[0953] The server makes predictions using the test data and generates evaluation scores using accuracy_score, confusion_matrix, etc. Run model.predict(X_test).

[0954] Input: trained machine learning model, test data

[0955] Output: Prediction results and evaluation scores (e.g. accuracy, confusion matrix)

[0956] Step 11:

[0957] Adjusting and outputting prediction results

[0958] The server adjusts the prediction results and evaluation scores according to the user's emotions and returns them to the device in an appropriate format, for example, adding a more detailed explanation if the emotion is negative.

[0959] Input: Prediction results, rating scores, sentiment data

[0960] Output: JSON data of adjusted prediction results and evaluation scores

[0961] Step 12:

[0962] Displaying prediction results

[0963] The device displays the prediction results and evaluation scores received from the server in an appropriate format depending on the user's emotions. For example, it displays them simply if the emotion is positive, and in detail if the emotion is negative.

[0964] Input: Adjusted prediction results and evaluation scores sent from the server

[0965] Output: Prediction results and evaluation scores displayed on the device

[0966] (Application example 2)

[0967] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0968] Conventional data prediction and classification systems lack user experience because they do not take user emotions into account when processing or displaying results. Additionally, users without the expertise of a data scientist find it difficult to use the systems.

[0969] The specification processing by the specification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a data input means, a means for dividing the input data into features and a response variable, a means for dividing the divided data into training data and test data, a means for selecting an appropriate machine learning model according to the type of problem, a means for training the selected model with the training data, a means for generating prediction results and evaluation scores of the model using the test data, a means for outputting the prediction results and evaluation scores, an emotion recognition means for recognizing a user's emotions in real time, and a means for adjusting data preprocessing and result display using emotion data acquired by the emotion recognition means. This enables data processing and result display according to the user's emotions, providing a more intuitive and user-friendly system.

[0970] "Data input means" refers to a device or interface that accepts data input by a user.

[0971] "Features" refer to important attributes extracted from data that are used for analysis such as prediction and classification.

[0972] A "target variable" refers to a specific value or label of data that is the target of prediction or classification.

[0973] "Splitting means" refers to a device or algorithm that performs processing to split input data into features and target variables.

[0974] "Training data" refers to the data set used to train a machine learning model.

[0975] "Test data" refers to a dataset that has not been used for training but is used to evaluate the performance of a trained machine learning model.

[0976] "Machine learning model selection means" refers to a device or algorithm for selecting an appropriate machine learning model based on the type of problem given.

[0977] "Model training means" refers to a device or algorithm that trains a selected machine learning model with training data.

[0978] "Prediction result generation means" refers to a device or algorithm that uses test data to generate prediction results and evaluation scores for a trained machine learning model.

[0979] "Output means" refers to a device or interface for presenting the prediction results and evaluation scores to the user in an appropriate format.

[0980] "Emotion recognition means" refers to a device or algorithm for recognizing a user's emotions in real time.

[0981] "Emotion data utilization means" refers to a device or algorithm that utilizes the emotion data acquired by the emotion recognition means to adjust data preprocessing and result display.

[0982] The present invention combines a data prediction and classification system with a user emotion recognition function, allowing users to experience data processing and result display according to their emotions, providing a more intuitive and user-friendly interface.

[0983] System Overview

[0984] The system consists of a data input means, a means for dividing features and target variables, a means for dividing training data and test data, a means for selecting a machine learning model, a means for training the model, a means for generating prediction results and evaluation scores, an output means, a means for recognizing emotions, and a means for using emotion data.

[0985] Hardware and software used

[0986] The system uses the following hardware and software:

[0987] 1. Hardware:

[0988] Webcam: Captures your facial expressions in real time.

[0989] Computer: Handles all processing.

[0990] 2. Software:

[0991] Python: An entire programming language.

[0992] OpenCV: A library for processing video from a webcam.

[0993] FER (Facial Expression Recognition) Library: A library for recognizing user emotions.

[0994] Pandas: A library for data preprocessing.

[0995] Scikit-learn: A library for machine learning models.

[0996] Requests: A library that communicates with the API.

[0997] Processing flow

[0998] server

[0999] The server does the following:

[1000] 1. Receive data entered by the user and convert it into a data frame using Pandas.

[1001] 2. Emotion data of the user is acquired by the emotion recognition means.

[1002] 3. Split the features and objective variables from the data frame and divide it into training data and test data.

[1003] 4. Select an appropriate machine learning model and train the selected model using the training data.

[1004] 5. Use the trained model to make predictions on the test data and generate an evaluation score.

[1005] 6. Use sentiment data to refine prediction results and evaluation scores into appropriate formats.

[1006] 7. Send the results back to the device.

[1007] Terminal

[1008] The terminal does the following:

[1009] 1. Accepts data and prediction / classification settings from the user, converts the input data into JSON format, and sends it to the server.

[1010] 2. Capture user emotion data in real time using a webcam and send it to the server.

[1011] 3. Receive the prediction results and evaluation scores returned from the server and display them in an appropriate format based on the user's sentiment.

[1012] User

[1013] The user does the following:

[1014] 1. Data is input into the system and emotion recognition is done automatically.

[1015] 2. Specify the target variable and problem type for prediction or classification.

[1016] 3. Wait for the server to process the data, and the results will be displayed on the device.

[1017] 4. Check your prediction results and evaluation scores and get feedback.

[1018] Specific examples

[1019] For example, if a user prepares purchase history data and performs a classification task, age and income are the features, and purchase history is the target variable. The system processes it as follows:

[1020] 1. Data is received and converted into a Pandas dataframe.

[1021] 2. The user's emotion data is acquired by the FER library.

[1022] 3. The data is split into features and target variables, and then split into training data and test data.

[1023] 4. A random forest classifier is selected and trained on the training data.

[1024] 5. The trained model is used to make predictions on the test data and generate an evaluation score.

[1025] 6. Taking into account the emotion data, the prediction results and evaluation scores are adjusted and sent back to the device.

[1026] 7. The prediction results and evaluation scores are displayed to the user.

[1027] Example prompt sentence:

[1028] Create a system like this: An application for an online shopping site that uses a webcam to recognize user emotions in real time and recommends appropriate products based on those emotions. You will use OpenCV and FER library to recognize user emotions, and the recommender system will use a machine learning model to recommend products based on the user and emotion data.

[1029] The above is a specific embodiment of the present invention. This system enables emotion-based data preprocessing and result display, allowing users to enjoy a more intuitive interface.

[1030] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1031] Step 1:

[1032] The terminal accepts data and prediction or classification settings from the user. The terminal converts the input data into JSON format and sends it to the server as a POST request. The input in this step is user-entered data, and the output is JSON-formatted data.

[1033] Step 2:

[1034] The device captures the user's facial expressions in real time using a webcam. The FER library is used to recognize the user's emotions from the captured video. The recognized emotion data is also sent to the server. The input in this step is the real-time video, and the output is the user's emotion data.

[1035] Step 3:

[1036] The server receives JSON-formatted data sent from the terminal and converts it into a data frame using Pandas. It then extracts features and target variables from the data frame. The input in this step is JSON-formatted data, and the output is a data frame split into features and target variables.

[1037] Step 4:

[1038] The server splits the data divided into features and objective variables into training data and test data. At the same time, it also performs preprocessing based on emotion data. The input in this step is the data frame divided into features and objective variables and emotion data, and the output is training data and test data.

[1039] Step 5:

[1040] The server selects an appropriate machine learning model depending on the problem type, for example, a random forest classifier for a classification problem. The input in this step is the problem type of the data, and the output is the selected machine learning model.

[1041] Step 6:

[1042] The server uses the selected machine learning model to train the model on the training data. The inputs in this step are the training data and the machine learning model, and the output is the trained model.

[1043] Step 7:

[1044] The server uses the trained model to predict the test data and generate an evaluation score. The inputs in this step are the test data and the trained model, and the outputs are the prediction results and the evaluation score.

[1045] Step 8:

[1046] The server uses the emotion data to adjust the prediction result and evaluation score according to the emotion. For example, if the user expresses a negative emotion, it adds a detailed explanation. The inputs in this step are the prediction result, the evaluation score, and the emotion data, and the output is the adjusted prediction result and evaluation score.

[1047] Step 9:

[1048] The device receives the prediction results and evaluation scores returned from the server and displays them in an appropriate format depending on the user's emotions. For example, a simple display for positive emotions and a detailed explanation for negative emotions. The input in this step is the adjusted prediction results and evaluation scores, and the output is the result displayed to the user.

[1049] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1050] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1051] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1052] [Third embodiment]

[1053] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1054] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1055] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1056] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1057] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1058] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1059] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1060] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1061] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1062] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1063] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1064] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1065] This invention is a no-code tool that allows users without the specialized knowledge of data scientists to easily predict and classify data.

[1066] System Overview

[1067] When a user inputs their data and request, the server automatically selects an appropriate machine learning model based on that data, trains it, and evaluates it, returning the results to the user via their device.

[1068] Server-side processing

[1069] The server processes the data as follows:

[1070] 1. Receive data entered by the user.

[1071] 2. Convert the received data into a Pandas DataFrame.

[1072] 3. Split the target variable and features.

[1073] 4. Split the data into training and test data.

[1074] 5. Select the appropriate model depending on the type of problem (classification or regression).

[1075] 6. Train the selected model on the training data.

[1076] 7. Use the test data to generate model predictions and evaluation scores.

[1077] 8. Output the generated prediction results and evaluation scores in JSON format.

[1078] Operation on the terminal side

[1079] The terminal performs the following actions:

[1080] 1. Accepts data and prediction or classification settings from the user as input.

[1081] 2. Convert the input data into a specified format and send it to the server as a POST request.

[1082] 3. Receive the prediction results and evaluation scores returned from the server.

[1083] 4. Display the received results to the user.

[1084] User operations

[1085] The user does the following:

[1086] 1. Enter data into the system (e.g., upload a CSV file or enter data manually).

[1087] 2. Specify the target variable for prediction or classification and the type of problem (classification or regression).

[1088] 3. The system processes the request on the server side and waits for the results to be returned.

[1089] 4. The results will be displayed on your device, and you can check the prediction results and evaluation score.

[1090] Specific examples

[1091] For example, suppose a user prepares the following dataset and performs a classification task:

[1092] Example dataset:

[1093] Feature 1: Age

[1094] Feature 2: Income

[1095] target: purchase history (0 or 1)

[1096] Once the user enters this data into the system and selects a classification task, the following happens automatically on the server side:

[1097] 1. Data is received and converted into a DataFrame.

[1098] 2. Age and income are set as features, and purchase history is set as the target variable.

[1099] 3. Split the dataset into training and test data.

[1100] 4. The server selects a random forest classifier for the classification problem and trains the model on the training data.

[1101] 5. The trained model is used to make predictions on the test data, generating results and evaluation scores.

[1102] 6. The generated prediction results and evaluation scores are sent back to the user's device and displayed on the screen.

[1103] This allows users to predict and classify data without specialized knowledge, making it easy to make data-based decisions.

[1104] The processing flow will be explained below.

[1105] Step 1:

[1106] The user inputs data into the system. The data can be uploaded as a CSV file or entered directly into the system's input form. This input specifies the target variable for prediction or classification and the type of problem (classification or regression).

[1107] Step 2:

[1108] The terminal receives the data entered by the user and converts it into JSON format, which includes features, target variables, and problem types.

[1109] Step 3:

[1110] The device sends JSON-formatted data to the server as a POST request to an API endpoint hosted by the system (e.g., http: / / localhost:5000 / predict).

[1111] Step 4:

[1112] The server receives the request and converts the JSON-formatted data into a Pandas DataFrame, which makes it easier to manipulate the data in your program.

[1113] Step 5:

[1114] The server splits the data frame into target variables and features. The target variables are the objects that the user wants to predict or classify.

[1115] Step 6:

[1116] The server splits the data into training and testing data, typically assigning 80% of the data to training data and 20% to testing data.

[1117] Step 7:

[1118] The server selects the appropriate machine learning model depending on the type of problem: a random forest classifier for classification problems, and a random forest regressor for regression problems.

[1119] Step 8:

[1120] The server trains the selected model using the training data, using the features and the target variable.

[1121] Step 9:

[1122] The server uses the trained model to make predictions on the test data, which produces the prediction results.

[1123] Step 10:

[1124] The server evaluates the model's performance using the test data and calculates a score that indicates the model's accuracy.

[1125] Step 11:

[1126] The server returns the generated prediction results and evaluation scores as a response in JSON format. The response includes a list of prediction results and evaluation scores.

[1127] Step 12:

[1128] The terminal receives the response from the server and displays the prediction results and evaluation scores to the user, allowing the user to check the results of the data analysis.

[1129] Example 1

[1130] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1131] Conventional data analysis tools are difficult to use for users without specialized knowledge and are cumbersome to operate. In addition, difficulties in data preprocessing and selecting appropriate machine learning models have led to problems with reduced accuracy in prediction results and evaluation scores.

[1132] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1133] In this invention, the server includes a data input means, a means for converting the input data into a data frame, a means for dividing the converted data into features and a target variable, a means for dividing the divided data into training data and test data, a means for selecting an appropriate model depending on the type of problem, a means for training the selected model with the training data, a means for generating prediction results and evaluation scores of the model using the test data, and a means for outputting the prediction results and evaluation scores. This enables users to easily predict and classify data without specialized knowledge.

[1134] The "data input means" is an interface for accepting data from a user.

[1135] The "means for converting into a data frame" is a processing mechanism for converting received data into a tabular data structure.

[1136] "Means for splitting features and objective variables" is a function for extracting and splitting the input data (features) required for analysis and prediction and the target output data (objective variables) from a data frame.

[1137] The "means for splitting into training data and test data" is a processing function for splitting the original data into a data set for model training and a data set for evaluation.

[1138] A "model selection method" is an algorithm for selecting an appropriate machine learning model based on the characteristics of the data and the type of problem (classification or regression).

[1139] "Means for training a model" refers to a process for training a selected machine learning model using training data to improve its predictive capabilities.

[1140] The "means for generating prediction results and evaluation scores" is a function that uses a trained model to make predictions on test data and calculates the prediction results and evaluation indicators such as accuracy.

[1141] The "means for outputting prediction results and evaluation scores" is an interface for providing the generated prediction results and evaluation scores to the user.

[1142] System Overview

[1143] The system of this invention is a no-code tool that allows users without data scientist expertise to easily perform data prediction and classification. It is mainly composed of three elements: a server, a terminal, and a user.

[1144] Server-side processing

[1145] The server receives the data provided by the user and converts it into a data frame format. The specific processing is done using the Python Pandas library. From this data frame, the features and target variables are extracted separately.

[1146] Next, the server splits the data into training and test data using the train_test_split function from scikit-learn. Depending on the type of problem (classification or regression), it selects an appropriate machine learning model: for classification, it selects a random forest classifier, and for regression, it selects a random forest regressor.

[1147] The selected model is trained on the training data. Then, predictions are made using the model on the test data, and prediction results and evaluation scores (e.g., metrics such as precision and recall) are generated. The generated prediction results and evaluation scores are converted into JSON format and sent to the user's device.

[1148] Operation on the terminal side

[1149] The terminal receives data and configuration information (such as classification and prediction requests) from the user, converts it into a specific format, and sends it to the server as a POST request. A simple user interface is provided, allowing the user to operate it intuitively.

[1150] After receiving the prediction results and evaluation scores from the server, the device displays this information to the user, who can then check the results through the input interface or dashboard.

[1151] User operations

[1152] Users input data into the system by uploading a CSV file or manually entering the data. They also specify the target variable for prediction or classification and select whether the problem is classification or regression.

[1153] For example, suppose a user prepares a dataset containing age and income information and performs a classification task on purchase history (0 or 1). Then, the user inputs the dataset into the system and specifies the classification task. In this case, an appropriate model is automatically selected and trained on the server side. As a result, the user can check the prediction results and evaluation scores based on the input data.

[1154] Prompt Sentence Examples

[1155] Below are some examples of prompts to input to the generative AI model.

[1156] "Using the following dataset, create a prompt to classify purchase history (0 or 1). The dataset contains information on age and income."

[1157] This prompt allows the user to ask the generative AI for assistance with the necessary data processing and application of machine learning models.

[1158] This invention allows users to easily perform advanced data analysis without specialized knowledge.

[1159] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1160] Step 1:

[1161] The server receives data entered by the user. The input is sent in the form of a POST request from the terminal. Specifically, this is done by the user uploading a CSV file or entering data directly into a form. The received data is temporarily stored so that it can be passed on to the next processing step as is.

[1162] Step 2:

[1163] The server converts the received data into a data frame using the Pandas library. At this time, the received data (e.g., in CSV format) is read using the Pandas read_csv function and converted into a data frame. The input is the contents of the CSV file, and the output is a Pandas data frame object.

[1164] Step 3:

[1165] The server separates the features and the response variable from the data frame. Specifically, it uses the column names of the data frame to extract the columns designated as response variables and separates the other columns as features. This allows the feature (X) and response variable (y) to be obtained from the input data (all data frames).

[1166] Step 4:

[1167] The server splits the data into training data and test data using the scikit-learn train_test_split function. The input is the features and the objective variable, and the output is the training dataset (X_train, y_train) and the test dataset (X_test, y_test). The split ratio is generally set to 80:20 by default.

[1168] Step 5:

[1169] The server determines the problem type and selects the appropriate machine learning model: the input is the problem type (classification or regression) specified by the user, and the output is the corresponding model object (e.g., a random forest classifier for classification, or a random forest regressor for regression).

[1170] Step 6:

[1171] The server trains the selected model using the training data. Specifically, it uses the model's fit method to input the training data (X_train, y_train) and output the trained model. In this step, the model parameters are optimized.

[1172] Step 7:

[1173] The server uses the trained model to make predictions on the test data. The input is the trained model and test data (X_test), and the output is the prediction result. Specifically, the server makes predictions using the model's predict method.

[1174] Step 8:

[1175] The server generates prediction results and evaluation scores. The inputs are the prediction results and the actual values ​​of the test data (y_test), and the output is the evaluation score (e.g., precision or recall). Specifically, the score is calculated using the evaluation function of scikit-learn.

[1176] Step 9:

[1177] The server converts the generated prediction results and evaluation scores into JSON format and sends them to the terminal. The terminal parses the received JSON data and displays it to the user. The input is the prediction results and evaluation scores, and the output is the information displayed in the user interface.

[1178] Step 10:

[1179] The user checks the prediction results and evaluation scores displayed on the device, and decides on the next action based on the results. Specifically, the results are displayed in the form of a dashboard or report, so the user only needs to refer to this.

[1180] (Application example 1)

[1181] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1182] Predicting failures in factory equipment and robots and efficiently performing maintenance work has a significant impact on reducing downtime and improving productivity. However, conventional predictive maintenance systems require advanced data science knowledge and experts, making them difficult for ordinary workers to use. As a result, there were issues such as not being able to perform maintenance at the appropriate time and making it difficult to prevent failures before they occur.

[1183] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1184] In this invention, the server includes a data input means, a means for dividing the input data into features and a target variable, a means for dividing the divided data into training data and test data, a means for selecting an appropriate machine learning model depending on the type of problem, a means for training the selected model with the training data, a means for generating prediction results and evaluation scores of the model using the test data, a means for outputting the prediction results and evaluation scores, and a means for receiving sensor data and performing pattern recognition and predictive maintenance. This enables even users without specialized knowledge to easily predict failures in equipment and robots in factories and perform maintenance work at appropriate times.

[1185] "Data input means" refers to a device or interface that accepts data input by a user.

[1186] A "feature" is an individual attribute or variable extracted from data for the purpose of making a prediction or classification.

[1187] The "dependent variable" refers to the data output as a result of prediction or classification, and is often called the "target."

[1188] "Training data" refers to the dataset used to train a machine learning model.

[1189] "Test data" refers to a dataset used to evaluate the performance of a trained machine learning model.

[1190] A "machine learning model" refers to an algorithm or mathematical model that makes predictions or classifications based on data.

[1191] "Prediction result" refers to the result derived by the machine learning model based on the input data.

[1192] An "evaluation score" is an index used to quantitatively evaluate the predictive performance of a machine learning model.

[1193] "Sensor Data" means physical or environmental data collected from a sensor device.

[1194] "Pattern recognition" refers to the technique of identifying specific schematic patterns or trends in data.

[1195] "Predictive maintenance" refers to the use of data analysis to make predictions and take appropriate maintenance actions to prevent equipment and system failures and malfunctions.

[1196] This invention is a no-code tool that enables users without the specialized knowledge of data scientists to easily predict and classify data. As an application example, we will explain how to realize a predictive maintenance system installed in a factory robot.

[1197] System Overview

[1198] We provide a system in which a server, a terminal, and a user work together to perform the following operations:

[1199] Hardware and Software

[1200] Hardware

[1201] Various sensors inside the robot (e.g. temperature, vibration, pressure, etc.)

[1202] Centralized control server in the factory

[1203] Factory manager's device (smartphone or PC)

[1204] software

[1205] Python

[1206] Pandas, Scikit-learn (machine learning library)

[1207] Flask (web server framework)

[1208] MQTT (sensor data communication protocol)

[1209] JSON (data format)

[1210] Server-side processing

[1211] The server processes the data as follows:

[1212] 1. Data reception: Receive data sent from each sensor using the MQTT protocol and convert it into a Pandas DataFrame.

[1213] 2. Data preprocessing: Completing missing and outlier values ​​and normalizing data.

[1214] 3. Data splitting: Split the data into features and target variables, and then split it into training data and test data.

[1215] 4. Model selection and training: Select an appropriate machine learning model (e.g., random forest, support vector machine, etc.) depending on the type of problem and train it on the training data.

[1216] 5. Prediction and Evaluation: The test data is used to generate predictions and evaluation scores for the model.

[1217] 6. Result output: The generated prediction results and evaluation scores are generated in JSON format and sent to the terminal.

[1218] Operation on the terminal side

[1219] The terminal performs the following actions:

[1220] 1. User input acceptance: Accepts data (e.g., CSV file upload) and prediction and classification settings as input from the factory manager.

[1221] 2. Send request: The entered data and settings are converted into a specified format and sent to the server as a POST request.

[1222] 3. Receiving and displaying results: The prediction results and evaluation scores returned from the server are received and displayed to the user.

[1223] User operations

[1224] The specific operation steps are as follows:

[1225] 1. Data input: Data acquired from sensors in the factory is input into the system. For example, prepare the following data set:

[1226] Temperature: 35.4, 35.6, 36.0, 36.2, 35.9

[1227] Vibration: 0.05, 0.07, 0.08, 0.09, 0.07

[1228] Pressure: 1.1, 1.2, 1.3, 1.4, 1.2

[1229] Fault history: 0, 0, 1, 0, 0

[1230] 2. Example prompt:

[1231] "Based on the following data, please predict whether or not there will be a breakdown within a week.

[1232] Temperature: 35.4, 35.6, 36.0, 36.2, 35.9

[1233] Vibration: 0.05, 0.07, 0.08, 0.09, 0.07

[1234] Pressure: 1.1, 1.2, 1.3, 1.4, 1.2

[1235] Fault history: 0, 0, 1, 0, 0"

[1236] 3. Prediction and maintenance: The server selects a machine learning model based on the input data, performs failure prediction, and returns the results to the user. The user then performs appropriate maintenance work based on the prediction results.

[1237] This allows even users without specialized knowledge to easily predict failures in equipment and robots within a factory, enabling maintenance work to be carried out at the appropriate time.

[1238] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1239] Step 1:

[1240] Data reception

[1241] The server receives data sent from sensors in the factory using the MQTT protocol. The received data is converted into a Pandas DataFrame. Here, the input is sensor data, and the output is data in Pandas DataFrame format. This conversion makes it easier to handle the data.

[1242] Step 2:

[1243] Data Preprocessing

[1244] The server receives a Pandas DataFrame and performs missing value imputation, outlier processing, and data normalization. The input is raw data in Pandas DataFrame format, and the output is preprocessed data. This preprocessing improves the quality of the data and increases the accuracy of the machine learning model.

[1245] Step 3:

[1246] Data partitioning

[1247] The server splits the preprocessed data into features and targets, and then splits it into training data and test data. The input is the preprocessed data, and the output is a training dataset and a test dataset. This splitting enables training and evaluation of machine learning models.

[1248] Step 4:

[1249] Model Selection and Training

[1250] The server selects an appropriate machine learning model depending on the type of problem (classification or regression) and trains the model using the training data. The input is the training dataset, and the output is the trained machine learning model. This step enables prediction and classification based on the data.

[1251] Step 5:

[1252] Prediction and Evaluation

[1253] The server uses the trained model to make predictions on the test data and generates prediction results and evaluation scores. The inputs are the trained model and the test dataset, and the outputs are the prediction results and evaluation scores. This allows the performance of the model to be quantitatively evaluated.

[1254] Step 6:

[1255] Result output

[1256] The server generates the generated prediction results and evaluation scores in JSON format and sends them to the terminal. The input is the prediction results and evaluation scores, and the output is JSON format data. This step allows the user to easily check the results.

[1257] Step 7:

[1258] Accepting User Input

[1259] The terminal accepts data (e.g., uploading a CSV file) and prediction and classification settings as input from the factory manager. The input is data and settings from the user, and the output is a request to the server. At this step, the user can operate the system and start executing prediction and classification tasks.

[1260] Step 8:

[1261] Send request

[1262] The terminal converts the input data and settings into a specified format and sends it to the server as a POST request. The input is data and settings from the user, and the output is a request to the server. In this step, the terminal requests the server to process the data.

[1263] Step 9:

[1264] Receive and display results

[1265] The terminal receives the prediction results and evaluation scores returned from the server and displays them to the user. The input is JSON-formatted data from the server, and the output is the results displayed to the user. This allows the user to check the prediction results and take appropriate maintenance action.

[1266] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1267] This invention is a system that combines a no-code tool that enables users without specialized data scientist knowledge to easily predict and classify data with an emotion engine that recognizes user emotions. The incorporation of the emotion engine enables data processing and display of results based on the user's emotions, providing a more user-friendly experience.

[1268] System Overview

[1269] When a user inputs data and requests, the server automatically selects an appropriate machine learning model based on the data, and then trains and evaluates it. In addition, an emotion engine recognizes the user's emotions and adjusts data preprocessing and result display based on the emotions. The results are returned to the user via the device.

[1270] Server-side processing

[1271] The server processes the data as follows:

[1272] 1. Receive data entered by the user and convert it to a Pandas DataFrame.

[1273] 2. The emotion engine recognizes the user's emotion and acquires the emotion data.

[1274] 3. Split the target variable and features from the data frame.

[1275] 4. Split the data into training and test data, which may be adjusted based on user sentiment.

[1276] 5. Select the appropriate machine learning model for the type of problem. The sentiment engine adjusts model selection as needed.

[1277] 6. Train the selected model on the training data.

[1278] 7. Use the test data to generate model predictions and evaluation scores.

[1279] 8. The generated prediction results and evaluation scores are adjusted according to the user's sentiment and output in an appropriate format.

[1280] Emotion Engine Operation

[1281] The emotion engine does the following:

[1282] 1. Recognize the user's emotions in real time while they are typing, for example by reading facial expressions through a camera or analyzing the tone of voice through voice input.

[1283] 2. Optimize data preprocessing based on the recognized emotion. For example, if a negative emotion is recognized, the system will perform more careful error detection.

[1284] 3. Adjust the display of the final prediction result and evaluation score depending on the emotion. If a positive emotion is recognized, the result display can be simplified, while if a negative emotion is recognized, a detailed explanation can be added.

[1285] Operation on the terminal side

[1286] The terminal performs the following actions:

[1287] 1. Accepts data and prediction or classification settings from the user as input.

[1288] 2. Convert the input data into JSON format and send it to the server as a POST request.

[1289] 3. The emotion engine is used to obtain the user's emotions in real time, and this information is also sent to the server.

[1290] 4. Receive the prediction results and evaluation scores returned from the server.

[1291] 5. Display the received results in an appropriate format based on the user's sentiment.

[1292] User operations

[1293] The user does the following:

[1294] 1. Input data into the system. Emotion recognition is automatic.

[1295] 2. Specify the target variable and problem type for prediction or classification.

[1296] 3. The system processes the data on the server side and waits for the results to be returned. During this time, emotion recognition is also performed in parallel.

[1297] 4. The results are displayed on the device, where you can check the prediction results and evaluation score, and receive emotional feedback.

[1298] Specific examples

[1299] For example, suppose a user prepares the following dataset and performs a classification task:

[1300] Example dataset:

[1301] Feature 1: Age

[1302] Feature 2: Income

[1303] target: purchase history (0 or 1)

[1304] When a user enters this data into the system and selects a classification task, the following occurs:

[1305] 1. Data is received and converted into a DataFrame.

[1306] 2. The emotion engine recognizes the user's emotions and acquires the data.

[1307] 3. Age and income are set as features, and purchase history is set as the objective variable.

[1308] 4. Split the dataset into training and test data, possibly with adjustments based on emotion recognition.

[1309] 5. The server selects a random forest classifier for the classification problem and trains the model on the training data.

[1310] 6. Use the trained model to make predictions on the test data and generate results and evaluation scores.

[1311] 7. The generated prediction results and evaluation scores are adjusted according to the user's sentiment and sent back to the device.

[1312] 8. The device displays the prediction results and evaluation scores to the user. Positive emotions are displayed in a simple format, while negative emotions are displayed with a detailed explanation.

[1313] This allows users to predict and classify data without specialized knowledge, and also receive appropriate feedback based on their emotions.

[1314] The processing flow will be explained below.

[1315] Step 1:

[1316] The user inputs data into the system. The data can be uploaded as a CSV file or entered directly into the system's input form. In addition, the target variable for prediction or classification and the type of problem (classification or regression) are specified.

[1317] Step 2:

[1318] The terminal receives the data entered by the user and converts it into JSON format, which includes features, target variables, and problem types.

[1319] Step 3:

[1320] The device sends JSON format data to the server as a POST request, which also includes emotion data for real-time recognition of the user's emotions.

[1321] Step 4:

[1322] The server receives the request and converts the JSON-formatted data into a Pandas DataFrame, which makes it easier to manipulate the data in your program.

[1323] Step 5:

[1324] The server splits the data frame into target variables and features. The target variables are the objects that the user wants to predict or classify.

[1325] Step 6:

[1326] The server splits the data into training and testing data, typically assigning 80% of the data to training data and 20% to testing data.

[1327] Step 7:

[1328] The emotion engine recognizes the user's emotions in real time as they are being input, and sends the emotion data to the server using techniques such as facial expression recognition and voice analysis.

[1329] Step 8:

[1330] The server selects an appropriate machine learning model depending on the type of problem: a random forest classifier for classification problems, a random forest regressor for regression problems, and model selection and hyperparameter tuning depending on the emotion data.

[1331] Step 9:

[1332] The server trains the selected model using the training data, using the features and the target variable.

[1333] Step 10:

[1334] The server uses the trained model to make predictions on the test data, which produces the prediction results.

[1335] Step 11:

[1336] The server evaluates the model's performance using the test data and calculates a score that indicates the model's accuracy.

[1337] Step 12:

[1338] The emotion engine analyzes the user's emotion data and adjusts the way predictions and rating scores are displayed: simple for positive emotions, and detailed explanations for negative emotions.

[1339] Step 13:

[1340] The server returns the generated prediction results, evaluation scores, and emotion-based adjustment information as a response in JSON format. The response includes a list of prediction results and evaluation scores.

[1341] Step 14:

[1342] The device receives the response from the server and displays the prediction results and evaluation scores to the user, providing tailored feedback based on the user's emotions.

[1343] As a concrete example, let's say a user prepares the following dataset and performs a classification task:

[1344] Example dataset:

[1345] Feature 1: Age

[1346] Feature 2: Income

[1347] target: purchase history (0 or 1)

[1348] When a user enters this data into the system and selects a classification task, the following occurs:

[1349] Steps 1-3:

[1350] The user inputs data, the emotion engine recognizes emotions in real time, and the device sends this data and emotion information to the server.

[1351] Steps 4-6:

[1352] The server receives the data, converts it into a DataFrame, splits it into target variables and features, and then splits the dataset into training data and test data.

[1353] Steps 7-12:

[1354] The server continues to process the data in cooperation with the emotion engine, selecting a random forest classifier for the classification problem and training the model on the training data, then making predictions using the test data to generate results and evaluation scores.

[1355] Steps 13-14:

[1356] The emotion engine adjusts according to the user's emotions and returns the prediction results and evaluation score to the device, which receives and displays them to the user. In this case, a simple display is provided for positive emotions, and a detailed explanation is provided for negative emotions.

[1357] This allows users to predict and classify data without specialized knowledge, and also receive appropriate feedback based on their emotions.

[1358] Example 2

[1359] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1360] Conventional data analysis systems require the specialized knowledge of data scientists, making them difficult for general users to use. Furthermore, they do not provide a user-friendly experience because they do not take into account the user's feelings when processing data or displaying results. This can lead to stress and frustration when users use the system.

[1361] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1362] In this invention, the server includes a data input means, a means for dividing the input data into features and a response variable, a means for dividing the divided data into training data and test data, a means for selecting an appropriate machine learning model according to the type of problem, a means for training the selected model with the training data, a means for generating prediction results and an evaluation score of the model using the test data, a means for outputting the prediction results and the evaluation score, a means for recognizing user emotions in real time, and a means for adjusting data preprocessing and result display based on the user emotions. This enables general users without the specialized knowledge of a data scientist to easily perform data analysis and receive feedback according to the user's emotions.

[1363] "Data input means" refers to a device or function for capturing data input by a user into the system.

[1364] A "feature" is an element of data that a machine learning model uses to make a prediction or classification.

[1365] A "target variable" is an element of data that a machine learning model generates as a prediction or classification result.

[1366] "Training data" is a data set used to train a machine learning model.

[1367] "Test data" is a dataset used to evaluate the performance of a trained machine learning model.

[1368] A "machine learning model" is a computational method or algorithm that learns patterns and regularities from data and makes predictions and classifications for new data.

[1369] The "emotion recognition means" is a device or function for recognizing the user's emotions in real time.

[1370] "Preprocessing means" refers to a device or function that formats and transforms data to make it easier to analyze data or train machine learning models.

[1371] "Feedback" refers to information provided to the user by the system regarding prediction results and evaluation scores, in order to promote the user's understanding and next action.

[1372] This invention is a system that incorporates an emotion engine that recognizes user emotions into a no-code tool that enables general users without the specialized knowledge of a data scientist to easily perform data analysis. This system allows users to receive feedback according to their emotions, providing a more user-friendly experience.

[1373] Server-side hardware and software

[1374] The server of this system uses the following hardware and software:

[1375] Hardware: A server machine with a powerful CPU and sufficient memory

[1376] software:

[1377] Pandas: Used to receive and transform data

[1378] scikit-learn: Used for data splitting, model selection, and model training

[1379] Emotion Engine: Recognize user emotions in real time

[1380] Process Overview

[1381] When a user inputs data and prediction or classification settings, the server automatically selects an appropriate machine learning model based on the data, and then trains and evaluates it. In addition, an emotion engine recognizes the user's emotions and adjusts data preprocessing and result display based on the emotions. The results are returned to the user via the device.

[1382] Terminal hardware and software

[1383] The terminals of this system use the following hardware and software:

[1384] Hardware: Devices with a camera and microphone (e.g., PC, tablet, smartphone)

[1385] software:

[1386] Front-end application: data entry and result display

[1387] Emotion Recognition API: Acquire emotion data from camera and voice input

[1388] System Operation

[1389] 1. Data Entry

[1390] A user uses a terminal to input data, for example, values ​​such as "age" and "income," and specifies a classification task.

[1391] 2. Emotion recognition

[1392] The device uses a built-in camera and microphone to recognize the user's emotions in real time and transmits the emotion data to a server.

[1393] 3. Data Transmission

[1394] The terminal converts the input data into JSON format and sends it to the server.

[1395] 4. Model Selection and Training

[1396] The server converts the received data into a Pandas DataFrame and uses scikit-learn to select and train an appropriate machine learning model. At the same time, the emotion engine retrieves the emotion data and adjusts data preprocessing as needed.

[1397] 5. Prediction and Evaluation

[1398] The server makes predictions on the test data and generates evaluation scores, which are adjusted based on the emotion data.

[1399] 6. Displaying the results

[1400] The device displays the results received from the server in an appropriate format depending on the user's emotions. For example, if the user has a positive emotion, the results are displayed simply, and if the user has a negative emotion, the results are displayed with a detailed explanation.

[1401] Specific examples

[1402] For example, suppose a user has the following dataset and is tasked with classification:

[1403] Example dataset:

[1404] Feature 1: Age

[1405] Feature 2: Income

[1406] target: purchase history (0 or 1)

[1407] Once the user enters this data into the system and selects a classification task, the system operates as follows:

[1408] 1. Data is received and converted into a DataFrame

[1409] The server receives the age and income data and converts it into a Pandas DataFrame.

[1410] 2. Emotion Recognition and Data Segmentation

[1411] The emotion engine recognizes the user's smile and collects the data. The target variable is purchase history, and age and income are divided as features.

[1412] 3. Model training and evaluation

[1413] The server selects and trains a random forest, generating predictions and evaluation scores for the test data.

[1414] 4. Adjusting and displaying the results

[1415] The server adjusts the results according to the user's emotions and sends them back to the device. The results are displayed simply with positive emotions.

[1416] Examples of prompt statements

[1417] "I want to classify data based on age, income, and purchase history. Please let me know the prediction results and evaluation scores."

[1418] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1419] Processing Steps

[1420] Step 1:

[1421] Data Entry

[1422] The user inputs data from the terminal. Specifically, the user inputs feature quantities such as "age" and "income" and objective variables such as "purchase history."

[1423] Input: Age (e.g., 30), Income (e.g., 50000), Purchase History (e.g., 1)

[1424] Output: Input data on the terminal

[1425] Step 2:

[1426] Sending input data

[1427] The terminal converts the input data into JSON format and sends it to the server as a POST request.

[1428] Input: Data entered by the user

[1429] Output: JSON data sent to the server ({"age": 30, "income": 50000, "purchase_history": 1})

[1430] Step 3:

[1431] Acquiring and sending emotion data

[1432] The device uses a camera and microphone to capture the user's emotional data in real time and transmits it to a server.

[1433] Input: Emotion data obtained through a camera or microphone

[1434] Output: Emotion data sent to the server (e.g., {"emotion": "positive"})

[1435] Step 4:

[1436] Receiving and Converting Data

[1437] The server receives the JSON data sent from the terminal and converts it into a DataFrame using the Pandas read_json method.

[1438] Input: JSON data sent from the terminal

[1439] Output: Pandas DataFrame format data

[1440] Step 5:

[1441] emotion recognition

[1442] The emotion engine analyzes the emotion data sent from the server and recognizes the user's emotions in real time.

[1443] Input: Emotion data sent from the server

[1444] Output: Recognized emotion data (e.g., {"emotion": "positive"})

[1445] Step 6:

[1446] Splitting data into features and target variables

[1447] The server splits the features (age, income) and the target variable (purchase history) from the DataFrame.

[1448] Input: Pandas DataFrame format data

[1449] Output: Data split into features and target variables (e.g., features: age and income, target variable: purchase history)

[1450] Step 7:

[1451] Splitting the data into training and test

[1452] The server splits the data into training and test data using the train_test_split function, adjusting the shuffling of the data based on the sentiment data if necessary.

[1453] Input: Data split into features and target variables

[1454] Output: training data and test data (e.g., training data: 80%, test data: 20%)

[1455] Step 8:

[1456] Model selection and adjustment

[1457] The server selects the appropriate machine learning model depending on the type of problem, and may adjust the model selection based on emotion data. For example, random forest is selected for classification problems.

[1458] Input: training data, problem type, emotion data

[1459] Output: The selected machine learning model (e.g., Random Forest)

[1460] Step 9:

[1461] Training the model

[1462] The server uses the selected model to train the training data. Run model.fit(X_train, y_train).

[1463] Input: Training data, selected machine learning model

[1464] Output: A trained machine learning model

[1465] Step 10:

[1466] Prediction and Evaluation

[1467] The server makes predictions using the test data and generates evaluation scores using accuracy_score, confusion_matrix, etc. Run model.predict(X_test).

[1468] Input: trained machine learning model, test data

[1469] Output: Prediction results and evaluation scores (e.g. accuracy, confusion matrix)

[1470] Step 11:

[1471] Adjusting and outputting prediction results

[1472] The server adjusts the prediction results and evaluation scores according to the user's emotions and returns them to the device in an appropriate format, for example, adding a more detailed explanation if the emotion is negative.

[1473] Input: Prediction results, rating scores, sentiment data

[1474] Output: JSON data of adjusted prediction results and evaluation scores

[1475] Step 12:

[1476] Displaying prediction results

[1477] The device displays the prediction results and evaluation scores received from the server in an appropriate format depending on the user's emotions. For example, it displays them simply if the emotion is positive, and in detail if the emotion is negative.

[1478] Input: Adjusted prediction results and evaluation scores sent from the server

[1479] Output: Prediction results and evaluation scores displayed on the device

[1480] (Application example 2)

[1481] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1482] Conventional data prediction and classification systems lack user experience because they do not take user emotions into account when processing or displaying results. Additionally, users without the expertise of a data scientist find it difficult to use the systems.

[1483] The specification processing by the specification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a data input means, a means for dividing the input data into features and a response variable, a means for dividing the divided data into training data and test data, a means for selecting an appropriate machine learning model according to the type of problem, a means for training the selected model with the training data, a means for generating prediction results and evaluation scores of the model using the test data, a means for outputting the prediction results and evaluation scores, an emotion recognition means for recognizing a user's emotions in real time, and a means for adjusting data preprocessing and result display using emotion data acquired by the emotion recognition means. This enables data processing and result display according to the user's emotions, providing a more intuitive and user-friendly system.

[1484] "Data input means" refers to a device or interface that accepts data input by a user.

[1485] "Features" refer to important attributes extracted from data that are used for analysis such as prediction and classification.

[1486] A "target variable" refers to a specific value or label of data that is the target of prediction or classification.

[1487] "Splitting means" refers to a device or algorithm that performs processing to split input data into features and target variables.

[1488] "Training data" refers to the data set used to train a machine learning model.

[1489] "Test data" refers to a dataset that has not been used for training but is used to evaluate the performance of a trained machine learning model.

[1490] "Machine learning model selection means" refers to a device or algorithm for selecting an appropriate machine learning model based on the type of problem given.

[1491] "Model training means" refers to a device or algorithm that trains a selected machine learning model with training data.

[1492] "Prediction result generation means" refers to a device or algorithm that uses test data to generate prediction results and evaluation scores for a trained machine learning model.

[1493] "Output means" refers to a device or interface for presenting the prediction results and evaluation scores to the user in an appropriate format.

[1494] "Emotion recognition means" refers to a device or algorithm for recognizing a user's emotions in real time.

[1495] "Emotion data utilization means" refers to a device or algorithm that utilizes the emotion data acquired by the emotion recognition means to adjust data preprocessing and result display.

[1496] The present invention combines a data prediction and classification system with a user emotion recognition function, allowing users to experience data processing and result display according to their emotions, providing a more intuitive and user-friendly interface.

[1497] System Overview

[1498] The system consists of a data input means, a means for dividing features and target variables, a means for dividing training data and test data, a means for selecting a machine learning model, a means for training the model, a means for generating prediction results and evaluation scores, an output means, a means for recognizing emotions, and a means for using emotion data.

[1499] Hardware and software used

[1500] The system uses the following hardware and software:

[1501] 1. Hardware:

[1502] Webcam: Captures your facial expressions in real time.

[1503] Computer: Handles all processing.

[1504] 2. Software:

[1505] Python: An entire programming language.

[1506] OpenCV: A library for processing video from a webcam.

[1507] FER (Facial Expression Recognition) Library: A library for recognizing user emotions.

[1508] Pandas: A library for data preprocessing.

[1509] Scikit-learn: A library for machine learning models.

[1510] Requests: A library that communicates with the API.

[1511] Processing flow

[1512] server

[1513] The server does the following:

[1514] 1. Receive data entered by the user and convert it into a data frame using Pandas.

[1515] 2. Emotion data of the user is acquired by the emotion recognition means.

[1516] 3. Split the features and objective variables from the data frame and divide it into training data and test data.

[1517] 4. Select an appropriate machine learning model and train the selected model using the training data.

[1518] 5. Use the trained model to make predictions on the test data and generate an evaluation score.

[1519] 6. Use sentiment data to refine prediction results and evaluation scores into appropriate formats.

[1520] 7. Send the results back to the device.

[1521] Terminal

[1522] The terminal does the following:

[1523] 1. Accepts data and prediction / classification settings from the user, converts the input data into JSON format, and sends it to the server.

[1524] 2. Capture user emotion data in real time using a webcam and send it to the server.

[1525] 3. Receive the prediction results and evaluation scores returned from the server and display them in an appropriate format based on the user's sentiment.

[1526] User

[1527] The user does the following:

[1528] 1. Data is input into the system and emotion recognition is done automatically.

[1529] 2. Specify the target variable and problem type for prediction or classification.

[1530] 3. Wait for the server to process the data, and the results will be displayed on the device.

[1531] 4. Check your prediction results and evaluation scores and get feedback.

[1532] Specific examples

[1533] For example, if a user prepares purchase history data and performs a classification task, age and income are the features, and purchase history is the target variable. The system processes it as follows:

[1534] 1. Data is received and converted into a Pandas dataframe.

[1535] 2. The user's emotion data is acquired by the FER library.

[1536] 3. The data is split into features and target variables, and then split into training data and test data.

[1537] 4. A random forest classifier is selected and trained on the training data.

[1538] 5. The trained model is used to make predictions on the test data and generate an evaluation score.

[1539] 6. Taking into account the emotion data, the prediction results and evaluation scores are adjusted and sent back to the device.

[1540] 7. The prediction results and evaluation scores are displayed to the user.

[1541] Example prompt sentence:

[1542] Create a system like this: An application for an online shopping site that uses a webcam to recognize user emotions in real time and recommends appropriate products based on those emotions. You will use OpenCV and FER library to recognize user emotions, and the recommender system will use a machine learning model to recommend products based on the user and emotion data.

[1543] The above is a specific embodiment of the present invention. This system enables emotion-based data preprocessing and result display, allowing users to enjoy a more intuitive interface.

[1544] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1545] Step 1:

[1546] The terminal accepts data and prediction or classification settings from the user. The terminal converts the input data into JSON format and sends it to the server as a POST request. The input in this step is user-entered data, and the output is JSON-formatted data.

[1547] Step 2:

[1548] The device captures the user's facial expressions in real time using a webcam. The FER library is used to recognize the user's emotions from the captured video. The recognized emotion data is also sent to the server. The input in this step is the real-time video, and the output is the user's emotion data.

[1549] Step 3:

[1550] The server receives JSON-formatted data sent from the terminal and converts it into a data frame using Pandas. It then extracts features and target variables from the data frame. The input in this step is JSON-formatted data, and the output is a data frame split into features and target variables.

[1551] Step 4:

[1552] The server splits the data divided into features and objective variables into training data and test data. At the same time, it also performs preprocessing based on emotion data. The input in this step is the data frame divided into features and objective variables and emotion data, and the output is training data and test data.

[1553] Step 5:

[1554] The server selects an appropriate machine learning model depending on the problem type, for example, a random forest classifier for a classification problem. The input in this step is the problem type of the data, and the output is the selected machine learning model.

[1555] Step 6:

[1556] The server uses the selected machine learning model to train the model on the training data. The inputs in this step are the training data and the machine learning model, and the output is the trained model.

[1557] Step 7:

[1558] The server uses the trained model to predict the test data and generate an evaluation score. The inputs in this step are the test data and the trained model, and the outputs are the prediction results and the evaluation score.

[1559] Step 8:

[1560] The server uses the emotion data to adjust the prediction result and evaluation score according to the emotion. For example, if the user expresses a negative emotion, it adds a detailed explanation. The inputs in this step are the prediction result, the evaluation score, and the emotion data, and the output is the adjusted prediction result and evaluation score.

[1561] Step 9:

[1562] The device receives the prediction results and evaluation scores returned from the server and displays them in an appropriate format depending on the user's emotions. For example, a simple display for positive emotions and a detailed explanation for negative emotions. The input in this step is the adjusted prediction results and evaluation scores, and the output is the result displayed to the user.

[1563] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1564] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1565] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1566] [Fourth embodiment]

[1567] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1568] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1569] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1570] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1571] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1572] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1573] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1574] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1575] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1576] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1577] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1578] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1579] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1580] This invention is a no-code tool that allows users without the specialized knowledge of data scientists to easily predict and classify data.

[1581] System Overview

[1582] When a user inputs their data and request, the server automatically selects an appropriate machine learning model based on that data, trains it, and evaluates it, returning the results to the user via their device.

[1583] Server-side processing

[1584] The server processes the data as follows:

[1585] 1. Receive data entered by the user.

[1586] 2. Convert the received data into a Pandas DataFrame.

[1587] 3. Split the target variable and features.

[1588] 4. Split the data into training and test data.

[1589] 5. Select the appropriate model depending on the type of problem (classification or regression).

[1590] 6. Train the selected model on the training data.

[1591] 7. Use the test data to generate model predictions and evaluation scores.

[1592] 8. Output the generated prediction results and evaluation scores in JSON format.

[1593] Operation on the terminal side

[1594] The terminal performs the following actions:

[1595] 1. Accepts data and prediction or classification settings from the user as input.

[1596] 2. Convert the input data into a specified format and send it to the server as a POST request.

[1597] 3. Receive the prediction results and evaluation scores returned from the server.

[1598] 4. Display the received results to the user.

[1599] User operations

[1600] The user does the following:

[1601] 1. Enter data into the system (e.g., upload a CSV file or enter data manually).

[1602] 2. Specify the target variable for prediction or classification and the type of problem (classification or regression).

[1603] 3. The system processes the request on the server side and waits for the results to be returned.

[1604] 4. The results will be displayed on your device, and you can check the prediction results and evaluation score.

[1605] Specific examples

[1606] For example, suppose a user prepares the following dataset and performs a classification task:

[1607] Example dataset:

[1608] Feature 1: Age

[1609] Feature 2: Income

[1610] target: purchase history (0 or 1)

[1611] Once the user enters this data into the system and selects a classification task, the following happens automatically on the server side:

[1612] 1. Data is received and converted into a DataFrame.

[1613] 2. Age and income are set as features, and purchase history is set as the target variable.

[1614] 3. Split the dataset into training and test data.

[1615] 4. The server selects a random forest classifier for the classification problem and trains the model on the training data.

[1616] 5. The trained model is used to make predictions on the test data, generating results and evaluation scores.

[1617] 6. The generated prediction results and evaluation scores are sent back to the user's device and displayed on the screen.

[1618] This allows users to predict and classify data without specialized knowledge, making it easy to make data-based decisions.

[1619] The processing flow will be explained below.

[1620] Step 1:

[1621] The user inputs data into the system. The data can be uploaded as a CSV file or entered directly into the system's input form. This input specifies the target variable for prediction or classification and the type of problem (classification or regression).

[1622] Step 2:

[1623] The terminal receives the data entered by the user and converts it into JSON format, which includes features, target variables, and problem types.

[1624] Step 3:

[1625] The device sends JSON-formatted data to the server as a POST request to an API endpoint hosted by the system (e.g., http: / / localhost:5000 / predict).

[1626] Step 4:

[1627] The server receives the request and converts the JSON-formatted data into a Pandas DataFrame, which makes it easier to manipulate the data in your program.

[1628] Step 5:

[1629] The server splits the data frame into target variables and features. The target variables are the objects that the user wants to predict or classify.

[1630] Step 6:

[1631] The server splits the data into training and testing data, typically assigning 80% of the data to training data and 20% to testing data.

[1632] Step 7:

[1633] The server selects the appropriate machine learning model depending on the type of problem: a random forest classifier for classification problems, and a random forest regressor for regression problems.

[1634] Step 8:

[1635] The server trains the selected model using the training data, using the features and the target variable.

[1636] Step 9:

[1637] The server uses the trained model to make predictions on the test data, which produces the prediction results.

[1638] Step 10:

[1639] The server evaluates the model's performance using the test data and calculates a score that indicates the model's accuracy.

[1640] Step 11:

[1641] The server returns the generated prediction results and evaluation scores as a response in JSON format. The response includes a list of prediction results and evaluation scores.

[1642] Step 12:

[1643] The terminal receives the response from the server and displays the prediction results and evaluation scores to the user, allowing the user to check the results of the data analysis.

[1644] Example 1

[1645] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1646] Conventional data analysis tools are difficult to use for users without specialized knowledge and are cumbersome to operate. In addition, difficulties in data preprocessing and selecting appropriate machine learning models have led to problems with reduced accuracy in prediction results and evaluation scores.

[1647] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1648] In this invention, the server includes a data input means, a means for converting the input data into a data frame, a means for dividing the converted data into features and a target variable, a means for dividing the divided data into training data and test data, a means for selecting an appropriate model depending on the type of problem, a means for training the selected model with the training data, a means for generating prediction results and evaluation scores of the model using the test data, and a means for outputting the prediction results and evaluation scores. This enables users to easily predict and classify data without specialized knowledge.

[1649] The "data input means" is an interface for accepting data from a user.

[1650] The "means for converting into a data frame" is a processing mechanism for converting received data into a tabular data structure.

[1651] "Means for splitting features and objective variables" is a function for extracting and splitting the input data (features) required for analysis and prediction and the target output data (objective variables) from a data frame.

[1652] The "means for splitting into training data and test data" is a processing function for splitting the original data into a data set for model training and a data set for evaluation.

[1653] A "model selection method" is an algorithm for selecting an appropriate machine learning model based on the characteristics of the data and the type of problem (classification or regression).

[1654] "Means for training a model" refers to a process for training a selected machine learning model using training data to improve its predictive capabilities.

[1655] The "means for generating prediction results and evaluation scores" is a function that uses a trained model to make predictions on test data and calculates the prediction results and evaluation indicators such as accuracy.

[1656] The "means for outputting prediction results and evaluation scores" is an interface for providing the generated prediction results and evaluation scores to the user.

[1657] System Overview

[1658] The system of this invention is a no-code tool that allows users without data scientist expertise to easily perform data prediction and classification. It is mainly composed of three elements: a server, a terminal, and a user.

[1659] Server-side processing

[1660] The server receives the data provided by the user and converts it into a data frame format. The specific processing is done using the Python Pandas library. From this data frame, the features and target variables are extracted separately.

[1661] Next, the server splits the data into training and test data using the train_test_split function from scikit-learn. Depending on the type of problem (classification or regression), it selects an appropriate machine learning model: for classification, it selects a random forest classifier, and for regression, it selects a random forest regressor.

[1662] The selected model is trained on the training data. Then, predictions are made using the model on the test data, and prediction results and evaluation scores (e.g., metrics such as precision and recall) are generated. The generated prediction results and evaluation scores are converted into JSON format and sent to the user's device.

[1663] Operation on the terminal side

[1664] The terminal receives data and configuration information (such as classification and prediction requests) from the user, converts it into a specific format, and sends it to the server as a POST request. A simple user interface is provided, allowing the user to operate it intuitively.

[1665] After receiving the prediction results and evaluation scores from the server, the device displays this information to the user, who can then check the results through the input interface or dashboard.

[1666] User operations

[1667] Users input data into the system by uploading a CSV file or manually entering the data. They also specify the target variable for prediction or classification and select whether the problem is classification or regression.

[1668] For example, suppose a user prepares a dataset containing age and income information and performs a classification task on purchase history (0 or 1). Then, the user inputs the dataset into the system and specifies the classification task. In this case, an appropriate model is automatically selected and trained on the server side. As a result, the user can check the prediction results and evaluation scores based on the input data.

[1669] Prompt Sentence Examples

[1670] Below are some examples of prompts to input to the generative AI model.

[1671] "Using the following dataset, create a prompt to classify purchase history (0 or 1). The dataset contains information on age and income."

[1672] This prompt allows the user to ask the generative AI for assistance with the necessary data processing and application of machine learning models.

[1673] This invention allows users to easily perform advanced data analysis without specialized knowledge.

[1674] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1675] Step 1:

[1676] The server receives data entered by the user. The input is sent in the form of a POST request from the terminal. Specifically, this is done by the user uploading a CSV file or entering data directly into a form. The received data is temporarily stored so that it can be passed on to the next processing step as is.

[1677] Step 2:

[1678] The server converts the received data into a data frame using the Pandas library. At this time, the received data (e.g., in CSV format) is read using the Pandas read_csv function and converted into a data frame. The input is the contents of the CSV file, and the output is a Pandas data frame object.

[1679] Step 3:

[1680] The server separates the features and the response variable from the data frame. Specifically, it uses the column names of the data frame to extract the columns designated as response variables and separates the other columns as features. This allows the feature (X) and response variable (y) to be obtained from the input data (all data frames).

[1681] Step 4:

[1682] The server splits the data into training data and test data using the scikit-learn train_test_split function. The input is the features and the objective variable, and the output is the training dataset (X_train, y_train) and the test dataset (X_test, y_test). The split ratio is generally set to 80:20 by default.

[1683] Step 5:

[1684] The server determines the problem type and selects the appropriate machine learning model: the input is the problem type (classification or regression) specified by the user, and the output is the corresponding model object (e.g., a random forest classifier for classification, or a random forest regressor for regression).

[1685] Step 6:

[1686] The server trains the selected model using the training data. Specifically, it uses the model's fit method to input the training data (X_train, y_train) and output the trained model. In this step, the model parameters are optimized.

[1687] Step 7:

[1688] The server uses the trained model to make predictions on the test data. The input is the trained model and test data (X_test), and the output is the prediction result. Specifically, the server makes predictions using the model's predict method.

[1689] Step 8:

[1690] The server generates prediction results and evaluation scores. The inputs are the prediction results and the actual values ​​of the test data (y_test), and the output is the evaluation score (e.g., precision or recall). Specifically, the score is calculated using the evaluation function of scikit-learn.

[1691] Step 9:

[1692] The server converts the generated prediction results and evaluation scores into JSON format and sends them to the terminal. The terminal parses the received JSON data and displays it to the user. The input is the prediction results and evaluation scores, and the output is the information displayed in the user interface.

[1693] Step 10:

[1694] The user checks the prediction results and evaluation scores displayed on the device, and decides on the next action based on the results. Specifically, the results are displayed in the form of a dashboard or report, so the user only needs to refer to this.

[1695] (Application example 1)

[1696] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1697] Predicting failures in factory equipment and robots and efficiently performing maintenance work has a significant impact on reducing downtime and improving productivity. However, conventional predictive maintenance systems require advanced data science knowledge and experts, making them difficult for ordinary workers to use. As a result, there were issues such as not being able to perform maintenance at the appropriate time and making it difficult to prevent failures before they occur.

[1698] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1699] In this invention, the server includes a data input means, a means for dividing the input data into features and a target variable, a means for dividing the divided data into training data and test data, a means for selecting an appropriate machine learning model depending on the type of problem, a means for training the selected model with the training data, a means for generating prediction results and evaluation scores of the model using the test data, a means for outputting the prediction results and evaluation scores, and a means for receiving sensor data and performing pattern recognition and predictive maintenance. This enables even users without specialized knowledge to easily predict failures in equipment and robots in factories and perform maintenance work at appropriate times.

[1700] "Data input means" refers to a device or interface that accepts data input by a user.

[1701] A "feature" is an individual attribute or variable extracted from data for the purpose of making a prediction or classification.

[1702] The "dependent variable" refers to the data output as a result of prediction or classification, and is often called the "target."

[1703] "Training data" refers to the dataset used to train a machine learning model.

[1704] "Test data" refers to a dataset used to evaluate the performance of a trained machine learning model.

[1705] A "machine learning model" refers to an algorithm or mathematical model that makes predictions or classifications based on data.

[1706] "Prediction result" refers to the result derived by the machine learning model based on the input data.

[1707] An "evaluation score" is an index used to quantitatively evaluate the predictive performance of a machine learning model.

[1708] "Sensor Data" means physical or environmental data collected from a sensor device.

[1709] "Pattern recognition" refers to the technique of identifying specific schematic patterns or trends in data.

[1710] "Predictive maintenance" refers to the use of data analysis to make predictions and take appropriate maintenance actions to prevent equipment and system failures and malfunctions.

[1711] This invention is a no-code tool that enables users without the specialized knowledge of data scientists to easily predict and classify data. As an application example, we will explain how to realize a predictive maintenance system installed in a factory robot.

[1712] System Overview

[1713] We provide a system in which a server, a terminal, and a user work together to perform the following operations:

[1714] Hardware and Software

[1715] Hardware

[1716] Various sensors inside the robot (e.g. temperature, vibration, pressure, etc.)

[1717] Centralized control server in the factory

[1718] Factory manager's device (smartphone or PC)

[1719] software

[1720] Python

[1721] Pandas, Scikit-learn (machine learning library)

[1722] Flask (web server framework)

[1723] MQTT (sensor data communication protocol)

[1724] JSON (data format)

[1725] Server-side processing

[1726] The server processes the data as follows:

[1727] 1. Data reception: Receive data sent from each sensor using the MQTT protocol and convert it into a Pandas DataFrame.

[1728] 2. Data preprocessing: Completing missing and outlier values ​​and normalizing data.

[1729] 3. Data splitting: Split the data into features and target variables, and then split it into training data and test data.

[1730] 4. Model selection and training: Select an appropriate machine learning model (e.g., random forest, support vector machine, etc.) depending on the type of problem and train it on the training data.

[1731] 5. Prediction and Evaluation: The test data is used to generate predictions and evaluation scores for the model.

[1732] 6. Result output: The generated prediction results and evaluation scores are generated in JSON format and sent to the terminal.

[1733] Operation on the terminal side

[1734] The terminal performs the following actions:

[1735] 1. User input acceptance: Accepts data (e.g., CSV file upload) and prediction and classification settings as input from the factory manager.

[1736] 2. Send request: The entered data and settings are converted into a specified format and sent to the server as a POST request.

[1737] 3. Receiving and displaying results: The prediction results and evaluation scores returned from the server are received and displayed to the user.

[1738] User operations

[1739] The specific operation steps are as follows:

[1740] 1. Data input: Data acquired from sensors in the factory is input into the system. For example, prepare the following data set:

[1741] Temperature: 35.4, 35.6, 36.0, 36.2, 35.9

[1742] Vibration: 0.05, 0.07, 0.08, 0.09, 0.07

[1743] Pressure: 1.1, 1.2, 1.3, 1.4, 1.2

[1744] Fault history: 0, 0, 1, 0, 0

[1745] 2. Example prompt:

[1746] "Based on the following data, please predict whether or not there will be a breakdown within a week.

[1747] Temperature: 35.4, 35.6, 36.0, 36.2, 35.9

[1748] Vibration: 0.05, 0.07, 0.08, 0.09, 0.07

[1749] Pressure: 1.1, 1.2, 1.3, 1.4, 1.2

[1750] Fault history: 0, 0, 1, 0, 0"

[1751] 3. Prediction and maintenance: The server selects a machine learning model based on the input data, performs failure prediction, and returns the results to the user. The user then performs appropriate maintenance work based on the prediction results.

[1752] This allows even users without specialized knowledge to easily predict failures in equipment and robots within a factory, enabling maintenance work to be carried out at the appropriate time.

[1753] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1754] Step 1:

[1755] Data reception

[1756] The server receives data sent from sensors in the factory using the MQTT protocol. The received data is converted into a Pandas DataFrame. Here, the input is sensor data, and the output is data in Pandas DataFrame format. This conversion makes it easier to handle the data.

[1757] Step 2:

[1758] Data Preprocessing

[1759] The server receives a Pandas DataFrame and performs missing value imputation, outlier processing, and data normalization. The input is raw data in Pandas DataFrame format, and the output is preprocessed data. This preprocessing improves the quality of the data and increases the accuracy of the machine learning model.

[1760] Step 3:

[1761] Data partitioning

[1762] The server splits the preprocessed data into features and targets, and then splits it into training data and test data. The input is the preprocessed data, and the output is a training dataset and a test dataset. This splitting enables training and evaluation of machine learning models.

[1763] Step 4:

[1764] Model Selection and Training

[1765] The server selects an appropriate machine learning model depending on the type of problem (classification or regression) and trains the model using the training data. The input is the training dataset, and the output is the trained machine learning model. This step enables prediction and classification based on the data.

[1766] Step 5:

[1767] Prediction and Evaluation

[1768] The server uses the trained model to make predictions on the test data and generates prediction results and evaluation scores. The inputs are the trained model and the test dataset, and the outputs are the prediction results and evaluation scores. This allows the performance of the model to be quantitatively evaluated.

[1769] Step 6:

[1770] Result output

[1771] The server generates the generated prediction results and evaluation scores in JSON format and sends them to the terminal. The input is the prediction results and evaluation scores, and the output is JSON format data. This step allows the user to easily check the results.

[1772] Step 7:

[1773] Accepting User Input

[1774] The terminal accepts data (e.g., uploading a CSV file) and prediction and classification settings as input from the factory manager. The input is data and settings from the user, and the output is a request to the server. At this step, the user can operate the system and start executing prediction and classification tasks.

[1775] Step 8:

[1776] Send request

[1777] The terminal converts the input data and settings into a specified format and sends it to the server as a POST request. The input is data and settings from the user, and the output is a request to the server. In this step, the terminal requests the server to process the data.

[1778] Step 9:

[1779] Receive and display results

[1780] The terminal receives the prediction results and evaluation scores returned from the server and displays them to the user. The input is JSON-formatted data from the server, and the output is the results displayed to the user. This allows the user to check the prediction results and take appropriate maintenance action.

[1781] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1782] This invention is a system that combines a no-code tool that enables users without specialized data scientist knowledge to easily predict and classify data with an emotion engine that recognizes user emotions. The incorporation of the emotion engine enables data processing and display of results based on the user's emotions, providing a more user-friendly experience.

[1783] System Overview

[1784] When a user inputs data and requests, the server automatically selects an appropriate machine learning model based on the data, and then trains and evaluates it. In addition, an emotion engine recognizes the user's emotions and adjusts data preprocessing and result display based on the emotions. The results are returned to the user via the device.

[1785] Server-side processing

[1786] The server processes the data as follows:

[1787] 1. Receive data entered by the user and convert it to a Pandas DataFrame.

[1788] 2. The emotion engine recognizes the user's emotion and acquires the emotion data.

[1789] 3. Split the target variable and features from the data frame.

[1790] 4. Split the data into training and test data, which may be adjusted based on user sentiment.

[1791] 5. Select the appropriate machine learning model for the type of problem. The sentiment engine adjusts model selection as needed.

[1792] 6. Train the selected model on the training data.

[1793] 7. Use the test data to generate model predictions and evaluation scores.

[1794] 8. The generated prediction results and evaluation scores are adjusted according to the user's sentiment and output in an appropriate format.

[1795] Emotion Engine Operation

[1796] The emotion engine does the following:

[1797] 1. Recognize the user's emotions in real time while they are typing, for example by reading facial expressions through a camera or analyzing the tone of voice through voice input.

[1798] 2. Optimize data preprocessing based on the recognized emotion. For example, if a negative emotion is recognized, the system will perform more careful error detection.

[1799] 3. Adjust the display of the final prediction result and evaluation score depending on the emotion. If a positive emotion is recognized, the result display can be simplified, while if a negative emotion is recognized, a detailed explanation can be added.

[1800] Operation on the terminal side

[1801] The terminal performs the following actions:

[1802] 1. Accepts data and prediction or classification settings from the user as input.

[1803] 2. Convert the input data into JSON format and send it to the server as a POST request.

[1804] 3. The emotion engine is used to obtain the user's emotions in real time, and this information is also sent to the server.

[1805] 4. Receive the prediction results and evaluation scores returned from the server.

[1806] 5. Display the received results in an appropriate format based on the user's sentiment.

[1807] User operations

[1808] The user does the following:

[1809] 1. Input data into the system. Emotion recognition is automatic.

[1810] 2. Specify the target variable and problem type for prediction or classification.

[1811] 3. The system processes the data on the server side and waits for the results to be returned. During this time, emotion recognition is also performed in parallel.

[1812] 4. The results are displayed on the device, where you can check the prediction results and evaluation score, and receive emotional feedback.

[1813] Specific examples

[1814] For example, suppose a user prepares the following dataset and performs a classification task:

[1815] Example dataset:

[1816] Feature 1: Age

[1817] Feature 2: Income

[1818] target: purchase history (0 or 1)

[1819] When a user enters this data into the system and selects a classification task, the following occurs:

[1820] 1. Data is received and converted into a DataFrame.

[1821] 2. The emotion engine recognizes the user's emotions and acquires the data.

[1822] 3. Age and income are set as features, and purchase history is set as the objective variable.

[1823] 4. Split the dataset into training and test data, possibly with adjustments based on emotion recognition.

[1824] 5. The server selects a random forest classifier for the classification problem and trains the model on the training data.

[1825] 6. Use the trained model to make predictions on the test data and generate results and evaluation scores.

[1826] 7. The generated prediction results and evaluation scores are adjusted according to the user's sentiment and sent back to the device.

[1827] 8. The device displays the prediction results and evaluation scores to the user. Positive emotions are displayed in a simple format, while negative emotions are displayed with a detailed explanation.

[1828] This allows users to predict and classify data without specialized knowledge, and also receive appropriate feedback based on their emotions.

[1829] The processing flow will be explained below.

[1830] Step 1:

[1831] The user inputs data into the system. The data can be uploaded as a CSV file or entered directly into the system's input form. In addition, the target variable for prediction or classification and the type of problem (classification or regression) are specified.

[1832] Step 2:

[1833] The terminal receives the data entered by the user and converts it into JSON format, which includes features, target variables, and problem types.

[1834] Step 3:

[1835] The device sends JSON format data to the server as a POST request, which also includes emotion data for real-time recognition of the user's emotions.

[1836] Step 4:

[1837] The server receives the request and converts the JSON-formatted data into a Pandas DataFrame, which makes it easier to manipulate the data in your program.

[1838] Step 5:

[1839] The server splits the data frame into target variables and features. The target variables are the objects that the user wants to predict or classify.

[1840] Step 6:

[1841] The server splits the data into training and testing data, typically assigning 80% of the data to training data and 20% to testing data.

[1842] Step 7:

[1843] The emotion engine recognizes the user's emotions in real time as they are being input, and sends the emotion data to the server using techniques such as facial expression recognition and voice analysis.

[1844] Step 8:

[1845] The server selects an appropriate machine learning model depending on the type of problem: a random forest classifier for classification problems, a random forest regressor for regression problems, and model selection and hyperparameter tuning depending on the emotion data.

[1846] Step 9:

[1847] The server trains the selected model using the training data, using the features and the target variable.

[1848] Step 10:

[1849] The server uses the trained model to make predictions on the test data, which produces the prediction results.

[1850] Step 11:

[1851] The server evaluates the model's performance using the test data and calculates a score that indicates the model's accuracy.

[1852] Step 12:

[1853] The emotion engine analyzes the user's emotion data and adjusts the way predictions and rating scores are displayed: simple for positive emotions, and detailed explanations for negative emotions.

[1854] Step 13:

[1855] The server returns the generated prediction results, evaluation scores, and emotion-based adjustment information as a response in JSON format. The response includes a list of prediction results and evaluation scores.

[1856] Step 14:

[1857] The device receives the response from the server and displays the prediction results and evaluation scores to the user, providing tailored feedback based on the user's emotions.

[1858] As a concrete example, let's say a user prepares the following dataset and performs a classification task:

[1859] Example dataset:

[1860] Feature 1: Age

[1861] Feature 2: Income

[1862] target: purchase history (0 or 1)

[1863] When a user enters this data into the system and selects a classification task, the following occurs:

[1864] Steps 1-3:

[1865] The user inputs data, the emotion engine recognizes emotions in real time, and the device sends this data and emotion information to the server.

[1866] Steps 4-6:

[1867] The server receives the data, converts it into a DataFrame, splits it into target variables and features, and then splits the dataset into training data and test data.

[1868] Steps 7-12:

[1869] The server continues to process the data in cooperation with the emotion engine, selecting a random forest classifier for the classification problem and training the model on the training data, then making predictions using the test data to generate results and evaluation scores.

[1870] Steps 13-14:

[1871] The emotion engine adjusts according to the user's emotions and returns the prediction results and evaluation score to the device, which receives and displays them to the user. In this case, a simple display is provided for positive emotions, and a detailed explanation is provided for negative emotions.

[1872] This allows users to predict and classify data without specialized knowledge, and also receive appropriate feedback based on their emotions.

[1873] Example 2

[1874] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1875] Conventional data analysis systems require the specialized knowledge of data scientists, making them difficult for general users to use. Furthermore, they do not provide a user-friendly experience because they do not take into account the user's feelings when processing data or displaying results. This can lead to stress and frustration when users use the system.

[1876] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1877] In this invention, the server includes a data input means, a means for dividing the input data into features and a response variable, a means for dividing the divided data into training data and test data, a means for selecting an appropriate machine learning model according to the type of problem, a means for training the selected model with the training data, a means for generating prediction results and an evaluation score of the model using the test data, a means for outputting the prediction results and the evaluation score, a means for recognizing user emotions in real time, and a means for adjusting data preprocessing and result display based on the user emotions. This enables general users without the specialized knowledge of a data scientist to easily perform data analysis and receive feedback according to the user's emotions.

[1878] "Data input means" refers to a device or function for capturing data input by a user into the system.

[1879] A "feature" is an element of data that a machine learning model uses to make a prediction or classification.

[1880] A "target variable" is an element of data that a machine learning model generates as a prediction or classification result.

[1881] "Training data" is a data set used to train a machine learning model.

[1882] "Test data" is a dataset used to evaluate the performance of a trained machine learning model.

[1883] A "machine learning model" is a computational method or algorithm that learns patterns and regularities from data and makes predictions and classifications for new data.

[1884] The "emotion recognition means" is a device or function for recognizing the user's emotions in real time.

[1885] "Preprocessing means" refers to a device or function that formats and transforms data to make it easier to analyze data or train machine learning models.

[1886] "Feedback" refers to information provided to the user by the system regarding prediction results and evaluation scores, in order to promote the user's understanding and next action.

[1887] This invention is a system that incorporates an emotion engine that recognizes user emotions into a no-code tool that enables general users without the specialized knowledge of a data scientist to easily perform data analysis. This system allows users to receive feedback according to their emotions, providing a more user-friendly experience.

[1888] Server-side hardware and software

[1889] The server of this system uses the following hardware and software:

[1890] Hardware: A server machine with a powerful CPU and sufficient memory

[1891] software:

[1892] Pandas: Used to receive and transform data

[1893] scikit-learn: Used for data splitting, model selection, and model training

[1894] Emotion Engine: Recognize user emotions in real time

[1895] Process Overview

[1896] When a user inputs data and prediction or classification settings, the server automatically selects an appropriate machine learning model based on the data, and then trains and evaluates it. In addition, an emotion engine recognizes the user's emotions and adjusts data preprocessing and result display based on the emotions. The results are returned to the user via the device.

[1897] Terminal hardware and software

[1898] The terminals of this system use the following hardware and software:

[1899] Hardware: Devices with a camera and microphone (e.g., PC, tablet, smartphone)

[1900] software:

[1901] Front-end application: data entry and result display

[1902] Emotion Recognition API: Acquire emotion data from camera and voice input

[1903] System Operation

[1904] 1. Data Entry

[1905] A user uses a terminal to input data, for example, values ​​such as "age" and "income," and specifies a classification task.

[1906] 2. Emotion recognition

[1907] The device uses a built-in camera and microphone to recognize the user's emotions in real time and transmits the emotion data to a server.

[1908] 3. Data Transmission

[1909] The terminal converts the input data into JSON format and sends it to the server.

[1910] 4. Model Selection and Training

[1911] The server converts the received data into a Pandas DataFrame and uses scikit-learn to select and train an appropriate machine learning model. At the same time, the emotion engine retrieves the emotion data and adjusts data preprocessing as needed.

[1912] 5. Prediction and Evaluation

[1913] The server makes predictions on the test data and generates evaluation scores, which are adjusted based on the emotion data.

[1914] 6. Displaying the results

[1915] The device displays the results received from the server in an appropriate format depending on the user's emotions. For example, if the user has a positive emotion, the results are displayed simply, and if the user has a negative emotion, the results are displayed with a detailed explanation.

[1916] Specific examples

[1917] For example, suppose a user has the following dataset and is tasked with classification:

[1918] Example dataset:

[1919] Feature 1: Age

[1920] Feature 2: Income

[1921] target: purchase history (0 or 1)

[1922] Once the user enters this data into the system and selects a classification task, the system operates as follows:

[1923] 1. Data is received and converted into a DataFrame

[1924] The server receives the age and income data and converts it into a Pandas DataFrame.

[1925] 2. Emotion Recognition and Data Segmentation

[1926] The emotion engine recognizes the user's smile and collects the data. The target variable is purchase history, and age and income are divided as features.

[1927] 3. Model training and evaluation

[1928] The server selects and trains a random forest, generating predictions and evaluation scores for the test data.

[1929] 4. Adjusting and displaying the results

[1930] The server adjusts the results according to the user's emotions and sends them back to the device. The results are displayed simply with positive emotions.

[1931] Examples of prompt statements

[1932] "I want to classify data based on age, income, and purchase history. Please let me know the prediction results and evaluation scores."

[1933] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1934] Processing Steps

[1935] Step 1:

[1936] Data Entry

[1937] The user inputs data from the terminal. Specifically, the user inputs feature quantities such as "age" and "income" and objective variables such as "purchase history."

[1938] Input: Age (e.g., 30), Income (e.g., 50000), Purchase History (e.g., 1)

[1939] Output: Input data on the terminal

[1940] Step 2:

[1941] Sending input data

[1942] The terminal converts the input data into JSON format and sends it to the server as a POST request.

[1943] Input: Data entered by the user

[1944] Output: JSON data sent to the server ({"age": 30, "income": 50000, "purchase_history": 1})

[1945] Step 3:

[1946] Acquiring and sending emotion data

[1947] The device uses a camera and microphone to capture the user's emotional data in real time and transmits it to a server.

[1948] Input: Emotion data obtained through a camera or microphone

[1949] Output: Emotion data sent to the server (e.g., {"emotion": "positive"})

[1950] Step 4:

[1951] Receiving and Converting Data

[1952] The server receives the JSON data sent from the terminal and converts it into a DataFrame using the Pandas read_json method.

[1953] Input: JSON data sent from the terminal

[1954] Output: Pandas DataFrame format data

[1955] Step 5:

[1956] emotion recognition

[1957] The emotion engine analyzes the emotion data sent from the server and recognizes the user's emotions in real time.

[1958] Input: Emotion data sent from the server

[1959] Output: Recognized emotion data (e.g., {"emotion": "positive"})

[1960] Step 6:

[1961] Splitting data into features and target variables

[1962] The server splits the features (age, income) and the target variable (purchase history) from the DataFrame.

[1963] Input: Pandas DataFrame format data

[1964] Output: Data split into features and target variables (e.g., features: age and income, target variable: purchase history)

[1965] Step 7:

[1966] Splitting the data into training and test

[1967] The server splits the data into training and test data using the train_test_split function, adjusting the shuffling of the data based on the sentiment data if necessary.

[1968] Input: Data split into features and target variables

[1969] Output: training data and test data (e.g., training data: 80%, test data: 20%)

[1970] Step 8:

[1971] Model selection and adjustment

[1972] The server selects the appropriate machine learning model depending on the type of problem, and may adjust the model selection based on emotion data. For example, random forest is selected for classification problems.

[1973] Input: training data, problem type, emotion data

[1974] Output: The selected machine learning model (e.g., Random Forest)

[1975] Step 9:

[1976] Training the model

[1977] The server uses the selected model to train the training data. Run model.fit(X_train, y_train).

[1978] Input: Training data, selected machine learning model

[1979] Output: A trained machine learning model

[1980] Step 10:

[1981] Prediction and Evaluation

[1982] The server makes predictions using the test data and generates evaluation scores using accuracy_score, confusion_matrix, etc. Run model.predict(X_test).

[1983] Input: trained machine learning model, test data

[1984] Output: Prediction results and evaluation scores (e.g. accuracy, confusion matrix)

[1985] Step 11:

[1986] Adjusting and outputting prediction results

[1987] The server adjusts the prediction results and evaluation scores according to the user's emotions and returns them to the device in an appropriate format, for example, adding a more detailed explanation if the emotion is negative.

[1988] Input: Prediction results, rating scores, sentiment data

[1989] Output: JSON data of adjusted prediction results and evaluation scores

[1990] Step 12:

[1991] Displaying prediction results

[1992] The device displays the prediction results and evaluation scores received from the server in an appropriate format depending on the user's emotions. For example, it displays them simply if the emotion is positive, and in detail if the emotion is negative.

[1993] Input: Adjusted prediction results and evaluation scores sent from the server

[1994] Output: Prediction results and evaluation scores displayed on the device

[1995] (Application example 2)

[1996] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1997] Conventional data prediction and classification systems lack user experience because they do not take user emotions into account when processing or displaying results. Additionally, users without the expertise of a data scientist find it difficult to use the systems.

[1998] The specification processing by the specification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a data input means, a means for dividing the input data into features and a response variable, a means for dividing the divided data into training data and test data, a means for selecting an appropriate machine learning model according to the type of problem, a means for training the selected model with the training data, a means for generating prediction results and evaluation scores of the model using the test data, a means for outputting the prediction results and evaluation scores, an emotion recognition means for recognizing a user's emotions in real time, and a means for adjusting data preprocessing and result display using emotion data acquired by the emotion recognition means. This enables data processing and result display according to the user's emotions, providing a more intuitive and user-friendly system.

[1999] "Data input means" refers to a device or interface that accepts data input by a user.

[2000] "Features" refer to important attributes extracted from data that are used for analysis such as prediction and classification.

[2001] A "target variable" refers to a specific value or label of data that is the target of prediction or classification.

[2002] "Splitting means" refers to a device or algorithm that performs processing to split input data into features and target variables.

[2003] "Training data" refers to the data set used to train a machine learning model.

[2004] "Test data" refers to a dataset that has not been used for training but is used to evaluate the performance of a trained machine learning model.

[2005] "Machine learning model selection means" refers to a device or algorithm for selecting an appropriate machine learning model based on the type of problem given.

[2006] "Model training means" refers to a device or algorithm that trains a selected machine learning model with training data.

[2007] "Prediction result generation means" refers to a device or algorithm that uses test data to generate prediction results and evaluation scores for a trained machine learning model.

[2008] "Output means" refers to a device or interface for presenting the prediction results and evaluation scores to the user in an appropriate format.

[2009] "Emotion recognition means" refers to a device or algorithm for recognizing a user's emotions in real time.

[2010] "Emotion data utilization means" refers to a device or algorithm that utilizes the emotion data acquired by the emotion recognition means to adjust data preprocessing and result display.

[2011] The present invention combines a data prediction and classification system with a user emotion recognition function, allowing users to experience data processing and result display according to their emotions, providing a more intuitive and user-friendly interface.

[2012] System Overview

[2013] The system consists of a data input means, a means for dividing features and target variables, a means for dividing training data and test data, a means for selecting a machine learning model, a means for training the model, a means for generating prediction results and evaluation scores, an output means, a means for recognizing emotions, and a means for using emotion data.

[2014] Hardware and software used

[2015] The system uses the following hardware and software:

[2016] 1. Hardware:

[2017] Webcam: Captures your facial expressions in real time.

[2018] Computer: Handles all processing.

[2019] 2. Software:

[2020] Python: An entire programming language.

[2021] OpenCV: A library for processing video from a webcam.

[2022] FER (Facial Expression Recognition) Library: A library for recognizing user emotions.

[2023] Pandas: A library for data preprocessing.

[2024] Scikit-learn: A library for machine learning models.

[2025] Requests: A library that communicates with the API.

[2026] Processing flow

[2027] server

[2028] The server does the following:

[2029] 1. Receive data entered by the user and convert it into a data frame using Pandas.

[2030] 2. Emotion data of the user is acquired by the emotion recognition means.

[2031] 3. Split the features and objective variables from the data frame and divide it into training data and test data.

[2032] 4. Select an appropriate machine learning model and train the selected model using the training data.

[2033] 5. Use the trained model to make predictions on the test data and generate an evaluation score.

[2034] 6. Use sentiment data to refine prediction results and evaluation scores into appropriate formats.

[2035] 7. Send the results back to the device.

[2036] Terminal

[2037] The terminal does the following:

[2038] 1. Accepts data and prediction / classification settings from the user, converts the input data into JSON format, and sends it to the server.

[2039] 2. Capture user emotion data in real time using a webcam and send it to the server.

[2040] 3. Receive the prediction results and evaluation scores returned from the server and display them in an appropriate format based on the user's sentiment.

[2041] User

[2042] The user does the following:

[2043] 1. Data is input into the system and emotion recognition is done automatically.

[2044] 2. Specify the target variable and problem type for prediction or classification.

[2045] 3. Wait for the server to process the data, and the results will be displayed on the device.

[2046] 4. Check your prediction results and evaluation scores and get feedback.

[2047] Specific examples

[2048] For example, if a user prepares purchase history data and performs a classification task, age and income are the features, and purchase history is the target variable. The system processes it as follows:

[2049] 1. Data is received and converted into a Pandas dataframe.

[2050] 2. The user's emotion data is acquired by the FER library.

[2051] 3. The data is split into features and target variables, and then split into training data and test data.

[2052] 4. A random forest classifier is selected and trained on the training data.

[2053] 5. The trained model is used to make predictions on the test data and generate an evaluation score.

[2054] 6. Taking into account the emotion data, the prediction results and evaluation scores are adjusted and sent back to the device.

[2055] 7. The prediction results and evaluation scores are displayed to the user.

[2056] Example prompt sentence:

[2057] Create a system like this: An application for an online shopping site that uses a webcam to recognize user emotions in real time and recommends appropriate products based on those emotions. You will use OpenCV and FER library to recognize user emotions, and the recommender system will use a machine learning model to recommend products based on the user and emotion data.

[2058] The above is a specific embodiment of the present invention. This system enables emotion-based data preprocessing and result display, allowing users to enjoy a more intuitive interface.

[2059] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2060] Step 1:

[2061] The terminal accepts data and prediction or classification settings from the user. The terminal converts the input data into JSON format and sends it to the server as a POST request. The input in this step is user-entered data, and the output is JSON-formatted data.

[2062] Step 2:

[2063] The device captures the user's facial expressions in real time using a webcam. The FER library is used to recognize the user's emotions from the captured video. The recognized emotion data is also sent to the server. The input in this step is the real-time video, and the output is the user's emotion data.

[2064] Step 3:

[2065] The server receives JSON-formatted data sent from the terminal and converts it into a data frame using Pandas. It then extracts features and target variables from the data frame. The input in this step is JSON-formatted data, and the output is a data frame split into features and target variables.

[2066] Step 4:

[2067] The server splits the data divided into features and objective variables into training data and test data. At the same time, it also performs preprocessing based on emotion data. The input in this step is the data frame divided into features and objective variables and emotion data, and the output is training data and test data.

[2068] Step 5:

[2069] The server selects an appropriate machine learning model depending on the problem type, for example, a random forest classifier for a classification problem. The input in this step is the problem type of the data, and the output is the selected machine learning model.

[2070] Step 6:

[2071] The server uses the selected machine learning model to train the model on the training data. The inputs in this step are the training data and the machine learning model, and the output is the trained model.

[2072] Step 7:

[2073] The server uses the trained model to predict the test data and generate an evaluation score. The inputs in this step are the test data and the trained model, and the outputs are the prediction results and the evaluation score.

[2074] Step 8:

[2075] The server uses the emotion data to adjust the prediction result and evaluation score according to the emotion. For example, if the user expresses a negative emotion, it adds a detailed explanation. The inputs in this step are the prediction result, the evaluation score, and the emotion data, and the output is the adjusted prediction result and evaluation score.

[2076] Step 9:

[2077] The device receives the prediction results and evaluation scores returned from the server and displays them in an appropriate format depending on the user's emotions. For example, a simple display for positive emotions and a detailed explanation for negative emotions. The input in this step is the adjusted prediction results and evaluation scores, and the output is the result displayed to the user.

[2078] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2079] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2080] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2081] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2082] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2083] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2084] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2085] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2086] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2087] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2088] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2089] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2090] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2091] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2092] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[2093] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[2094] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[2095] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[2096] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[2097] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[2098] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[2099] The following is further disclosed regarding the above embodiment.

[2100] (Claim 1)

[2101] a data input means;

[2102] A means for dividing input data into features and target variables;

[2103] means for splitting the split data into training data and test data;

[2104] A means of selecting the appropriate machine learning model depending on the type of problem;

[2105] means for training the selected model with training data;

[2106] means for generating prediction results and evaluation scores for the model using the test data;

[2107] means for outputting the prediction result and the evaluation score;

[2108] A system including:

[2109] (Claim 2)

[2110] 10. The system of claim 1, wherein the system selects a random forest classifier when the problem type is classification and selects a random forest regressor when the problem type is regression.

[2111] (Claim 3)

[2112] 2. The system according to claim 1, wherein the data input means accepts input from a user and displays the prediction results and evaluation scores to the user.

[2113] "Example 1"

[2114] (Claim 1)

[2115] a data input means;

[2116] A means for converting input data into a data frame;

[2117] A means for dividing the transformed data into features and target variables;

[2118] means for splitting the split data into training data and test data;

[2119] A means of selecting the appropriate model depending on the type of problem;

[2120] means for training the selected model with training data;

[2121] means for generating prediction results and evaluation scores for the model using the test data;

[2122] means for outputting the prediction result and the evaluation score;

[2123] A system including:

[2124] (Claim 2)

[2125] 10. The system of claim 1, wherein the system selects a random forest classifier when the problem type is classification and selects a random forest regressor when the problem type is regression.

[2126] (Claim 3)

[2127] 2. The system according to claim 1, wherein the data input means accepts input from a user and displays the prediction results and evaluation scores to the user.

[2128] "Application Example 1"

[2129] (Claim 1)

[2130] a data input means;

[2131] A means for dividing input data into features and target variables;

[2132] means for splitting the split data into training data and test data;

[2133] A means of selecting the appropriate machine learning model depending on the type of problem;

[2134] means for training the selected model with training data;

[2135] means for generating prediction results and evaluation scores for the model using the test data;

[2136] means for outputting the prediction result and the evaluation score;

[2137] means for receiving sensor data and performing pattern recognition and predictive maintenance;

[2138] A system including:

[2139] (Claim 2)

[2140] 10. The system of claim 1, wherein the system selects a random forest classifier when the problem type is classification and selects a random forest regressor when the problem type is regression.

[2141] (Claim 3)

[2142] 2. The system according to claim 1, wherein the data input means accepts input from a user and displays the prediction results and evaluation scores to the user.

[2143] (Claim 4)

[2144] 2. The system according to claim 1, wherein the sensor data receiving means generates a predictive maintenance result based on the machine failure prediction, and includes means for notifying the user of the maintenance result.

[2145] "Example 2: Combining Emotion Engines"

[2146] (Claim 1)

[2147] a data input means;

[2148] A means for dividing input data into features and target variables;

[2149] means for splitting the split data into training data and test data;

[2150] A means of selecting the appropriate machine learning model depending on the type of problem;

[2151] means for training the selected model with training data;

[2152] means for generating prediction results and evaluation scores for the model using the test data;

[2153] means for outputting the prediction result and the evaluation score;

[2154] means for recognizing user emotions in real time;

[2155] a means for adjusting data preprocessing and result display based on user sentiment;

[2156] A system including:

[2157] (Claim 2)

[2158] 10. The system of claim 1, further comprising: means for selecting a random forest classifier when the problem type is classification and a random forest regressor when the problem type is regression; and means for adjusting the model selection based on user sentiment.

[2159] (Claim 3)

[2160] 2. The system according to claim 1, wherein the data input means accepts input from a user, and provides feedback according to the user's emotions when displaying the prediction results and evaluation scores to the user.

[2161] "Application example 2 when combining emotion engines"

[2162] (Claim 1)

[2163] a data input means;

[2164] A means for dividing input data into features and target variables;

[2165] means for splitting the split data into training data and test data;

[2166] A means of selecting the appropriate machine learning model depending on the type of problem;

[2167] means for training the selected model with training data;

[2168] means for generating prediction results and evaluation scores for the model using the test data;

[2169] means for outputting the prediction result and the evaluation score;

[2170] emotion recognition means for recognizing the user's emotions in real time;

[2171] a means for adjusting data preprocessing and result display using emotion data acquired by the emotion recognition means;

[2172] A system including:

[2173] (Claim 2)

[2174] 10. The system of claim 1, wherein the system selects a random forest classifier when the problem type is classification and selects a random forest regressor when the problem type is regression.

[2175] (Claim 3)

[2176] 2. The system according to claim 1, wherein the data input means accepts input from a user and displays the prediction results and evaluation scores to the user. [Explanation of symbols]

[2177] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a data input means; A means for dividing input data into features and target variables; means for splitting the split data into training data and test data; A means of selecting the appropriate machine learning model depending on the type of problem; means for training the selected model with training data; means for generating prediction results and evaluation scores for the model using the test data; means for outputting the prediction result and the evaluation score; A system including:

2. The system of claim 1 , wherein the system selects a random forest classifier when the problem type is classification and a random forest regressor when the problem type is regression.

3. 2. The system according to claim 1, wherein the data input means accepts input from a user and displays the prediction results and evaluation scores to the user.

Images