System

Abstract

A system is provided.SOLUTION: A system comprising: means for collecting diverse information from a database; means for pre-processing the collected data; means for training a machine learning model using the pre-processed data; means for generating use cases that may occur in a market based on the trained model; means for performing a test of a system based on the generated use cases; and means for analyzing the test results to identify faults or improvements in the system.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] With conventional testing methods, it is extremely difficult to cover all use cases that may occur in the market, which often results in fatal defects being released into the market and disappointing users. To solve this problem, a method is needed to comprehensively test a variety of use cases that may occur in the market based on a variety of information. [Means for solving the problem]

[0005] The present invention provides a system including means for collecting various information from a database, means for preprocessing the collected data, means for training a machine learning model using the preprocessed data, means for generating use cases that may arise in the market based on the trained model, means for testing the system based on the generated use cases, and means for analyzing the test results and identifying defects and areas for improvement in the system, thereby conducting tests that cover various user behaviors and improving the quality of the tests.

[0006] A "database" is a system for systematically storing, searching, and managing large amounts of data.

[0007] "Information" is a general term for a variety of data stored in a database, such as usage history, inquiry details, error logs, and feedback.

[0008] "Preprocessing" refers to the process of cleaning, normalizing, and complementing collected data to prepare it in a form suitable for analysis.

[0009] A "machine learning model" is a collection of algorithms that find patterns and rules in data and use them to make future predictions and classifications.

[0010] "Training" is the process of feeding a machine learning model a large amount of data to learn from and improve the model's accuracy.

[0011] A "use case" is a scenario or simulation of how a user would behave in a particular situation.

[0012] "Testing" is the process of operating and checking a system to ensure that it functions correctly.

[0013] "Analysis" refers to the process of examining test results and data in detail to identify defects and areas for improvement.

[0014] A "defect" is a condition or error that does not meet expected functionality or performance.

[0015] "Improvements" are potential fixes or changes that could make the system work better. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] The present invention relates to an AI system for testing a wide range of use cases. The system collects necessary information from a database, preprocesses it, trains a machine learning model, generates use cases based on the model, and finally executes tests based on these use cases.

[0038] The system configuration is as follows: First, the server collects various information from the database, such as usage history, inquiry details, and error logs. This information is also obtained from the customer support organization and other related systems.

[0039] The server then preprocesses the collected data, which involves cleaning the data, removing incomplete and duplicate data, and normalizing and standardizing the data to create a dataset suitable for analysis.

[0040] Once preprocessing is complete, the server begins training the machine learning model using the preprocessed data. Natural language processing techniques are used for training, and the model learns user behavior patterns. Multiple models are tested, and the best-performing model is selected.

[0041] After training is complete, the server uses the trained AI model to estimate and generate potential use cases in the market. These use cases are expressed as specific scenarios, such as "what happens when a user launches a new smartphone app for the first time" or "what error message should be displayed if a Wi-Fi connection fails."

[0042] The device then receives the use cases generated by the server and runs tests based on them to verify that each system function works as expected, such as displaying a message when a Wi-Fi connection error occurs and the reconnection option.

[0043] Once the test is complete, the device collects the test results and sends them to the server. The server analyzes the received test results and identifies any system defects or areas for improvement. The analysis results are compiled into a report and provided to the developer. This allows specific improvement proposals to be presented, improving the quality of the system.

[0044] As a concrete example, consider the following scenario: A user purchases a new smartwatch and fails to connect to Wi-Fi when starting setup. In this case, the server generates this scenario based on past user data and runs a test on the device. If the test results show that the error message is inappropriate, the server reports the results to the developer and provides feedback to improve the message content.

[0045] In this way, the present invention enables an AI system using diverse data to comprehensively test a variety of use cases and improve the quality of the tests.

[0046] The processing flow will be explained below.

[0047] Step 1:

[0048] The server collects information such as usage history, inquiry details, and error logs from the database, including data from the customer support organization and related systems, and is configured to always retrieve the latest data.

[0049] Step 2:

[0050] The server cleans the collected data, removing incomplete and duplicate data and filling in outliers and missing values. At this stage, data integrity is ensured.

[0051] Step 3:

[0052] The server normalizes the cleaned data and standardizes data of different scales, thereby preparing the data so that the machine learning model can learn efficiently.

[0053] Step 4:

[0054] The server uses the preprocessed data to train a machine learning model, generating training and test datasets and testing different learning algorithms to select the best model.

[0055] Step 5:

[0056] The server uses the selected machine learning model to estimate possible use cases in the market, predicting user behavior patterns and generating use cases as specific scenarios.

[0057] Step 6:

[0058] The server stores the generated use cases in a database and converts them into a form that can be executed on the system under test.

[0059] Step 7:

[0060] The terminal receives the use case data provided by the server, sets up the test environment, and prepares the test according to the pre-defined distribution rules.

[0061] Step 8:

[0062] The terminal executes system tests based on the received use cases, verifying that the system functions correctly for each use case.

[0063] Step 9:

[0064] The terminal records detailed error logs, warning messages, and operation results that occur during the test, and collects test results in real time.

[0065] Step 10:

[0066] The terminals send the collected test results to a server, which allows the test results to be centrally managed.

[0067] Step 11:

[0068] The server analyzes the received test results, carefully examining the results and identifying system defects and areas for improvement.

[0069] Step 12:

[0070] The server creates a report based on the analysis results and provides it to the developers, including specific suggestions for improvement and providing feedback to help improve the quality of the system.

[0071] This is the detailed flow of the system's program processing, which enables comprehensive testing of a wide range of use cases and provides a high-quality system without any defects.

[0072] Example 1

[0073] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0074] Conventional systems face the problem of being unable to comprehensively test a wide range of use cases, making it difficult to properly identify system defects and areas that need optimization. In particular, it is difficult to process and analyze a wide range of data and predict user behavior patterns, which can lead to inappropriate feedback being provided.

[0075] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0076] In this invention, the server includes means for collecting various information from a database, means for preprocessing the collected data, means for training a machine learning model using the preprocessed data, means for generating use cases that may occur in the market based on the trained model, means for executing system tests based on the generated use cases, means for analyzing the test results and identifying system defects and areas for improvement, and means for compiling the analysis results as a report and feeding them back to the developer. This makes it possible to comprehensively test various use cases, and appropriately identify and improve system defects and areas for optimization.

[0077] 1. A "database" is a collection of data that stores information systematically and can be searched and retrieved as needed.

[0078] 2. "Means of collecting information" refers to the function of obtaining data such as usage history, inquiry details, and error logs from databases and related systems.

[0079] 3. "Preprocessing means" refers to the function of cleaning, normalizing, and standardizing collected data, converting it into a state suitable for analysis.

[0080] 4. "Means for training machine learning models" refers to the function of training models based on algorithms using collected data to improve predictive accuracy.

[0081] 5. "Means for generating use cases" refers to the function of using a trained model to create specific scenarios that could occur in the market.

[0082] 6. "Means of executing tests" refers to the testing process that verifies each function of the system based on the generated use cases.

[0083] 7. "Means for analyzing test results" refers to the ability to analyze data obtained from tests and evaluate system performance and defects.

[0084] 8. "Means of providing feedback" refers to the function of compiling the analysis results into a report and presenting improvements to the system to developers and other stakeholders.

[0085] 9. "Test automation tool" means software for automatically testing each function of a system.

[0086] 10. A "dashboard" is an interface that displays analytical data and important indicators in a visually easy-to-understand manner.

[0087] This invention relates to an AI system that comprehensively tests various use cases. The system collects information from a database, preprocesses it, trains a machine learning model, generates use cases based on the model, and finally performs testing to improve the quality of the system.

[0088] The system configuration is as follows: First, the server collects information from the database and other related systems. Specifically, it obtains data such as usage history, inquiry details, and error logs from SQL Server and MongoDB. It can also obtain data in real time from the customer support organization and other related systems via API.

[0089] The server then preprocesses the collected data using Python and the pandas library. First, incomplete and duplicate data is removed, and missing values ​​are imputed as appropriate. Next, the data is normalized and standardized to generate a dataset suitable for analysis. For example, text data is tokenized, and numerical data is converted to Z-scores.

[0090] Once preprocessing is complete, the server begins training the machine learning model using the preprocessed data. TensorFlow and PyTorch are used for training, and natural language processing techniques are used to learn user behavior patterns. Multiple models are tested, and the best-performing model is selected through cross-validation.

[0091] After training is complete, the server uses the trained AI model to generate potential use cases that could occur in the market. This is expressed as a specific scenario using the generative AI model. Examples include "what happens when a user launches a new smartphone app for the first time" and "what error message should be displayed if a Wi-Fi connection fails." Examples of prompts include:

[0092] "Generate error messages that may occur when a user launches a new smartphone app for the first time."

[0093] The terminal then receives the use cases generated from the server and executes system tests based on them. Test automation tools (e.g., Selenium, Appium) are installed on the terminal, and these are used to verify whether each function of the system works as expected. For example, based on the scenario of "What happens when a Wi-Fi connection error occurs," the operation of the reconnection option and the display of error messages are tested.

[0094] Once the test is complete, the device collects the test results and sends them to a server, where they are analyzed using Python or R. The analysis includes statistical evaluations such as the frequency of error messages and system response times.

[0095] Finally, the server compiles the analyzed test results into a report. This report details any system defects or areas for improvement and is provided to developers. For example, it includes specific suggestions for improvement, such as "the reconnection function in the event of a Wi-Fi connection error does not work properly." This feedback is used to optimize the system and improve quality.

[0096] As described above, the present invention can improve the quality of a system by comprehensively testing a variety of use cases and appropriately identifying system defects and areas for improvement.

[0097] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0098] Step 1:

[0099] The server collects information from the database. The input is data such as usage history, inquiry details, and error logs from the database and related systems. The output is the collected raw data, which is obtained using SQL queries or API requests. Specifically, the past six months of inquiry details are obtained from the customer support system via API.

[0100] Step 2:

[0101] The server preprocesses the collected data. The input is the raw data collected in step 1, and the output is cleaned and normalized data. Specifically, it uses Python and the pandas library to remove incomplete and duplicate data and impute missing values ​​as appropriate. It also normalizes and standardizes the data to generate a dataset suitable for analysis. For example, it tokenizes text data and converts numerical data into Z-scores.

[0102] Step 3:

[0103] The server uses the preprocessed data to train a machine learning model. The input is the preprocessed data, and the output is a trained machine learning model. TensorFlow or PyTorch is used for training, and natural language processing techniques are used to learn user behavior patterns. Specifically, multiple models (e.g., random forests, neural networks) are tried, and the best-performing model is selected through cross-validation.

[0104] Step 4:

[0105] The server uses the trained AI model to generate use cases that could occur in the market. The input is a trained machine learning model and a prompt based on it, and the output is a specific use case scenario. For example, it generates "what happens when a user launches a new smartphone app for the first time" or "what error message should be displayed if Wi-Fi connection fails."

[0106] Step 5:

[0107] The terminal receives use cases generated from the server and executes system tests based on them. The input is the generated use case scenario, and the output is the test result. Test automation tools (e.g., Selenium, Appium) are installed on the terminal, and these are used to check whether each function of the system works as expected. For example, based on the scenario of "What happens when a Wi-Fi connection error occurs," the operation of the reconnection option and the display of error messages are tested.

[0108] Step 6:

[0109] Once the test is complete, the terminal collects the test results and sends them to the server. The input is the test results, and the output is the analysis data. The server analyzes the test results using Python or R. Specifically, statistical evaluations such as the frequency of error messages and system response speed are included.

[0110] Step 7:

[0111] The server creates a report based on the analysis of the test results. The input is the analysis data, and the output is a report. This report details any system defects or areas for improvement and is provided to developers. For example, it includes specific improvement suggestions such as "The reconnection function in the event of a Wi-Fi connection error does not work properly." This feedback is used to optimize the system and improve quality.

[0112] (Application example 1)

[0113] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0114] When operating factory robots, there is a need to efficiently perform comprehensive testing for a variety of possible use cases. Currently, manual scenario creation and test execution is the norm, which requires a significant amount of man-hours and has limitations on the quality and comprehensiveness of the tests. In addition, the process of analyzing test results and identifying system defects and areas for improvement is not efficient, which poses the issue of taking a long time to improve the quality of the entire system.

[0115] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0116] In this invention, the server includes means for collecting various information from a database, means for preprocessing the collected data, and means for training a machine learning model using the preprocessed data, which makes it possible to generate various use cases that arise in the operation of factory robots, run system tests based on the use cases, and analyze the test results to identify system defects and areas for improvement.

[0117] A "database" is a system that stores a variety of information in a structured manner and enables efficient query and data retrieval.

[0118] "Preprocessing" is the process of removing incomplete and duplicate data from collected data, and converting it into a format suitable for analysis by normalizing and standardizing it.

[0119] A "machine learning model" is an algorithm that learns from collected and preprocessed data and performs estimation and classification on unknown data.

[0120] A "use case" is a specific example that shows how a system works under specific scenarios or conditions.

[0121] A "factory robot" is a mechanical device used to automate specific tasks on a manufacturing line or in a work environment within a factory.

[0122] "Test results" are evaluation data on the success or failure of operations and performance obtained when a system test is executed.

[0123] A "failure" is a phenomenon in which a system does not function as expected or causes an error.

[0124] "Improvements" are modifications or additional functions that are necessary to improve the performance or functionality of the system.

[0125] The present invention relates to an AI system for conducting tests covering a wide range of use cases. This system can efficiently test a wide range of use cases in the operation of factory robots.

[0126] The system configuration is as follows: First, the server collects various information from the database, such as usage history, error logs, and operation history. This information may also be obtained from sensors and robot control systems within the factory.

[0127] The server then preprocesses the collected data. The preprocessing step involves cleaning the data, removing incomplete and duplicate data, and normalizing and standardizing it to create a dataset suitable for analysis. This is done using Python and the Pandas library.

[0128] Once the preprocessing is complete, the server begins training the machine learning model using the preprocessed data. Training uses natural language processing techniques to learn the behavioral patterns of the factory robots. Using deep learning frameworks such as TensorFlow, multiple models are tested and the best-performing model is selected.

[0129] After training is complete, the server uses the trained AI model to estimate and generate use cases that may occur in the operation of the factory robot. These use cases are expressed as specific scenarios. For example, they include "what to do when a factory robot misidentifies a part during production line work" and "what to do in an emergency when it collides with an obstacle."

[0130] The terminal then receives the use cases generated by the server and executes system tests based on them. The tests verify whether each function of the robot operates as expected. For example, they test the message display and re-operation option when a part recognition error occurs.

[0131] Once the test is complete, the device collects the test results and sends them to the server. The server analyzes the received test results and identifies any system defects or areas for improvement. The analysis results are compiled into a report and provided to the developer. This allows specific improvement proposals to be presented, improving the quality of the system.

[0132] As a concrete example, consider the following scenario: If a factory robot misidentifies a part during line work, resulting in a stoppage of work, the server generates this use case based on past robot operation data and runs a test on the terminal. If the test results show that the error message is inappropriate, the server reports the result to the developer and provides feedback to improve the message content.

[0133] In this way, the present invention enables an AI system using diverse data to comprehensively test a variety of use cases for factory robots, thereby improving the quality of the tests.

[0134] Example prompt sentence:

[0135] "Generate a scenario to test the behavior of a factory robot when it misidentifies a part during production line operation."

[0136] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0137] Step 1:

[0138] The server collects information such as usage history, error logs, and operational history from the database and imports it as data. The input is data obtained from sensors and robot control systems in the factory. The output is a raw data set.

[0139] Step 2:

[0140] The server preprocesses the collected data. Specifically, it cleans the data, removes incomplete or duplicate data, and normalizes and standardizes the data to convert it into a format suitable for analysis. The input is the raw dataset. The output is the preprocessed dataset.

[0141] Step 3:

[0142] The server uses the preprocessed data to train a machine learning model. It uses a deep learning framework such as TensorFlow to train the model on the data. The input is the preprocessed dataset. The output is a trained AI model.

[0143] Step 4:

[0144] The server uses a trained AI model to estimate and generate use cases that may occur in the operation of factory robots. Specifically, it runs an algorithm that generates new use cases based on past operational data. The inputs are the trained AI model and operational data. The output is a use case scenario.

[0145] Step 5:

[0146] The terminal receives the use cases generated from the server and executes system tests based on them. Specifically, the terminal controls the robot according to the operation instructions in the use cases and executes the tests. The input is the use case scenario. The output is the test result data.

[0147] Step 6:

[0148] The terminal collects the test results and sends them to the server. The input is the test result data. The output is the test result sent to the server.

[0149] Step 7:

[0150] The server analyzes the received test results and identifies system defects and areas for improvement. Specifically, it analyzes the test results and runs an algorithm to extract defects and areas for improvement. The input is the test results. The output is an analysis report.

[0151] By performing specific operations at each step, it is possible to comprehensively test a variety of use cases and improve the quality of the tests.

[0152] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0153] ---

[0154] This invention relates to an AI system for comprehensive testing of various use cases, and is characterized by its configuration that incorporates an emotion engine that recognizes user emotions. This system collects various information, including emotion data, preprocesses it, trains a machine learning model, generates use cases based on the model, and tests the system taking into account the emotion data.

[0155] The system configuration is as follows: First, the server collects information such as usage history, inquiry details, error logs, and user emotional data from the database. Emotional data is obtained from the user's text input, facial expressions detected by sensors, tone of voice, etc.

[0156] The server then cleans the collected data, removing incomplete and duplicate data and imputing outliers and missing values. At this stage, data integrity is ensured, and sentiment data is also preprocessed.

[0157] Once preprocessing is complete, the server uses the preprocessed data to train a machine learning model. It generates training and test datasets, tests different learning algorithms, and selects the best model. Sentiment data is also used for training, allowing for more accurate prediction of user behavior patterns.

[0158] After training is complete, the server uses the trained AI model to predict potential use cases that may arise in the market. It generates use cases as specific scenarios, taking into account not only the user's behavioral patterns but also their emotions at the time. Examples include "the stress response when a connection error occurs when a user launches a new smartphone app for the first time" and "the behavior of completing setup while showing positive emotions."

[0159] The device then receives the use cases generated by the server and runs tests on the system based on them. The tests verify that each function of the system works as expected. By taking into account emotional data, it is possible to identify user stress points and functions that generate high satisfaction.

[0160] Once the test is complete, the device collects the test results and sends them to the server. The server analyzes the received test results and identifies any system defects or areas for improvement. The analysis results are compiled into a report and provided to the developer. This allows specific improvement proposals to be presented, improving the quality of the system.

[0161] As a concrete example, consider the following scenario: A user purchases a new smartwatch and may experience Wi-Fi connection failure when starting setup. In this case, the server generates this scenario based on past user data and emotional data, and runs a test on the device. If the test results show that the error message is inappropriate, the server reports the results to the developer and provides feedback to improve the message content.

[0162] In this way, the present invention enables AI systems based on emotion data to comprehensively test a variety of use cases and improve the quality of the tests.

[0163] The processing flow will be explained below.

[0164] ---

[0165] Step 1:

[0166] The server collects usage history, inquiry details, error logs, and user emotional data from a database. Emotional data is collected from multiple sources, including user text input, facial expressions obtained by sensors, and tone of voice.

[0167] Step 2:

[0168] The server cleans the collected data, removing incomplete and duplicate data and imputing outliers and missing values. At this stage, sentiment data is also preprocessed to ensure consistency.

[0169] Step 3:

[0170] The server normalizes the cleaned data, standardizing data on different scales and converting it into a format suitable for analysis. Emotion data is also processed on a consistent scale.

[0171] Step 4:

[0172] The server uses the preprocessed data to train a machine learning model, generating training and test datasets and testing multiple learning algorithms to select the best model. It also trains the model on emotion data to improve its ability to predict user behavior and emotions.

[0173] Step 5:

[0174] The server uses the trained AI model to generate potential use cases in the market. It creates specific scenarios that take into account user behavior and emotions. For example, it could be a scenario where the Wi-Fi connection fails during smartwatch setup, causing frustration for the user.

[0175] Step 6:

[0176] The server stores the generated use cases in a database and converts them into a format that can be executed on the system under test. The use cases also contain emotional data to accurately evaluate the system's response.

[0177] Step 7:

[0178] The terminal receives the use case data provided by the server and sets up the test environment, which is configured to prepare the system for testing based on each use case.

[0179] Step 8:

[0180] The device tests the system based on the received use cases, verifying the system's behavior for each use case, and verifying whether the user's emotional responses are as expected.

[0181] Step 9:

[0182] The device records detailed error logs and warning messages that occur during the test, as well as operational results and user emotional data, and collects this information as test results.

[0183] Step 10:

[0184] The device sends the collected test results to a server, which includes information about the system's behavior and the user's emotions.

[0185] Step 11:

[0186] The server analyzes the received test results, carefully examining the results to identify system defects and areas for improvement. The analysis also includes emotional data, providing feedback based on the user's emotions.

[0187] Step 12:

[0188] The server creates a report based on the analysis results and provides it to the developer, including specific suggestions for improvement, as feedback to improve the quality of the system.

[0189] For example, if a user purchases a new smartwatch and fails to connect to Wi-Fi when starting setup, the server will predict the stress response to a Wi-Fi connection error based on past user data and emotional data, and generate this scenario. If a test is run on the device and the error message is inappropriate, the server will report the results to the developer and provide feedback to improve the message content.

[0190] ---

[0191] The above is a detailed flow of the program processing for a system that combines an emotion engine. This method enables testing that takes into account user behavior patterns and emotions, improving the quality of the system.

[0192] Example 2

[0193] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0194] Conventional system testing has the problem that it is difficult to consider the user's emotional state, making it difficult to accurately identify test obstacles and areas for improvement in the user experience. It is also difficult to cover a wide range of use cases, limiting the improvement of system quality.

[0195] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for collecting various information from a database, a preprocessing means for cleaning and normalizing the collected data, a means for training a machine learning model using the preprocessed data, and a means for generating use cases that take emotion data into consideration based on the trained model. This enables comprehensive testing of various use cases that take emotion data into consideration, thereby improving the quality of the system.

[0196] A "database" is a system for storing, managing, and retrieving information in an organized manner.

[0197] An "information gathering means" is a method or device for obtaining the required data from a database.

[0198] A "preprocessing means" is a method or device for cleaning and normalizing collected data.

[0199] A "means for training a machine learning model" is a method or apparatus for training a model based on an algorithm using pre-processed data.

[0200] A "use case generator" is a method or device for generating market scenarios based on a trained model.

[0201] A "system test execution means" is a method or device for verifying the functionality of the system based on the generated use cases.

[0202] "Test result analysis means" refers to a method or device for analyzing test results and identifying defects and areas for improvement.

[0203] "Emotion data" is information about the user's emotional state based on text input, facial expression data, tone of voice, and the like.

[0204] "Natural language processing technology" is a technology that enables computers to understand, analyze, and generate human language.

[0205] System Program Overview

[0206] This system is an AI system that collects and analyzes various information, including user sentiment data, generates use cases using machine learning models, and tests the system. The specific software used includes the Pandas library for data processing, the Scikit-learn library for machine learning, and the GPT-3 NLP model.

[0207] Data collection

[0208] The server collects the following information from the database:

[0209] Usage history

[0210] Inquiry details

[0211] Error Log

[0212] User emotional data (text input, facial expression data, tone of voice, etc.)

[0213] For example, the server uses an SQL query to select all records in the "Usage History" table from the database. Emotion data is obtained by acquiring the user's facial expression and voice data through the sensor API.

[0214] Data Preprocessing

[0215] The server cleans and normalizes the collected data, removing incomplete and duplicate data and imputing outliers and missing values. It uses the Pandas library to impute NaNs (missing values) in the data frame with the mean, detects outliers using statistical methods, and replaces values ​​beyond ±3 standard deviations with the median.

[0216] Training a machine learning model

[0217] The server uses the preprocessed data to train the machine learning model, which includes the following steps:

[0218] Creating training and test datasets

[0219] Trying different learning algorithms

[0220] Selecting the best model

[0221] The server uses the Scikit-learn library to split the dataset into a training set and a test set in a ratio of 8:2, and runs random forests and support vector machines. The resulting F1 scores are compared to select the best model.

[0222] Use Case Generation

[0223] The server generates use cases using a trained AI model, taking into account emotional data and creating specific scenarios. For example, the server uses an NLP model (e.g., GPT-3) to generate a scenario such as "emotional reactions when a user encounters a connection error when launching a new smartphone app for the first time."

[0224] An example prompt is "Generate a scenario in which you purchase a new smartwatch and the Wi-Fi connection fails when you begin setup."

[0225] Running the tests

[0226] The device receives use cases sent from the server and executes system tests, taking into account emotional data and verifying that each function works as expected. It emulates a smartphone app, generates errors in Wi-Fi connection scenarios, and monitors the user's stress level in real time. It also measures the display time of error messages and the response speed of the user interface.

[0227] Analyzing test results

[0228] The device collects test results and sends them to the server. The server analyzes the results and identifies defects and areas for improvement. Specifically, the program sends runtime log files and stress level measurement data in JSON format to the server. The server analyzes the received data and generates a heat map of defects using a Python script and the Matplotlib library. The report can be downloaded in PDF format.

[0229] In this way, AI systems based on emotion data can comprehensively test a variety of use cases and improve the quality of the tests.

[0230] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0231] Specific explanation of processing steps

[0232] Step 1: Collect data

[0233] The server collects various information from the database. The input of this step is the database, and the output is the collected information (usage history, inquiry content, error log, emotion data).

[0234] Specific behavior:

[0235] The server uses an SQL query to select all records in the "Usage History" table from the database. Emotion data is also obtained from the user's facial expressions and voice data via the sensor API.

[0236] Step 2: Clean and normalize the data

[0237] The server cleans and normalizes the collected data. The input of this step is the collected data and the output is the cleaned and normalized data.

[0238] Specific behavior:

[0239] The server uses the Pandas library to impute missing values ​​in the data frame with the mean and remove duplicates, and also uses statistical methods to detect outliers and replace values ​​beyond ±3 standard deviations with the median.

[0240] Step 3: Train the machine learning model

[0241] The server uses the preprocessed data to train a machine learning model. The input of this step is the preprocessed data, and the output is the trained model.

[0242] Specific behavior:

[0243] The server uses the Scikit-learn library to split the dataset into a training set and a test set in a ratio of 8:2. It then tries out models such as random forest and support vector machine, and selects the best model based on the F1 score.

[0244] Step 4: Generate Use Cases

[0245] The server generates use cases using the trained AI model. The inputs of this step are the trained model and emotion data, and the output is a specific use case scenario.

[0246] Specific behavior:

[0247] The server generates a scenario by inputting a prompt into an NLP model (e.g., GPT-3) such as "the emotional reaction when a user encounters a connection error when launching a new smartphone app for the first time."

[0248] Step 5: Receive use cases and test execution

[0249] The terminal receives the use case sent from the server and executes the system test. The input of this step is the use case scenario, and the output is the test result.

[0250] Specific behavior:

[0251] The device emulates smartphone apps and simulates Wi-Fi connection errors, while monitoring users' stress levels, reaction times, and other metrics in real time during the test.

[0252] Step 6: Submit and analyze test results

[0253] The terminal collects the test results and sends them to the server, which analyzes the received test results and identifies system defects and areas for improvement. The input of this step is the test results, and the output is an analyzed report.

[0254] Specific behavior:

[0255] The device sends runtime log files and stress level measurements in JSON format to a server, which uses a Python script and the Matplotlib library to generate a heat map of defects and compiles it into a PDF report.

[0256] Example of input prompt for generative AI model

[0257] Below are some example prompts to input to a generative AI model:

[0258] "Generate a scenario where you purchase a new smartwatch and fail to connect to Wi-Fi when starting setup."

[0259] The above are the specific processing steps of the system process, and by covering a variety of use cases that combine emotion data, the quality of the system can be improved.

[0260] (Application example 2)

[0261] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0262] In conventional systems, data collection and testing are primarily limited to technical aspects, and evaluation of customer emotional data and usage experience is insufficient. This limits the improvement of user satisfaction, and there is a problem that customer experience in physical stores is not sufficiently improved. In addition, it is difficult to perform simulations and propose improvement proposals using emotional data in real time.

[0263] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0264] In this invention, the server includes means for collecting various information from a data storage device, means for preprocessing the collected data, means for training a computational model using the preprocessed data, means for generating a simulation based on the trained model, means for testing the device based on the generated simulation, means for analyzing the test results and identifying device defects and areas for improvement, means for collecting and analyzing customer emotion data in real time, and means for presenting improvement proposals based on the analysis results. This enables an improved customer experience in physical stores. By acquiring and analyzing customer emotion data in real time, appropriate responses and improvements can be made immediately, thereby improving user satisfaction.

[0265] A "data storage device" is a device for storing various information.

[0266] "Preprocessing" refers to the process of cleaning and normalizing the collected data.

[0267] A "computational model" is a model for data analysis that is trained using machine learning algorithms.

[0268] "Simulation" refers to the virtual reproduction of use cases and situations that may occur in the market based on a trained model.

[0269] "Device testing" is the process of verifying whether a device or system actually operates as expected based on the generated simulation.

[0270] "Test results" refer to the measurement data and operation history obtained when a device test is executed.

[0271] "Analysis" is the process of identifying equipment defects and areas for improvement based on test results.

[0272] "Emotion data" refers to data relating to emotions acquired from the customer's facial expressions, voice, etc.

[0273] "Real-time" refers to the immediate processing and reflection of ongoing events and situations.

[0274] "Customer experience" refers to the experiences a customer has with a product or service and everything related to it.

[0275] "Improvement proposals" are proposals for improving systems and services that are based on the analysis results.

[0276] This invention relates to a system for improving customer experience in physical stores by combining and analyzing various information collected from data storage devices and customer emotion data collected in real time.

[0277] The server collects usage history, inquiry details, error logs, and customer sentiment data from the data storage device. This sentiment data is obtained from user text input, facial expressions, tone of voice, etc. The server then cleans and normalizes the collected data, filling in outliers and missing values. This stage ensures data integrity. The sentiment data is also preprocessed in the same way.

[0278] After preprocessing is complete, the server uses the preprocessed data to train a machine learning model. This training involves trying different learning algorithms and selecting the best model. Using emotion data for training improves the accuracy of predicting user behavior patterns. After training is complete, the server uses the trained machine learning model to generate simulations of possible market events. These are generated as specific scenarios, taking into account user behavior patterns and emotions at the time.

[0279] The terminal then receives the simulation generated by the server and runs a system test based on it. The terminal is equipped with a camera and a voice recognition device, which allows it to collect and analyze customer emotion data in real time. For example, a camera installed at the entrance can analyze the facial expressions of customers entering the store to determine whether they are satisfied or stressed.

[0280] Once the test is complete, the device collects the test results and sends them to the server. The server then analyzes the results and identifies any equipment defects or areas for improvement. It also generates a report based on the analysis results, presenting specific improvement proposals. This allows for rapid improvements to be made to services and layouts in physical stores.

[0281] As a concrete example, consider a system in which a camera is installed at the entrance of a store and the facial expressions of customers as they enter the store are analyzed in real time. The camera and a voice recognition device work together to acquire text data and facial expression data, which are then collected as emotion data. Based on this data, store staff can correct their customer service responses in real time, and the system can automatically suggest improvements to the store layout.

[0282] Example prompt for a generative AI model:

[0283] "Generate optimization proposals for store layout and customer service methods based on today's customer facial expression data."

[0284] This prompt is expected to enable the generative AI model to provide specific improvement suggestions based on emotional data.

[0285] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0286] Step 1:

[0287] The server collects usage history, inquiry details, error logs, and customer sentiment data from the data storage device. Specifically, the server obtains these data through API and stores them in a database. The input data is raw data, and the output is pre-processed, consistent data.

[0288] Step 2:

[0289] The server preprocesses the collected data. Preprocessing includes cleaning and normalizing the data, removing incomplete and duplicate data, and imputing outliers and missing values. Specifically, these operations are performed using the Python pandas library. The input data is raw data, and the output data is cleaned data.

[0290] Step 3:

[0291] The server trains a computational model using the preprocessed data. It tries different machine learning algorithms and selects the best model. It trains the model with particular emphasis on emotion data. Specifically, it uses libraries such as Scikit-learn and TensorFlow. The input data is the preprocessed data, and the output data is the trained model.

[0292] Step 4:

[0293] The server generates a simulation based on the trained model. This involves creating specific scenarios that take into account the user's behavioral patterns and emotions at the time. For example, it includes scenarios such as "customer satisfaction when trying a new product." The input data are the trained model and user data, and the output data is the simulation scenario.

[0294] Step 5:

[0295] The terminal receives the simulation generated by the server and executes the system test based on it. Specifically, it collects and analyzes customer emotion data in real time using a camera and a voice recognition device. The input data is the simulation scenario, and the output data is the test results.

[0296] Step 6:

[0297] The terminal collects the test results and sends them to the server. The server analyzes the received test results and identifies any equipment defects or areas for improvement. Specifically, a Python analysis tool is used for the analysis. The input data is the test results, and the output data is an analysis report.

[0298] Step 7:

[0299] The server creates a report based on the analysis results, proposing specific improvement proposals, and provides it to developers. For example, it proposes improvements to the store layout or changes to customer service methods. The input data is the analysis results, and the output data is a report of the improvement proposals.

[0300] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0301] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0302] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0303] [Second embodiment]

[0304] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0305] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0306] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0307] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0308] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0309] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0310] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0311] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0312] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0313] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0314] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0315] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0316] ---

[0317] The present invention relates to an AI system for testing a wide range of use cases. The system collects necessary information from a database, preprocesses it, trains a machine learning model, generates use cases based on the model, and finally executes tests based on these use cases.

[0318] The system configuration is as follows: First, the server collects various information from the database, such as usage history, inquiry details, and error logs. This information is also obtained from the customer support organization and other related systems.

[0319] The server then preprocesses the collected data, which involves cleaning the data, removing incomplete and duplicate data, and normalizing and standardizing the data to create a dataset suitable for analysis.

[0320] Once preprocessing is complete, the server begins training the machine learning model using the preprocessed data. Natural language processing techniques are used for training, and the model learns user behavior patterns. Multiple models are tested, and the best-performing model is selected.

[0321] After training is complete, the server uses the trained AI model to estimate and generate potential use cases in the market. These use cases are expressed as specific scenarios, such as "what happens when a user launches a new smartphone app for the first time" or "what error message should be displayed if a Wi-Fi connection fails."

[0322] The device then receives the use cases generated by the server and runs tests based on them to verify that each system function works as expected, such as displaying a message when a Wi-Fi connection error occurs and the reconnection option.

[0323] Once the test is complete, the device collects the test results and sends them to the server. The server analyzes the received test results and identifies any system defects or areas for improvement. The analysis results are compiled into a report and provided to the developer. This allows specific improvement proposals to be presented, improving the quality of the system.

[0324] As a concrete example, consider the following scenario: A user purchases a new smartwatch and fails to connect to Wi-Fi when starting setup. In this case, the server generates this scenario based on past user data and runs a test on the device. If the test results show that the error message is inappropriate, the server reports the results to the developer and provides feedback to improve the message content.

[0325] In this way, the present invention enables an AI system using diverse data to comprehensively test a variety of use cases and improve the quality of the tests.

[0326] The processing flow will be explained below.

[0327] ---

[0328] Step 1:

[0329] The server collects information such as usage history, inquiry details, and error logs from the database, including data from the customer support organization and related systems, and is configured to always retrieve the latest data.

[0330] Step 2:

[0331] The server cleans the collected data, removing incomplete and duplicate data and filling in outliers and missing values. At this stage, data integrity is ensured.

[0332] Step 3:

[0333] The server normalizes the cleaned data and standardizes data of different scales, thereby preparing the data so that the machine learning model can learn efficiently.

[0334] Step 4:

[0335] The server uses the preprocessed data to train a machine learning model, generating training and test datasets and testing different learning algorithms to select the best model.

[0336] Step 5:

[0337] The server uses the selected machine learning model to estimate possible use cases in the market, predicting user behavior patterns and generating use cases as specific scenarios.

[0338] Step 6:

[0339] The server stores the generated use cases in a database and converts them into a form that can be executed on the system under test.

[0340] Step 7:

[0341] The terminal receives the use case data provided by the server, sets up the test environment, and prepares the test according to the pre-defined distribution rules.

[0342] Step 8:

[0343] The terminal executes system tests based on the received use cases, verifying that the system functions correctly for each use case.

[0344] Step 9:

[0345] The terminal records detailed error logs, warning messages, and operation results that occur during the test, and collects test results in real time.

[0346] Step 10:

[0347] The terminals send the collected test results to a server, which allows the test results to be centrally managed.

[0348] Step 11:

[0349] The server analyzes the received test results, carefully examining the results and identifying system defects and areas for improvement.

[0350] Step 12:

[0351] The server creates a report based on the analysis results and provides it to the developers, including specific suggestions for improvement and providing feedback to help improve the quality of the system.

[0352] ---

[0353] This is the detailed flow of the system's program processing, which enables comprehensive testing of a wide range of use cases and provides a high-quality system without any defects.

[0354] Example 1

[0355] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0356] Conventional systems face the problem of being unable to comprehensively test a wide range of use cases, making it difficult to properly identify system defects and areas that need optimization. In particular, it is difficult to process and analyze a wide range of data and predict user behavior patterns, which can lead to inappropriate feedback being provided.

[0357] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0358] In this invention, the server includes means for collecting various information from a database, means for preprocessing the collected data, means for training a machine learning model using the preprocessed data, means for generating use cases that may occur in the market based on the trained model, means for executing system tests based on the generated use cases, means for analyzing the test results and identifying system defects and areas for improvement, and means for compiling the analysis results as a report and feeding them back to the developer. This makes it possible to comprehensively test various use cases, and appropriately identify and improve system defects and areas for optimization.

[0359] 1. A "database" is a collection of data that stores information systematically and can be searched and retrieved as needed.

[0360] 2. "Means of collecting information" refers to the function of obtaining data such as usage history, inquiry details, and error logs from databases and related systems.

[0361] 3. "Preprocessing means" refers to the function of cleaning, normalizing, and standardizing collected data, converting it into a state suitable for analysis.

[0362] 4. "Means for training machine learning models" refers to the function of training models based on algorithms using collected data to improve predictive accuracy.

[0363] 5. "Means for generating use cases" refers to the function of using a trained model to create specific scenarios that could occur in the market.

[0364] 6. "Means of executing tests" refers to the testing process that verifies each function of the system based on the generated use cases.

[0365] 7. "Means for analyzing test results" refers to the ability to analyze data obtained from tests and evaluate system performance and defects.

[0366] 8. "Means of providing feedback" refers to the function of compiling the analysis results into a report and presenting improvements to the system to developers and other stakeholders.

[0367] 9. "Test automation tool" means software for automatically testing each function of a system.

[0368] 10. A "dashboard" is an interface that displays analytical data and important indicators in a visually easy-to-understand manner.

[0369] This invention relates to an AI system that comprehensively tests various use cases. The system collects information from a database, preprocesses it, trains a machine learning model, generates use cases based on the model, and finally performs testing to improve the quality of the system.

[0370] The system configuration is as follows: First, the server collects information from the database and other related systems. Specifically, it obtains data such as usage history, inquiry details, and error logs from SQL Server and MongoDB. It can also obtain data in real time from the customer support organization and other related systems via API.

[0371] The server then preprocesses the collected data using Python and the pandas library. First, incomplete and duplicate data is removed, and missing values ​​are imputed as appropriate. Next, the data is normalized and standardized to generate a dataset suitable for analysis. For example, text data is tokenized, and numerical data is converted to Z-scores.

[0372] Once preprocessing is complete, the server begins training the machine learning model using the preprocessed data. TensorFlow and PyTorch are used for training, and natural language processing techniques are used to learn user behavior patterns. Multiple models are tested, and the best-performing model is selected through cross-validation.

[0373] After training is complete, the server uses the trained AI model to generate potential use cases that could occur in the market. This is expressed as a specific scenario using the generative AI model. Examples include "what happens when a user launches a new smartphone app for the first time" and "what error message should be displayed if a Wi-Fi connection fails." Examples of prompts include:

[0374] "Generate error messages that may occur when a user launches a new smartphone app for the first time."

[0375] The terminal then receives the use cases generated from the server and executes system tests based on them. Test automation tools (e.g., Selenium, Appium) are installed on the terminal, and these are used to verify whether each function of the system works as expected. For example, based on the scenario of "What happens when a Wi-Fi connection error occurs," the operation of the reconnection option and the display of error messages are tested.

[0376] Once the test is complete, the device collects the test results and sends them to a server, where they are analyzed using Python or R. The analysis includes statistical evaluations such as the frequency of error messages and system response times.

[0377] Finally, the server compiles the analyzed test results into a report. This report details any system defects or areas for improvement and is provided to developers. For example, it includes specific suggestions for improvement, such as "the reconnection function in the event of a Wi-Fi connection error does not work properly." This feedback is used to optimize the system and improve quality.

[0378] As described above, the present invention can improve the quality of a system by comprehensively testing a variety of use cases and appropriately identifying system defects and areas for improvement.

[0379] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0380] Step 1:

[0381] The server collects information from the database. The input is data such as usage history, inquiry details, and error logs from the database and related systems. The output is the collected raw data, which is obtained using SQL queries or API requests. Specifically, the past six months of inquiry details are obtained from the customer support system via API.

[0382] Step 2:

[0383] The server preprocesses the collected data. The input is the raw data collected in step 1, and the output is cleaned and normalized data. Specifically, it uses Python and the pandas library to remove incomplete and duplicate data and impute missing values ​​as appropriate. It also normalizes and standardizes the data to generate a dataset suitable for analysis. For example, it tokenizes text data and converts numerical data into Z-scores.

[0384] Step 3:

[0385] The server uses the preprocessed data to train a machine learning model. The input is the preprocessed data, and the output is a trained machine learning model. TensorFlow or PyTorch is used for training, and natural language processing techniques are used to learn user behavior patterns. Specifically, multiple models (e.g., random forests, neural networks) are tried, and the best-performing model is selected through cross-validation.

[0386] Step 4:

[0387] The server uses the trained AI model to generate use cases that could occur in the market. The input is a trained machine learning model and a prompt based on it, and the output is a specific use case scenario. For example, it generates "what happens when a user launches a new smartphone app for the first time" or "what error message should be displayed if Wi-Fi connection fails."

[0388] Step 5:

[0389] The terminal receives use cases generated from the server and executes system tests based on them. The input is the generated use case scenario, and the output is the test result. Test automation tools (e.g., Selenium, Appium) are installed on the terminal, and these are used to check whether each function of the system works as expected. For example, based on the scenario of "What happens when a Wi-Fi connection error occurs," the operation of the reconnection option and the display of error messages are tested.

[0390] Step 6:

[0391] Once the test is complete, the terminal collects the test results and sends them to the server. The input is the test results, and the output is the analysis data. The server analyzes the test results using Python or R. Specifically, statistical evaluations such as the frequency of error messages and system response speed are included.

[0392] Step 7:

[0393] The server creates a report based on the analysis of the test results. The input is the analysis data, and the output is a report. This report details any system defects or areas for improvement and is provided to developers. For example, it includes specific improvement suggestions such as "The reconnection function in the event of a Wi-Fi connection error does not work properly." This feedback is used to optimize the system and improve quality.

[0394] (Application example 1)

[0395] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0396] When operating factory robots, there is a need to efficiently perform comprehensive testing for a variety of possible use cases. Currently, manual scenario creation and test execution is the norm, which requires a significant amount of man-hours and has limitations on the quality and comprehensiveness of the tests. In addition, the process of analyzing test results and identifying system defects and areas for improvement is not efficient, which poses the issue of taking a long time to improve the quality of the entire system.

[0397] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0398] In this invention, the server includes means for collecting various information from a database, means for preprocessing the collected data, and means for training a machine learning model using the preprocessed data, which makes it possible to generate various use cases that arise in the operation of factory robots, run system tests based on the use cases, and analyze the test results to identify system defects and areas for improvement.

[0399] A "database" is a system that stores a variety of information in a structured manner and enables efficient query and data retrieval.

[0400] "Preprocessing" is the process of removing incomplete and duplicate data from collected data, and converting it into a format suitable for analysis by normalizing and standardizing it.

[0401] A "machine learning model" is an algorithm that learns from collected and preprocessed data and performs estimation and classification on unknown data.

[0402] A "use case" is a specific example that shows how a system works under specific scenarios or conditions.

[0403] A "factory robot" is a mechanical device used to automate specific tasks on a manufacturing line or in a work environment within a factory.

[0404] "Test results" are evaluation data on the success or failure of operations and performance obtained when a system test is executed.

[0405] A "failure" is a phenomenon in which a system does not function as expected or causes an error.

[0406] "Improvements" are modifications or additional functions that are necessary to improve the performance or functionality of the system.

[0407] The present invention relates to an AI system for conducting tests covering a wide range of use cases. This system can efficiently test a wide range of use cases in the operation of factory robots.

[0408] The system configuration is as follows: First, the server collects various information from the database, such as usage history, error logs, and operation history. This information may also be obtained from sensors and robot control systems within the factory.

[0409] The server then preprocesses the collected data. The preprocessing step involves cleaning the data, removing incomplete and duplicate data, and normalizing and standardizing it to create a dataset suitable for analysis. This is done using Python and the Pandas library.

[0410] Once the preprocessing is complete, the server begins training the machine learning model using the preprocessed data. Training uses natural language processing techniques to learn the behavioral patterns of the factory robots. Using deep learning frameworks such as TensorFlow, multiple models are tested and the best-performing model is selected.

[0411] After training is complete, the server uses the trained AI model to estimate and generate use cases that may occur in the operation of the factory robot. These use cases are expressed as specific scenarios. For example, they include "what to do when a factory robot misidentifies a part during production line work" and "what to do in an emergency when it collides with an obstacle."

[0412] The terminal then receives the use cases generated by the server and executes system tests based on them. The tests verify whether each function of the robot operates as expected. For example, they test the message display and re-operation option when a part recognition error occurs.

[0413] Once the test is complete, the device collects the test results and sends them to the server. The server analyzes the received test results and identifies any system defects or areas for improvement. The analysis results are compiled into a report and provided to the developer. This allows specific improvement proposals to be presented, improving the quality of the system.

[0414] As a concrete example, consider the following scenario: If a factory robot misidentifies a part during line work, resulting in a stoppage of work, the server generates this use case based on past robot operation data and runs a test on the terminal. If the test results show that the error message is inappropriate, the server reports the result to the developer and provides feedback to improve the message content.

[0415] In this way, the present invention enables an AI system using diverse data to comprehensively test a variety of use cases for factory robots, thereby improving the quality of the tests.

[0416] Example prompt sentence:

[0417] "Generate a scenario to test the behavior of a factory robot when it misidentifies a part during production line operation."

[0418] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0419] Step 1:

[0420] The server collects information such as usage history, error logs, and operational history from the database and imports it as data. The input is data obtained from sensors and robot control systems in the factory. The output is a raw data set.

[0421] Step 2:

[0422] The server preprocesses the collected data. Specifically, it cleans the data, removes incomplete or duplicate data, and normalizes and standardizes the data to convert it into a format suitable for analysis. The input is the raw dataset. The output is the preprocessed dataset.

[0423] Step 3:

[0424] The server uses the preprocessed data to train a machine learning model. It uses a deep learning framework such as TensorFlow to train the model on the data. The input is the preprocessed dataset. The output is a trained AI model.

[0425] Step 4:

[0426] The server uses a trained AI model to estimate and generate use cases that may occur in the operation of factory robots. Specifically, it runs an algorithm that generates new use cases based on past operational data. The inputs are the trained AI model and operational data. The output is a use case scenario.

[0427] Step 5:

[0428] The terminal receives the use cases generated from the server and executes system tests based on them. Specifically, the terminal controls the robot according to the operation instructions in the use cases and executes the tests. The input is the use case scenario. The output is the test result data.

[0429] Step 6:

[0430] The terminal collects the test results and sends them to the server. The input is the test result data. The output is the test result sent to the server.

[0431] Step 7:

[0432] The server analyzes the received test results and identifies system defects and areas for improvement. Specifically, it analyzes the test results and runs an algorithm to extract defects and areas for improvement. The input is the test results. The output is an analysis report.

[0433] By performing specific operations at each step, it is possible to comprehensively test a variety of use cases and improve the quality of the tests.

[0434] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0435] ---

[0436] This invention relates to an AI system for comprehensive testing of various use cases, and is characterized by its configuration that incorporates an emotion engine that recognizes user emotions. This system collects various information, including emotion data, preprocesses it, trains a machine learning model, generates use cases based on the model, and tests the system taking into account the emotion data.

[0437] The system configuration is as follows: First, the server collects information such as usage history, inquiry details, error logs, and user emotional data from the database. Emotional data is obtained from the user's text input, facial expressions detected by sensors, tone of voice, etc.

[0438] The server then cleans the collected data, removing incomplete and duplicate data and imputing outliers and missing values. At this stage, data integrity is ensured, and sentiment data is also preprocessed.

[0439] Once preprocessing is complete, the server uses the preprocessed data to train a machine learning model. It generates training and test datasets, tests different learning algorithms, and selects the best model. Sentiment data is also used for training, allowing for more accurate prediction of user behavior patterns.

[0440] After training is complete, the server uses the trained AI model to predict potential use cases that may arise in the market. It generates use cases as specific scenarios, taking into account not only the user's behavioral patterns but also their emotions at the time. Examples include "the stress response when a connection error occurs when a user launches a new smartphone app for the first time" and "the behavior of completing setup while showing positive emotions."

[0441] The device then receives the use cases generated by the server and runs tests on the system based on them. The tests verify that each function of the system works as expected. By taking into account emotional data, it is possible to identify user stress points and functions that generate high satisfaction.

[0442] Once the test is complete, the device collects the test results and sends them to the server. The server analyzes the received test results and identifies any system defects or areas for improvement. The analysis results are compiled into a report and provided to the developer. This allows specific improvement proposals to be presented, improving the quality of the system.

[0443] As a concrete example, consider the following scenario: A user purchases a new smartwatch and may experience Wi-Fi connection failure when starting setup. In this case, the server generates this scenario based on past user data and emotional data, and runs a test on the device. If the test results show that the error message is inappropriate, the server reports the results to the developer and provides feedback to improve the message content.

[0444] In this way, the present invention enables AI systems based on emotion data to comprehensively test a variety of use cases and improve the quality of the tests.

[0445] The processing flow will be explained below.

[0446] ---

[0447] Step 1:

[0448] The server collects usage history, inquiry details, error logs, and user emotional data from a database. Emotional data is collected from multiple sources, including user text input, facial expressions obtained by sensors, and tone of voice.

[0449] Step 2:

[0450] The server cleans the collected data, removing incomplete and duplicate data and imputing outliers and missing values. At this stage, sentiment data is also preprocessed to ensure consistency.

[0451] Step 3:

[0452] The server normalizes the cleaned data, standardizing data on different scales and converting it into a format suitable for analysis. Emotion data is also processed on a consistent scale.

[0453] Step 4:

[0454] The server uses the preprocessed data to train a machine learning model, generating training and test datasets and testing multiple learning algorithms to select the best model. It also trains the model on emotion data to improve its ability to predict user behavior and emotions.

[0455] Step 5:

[0456] The server uses the trained AI model to generate potential use cases in the market. It creates specific scenarios that take into account user behavior and emotions. For example, it could be a scenario where the Wi-Fi connection fails during smartwatch setup, causing frustration for the user.

[0457] Step 6:

[0458] The server stores the generated use cases in a database and converts them into a format that can be executed on the system under test. The use cases also contain emotional data to accurately evaluate the system's response.

[0459] Step 7:

[0460] The terminal receives the use case data provided by the server and sets up the test environment, which is configured to prepare the system for testing based on each use case.

[0461] Step 8:

[0462] The device tests the system based on the received use cases, verifying the system's behavior for each use case, and verifying whether the user's emotional responses are as expected.

[0463] Step 9:

[0464] The device records detailed error logs and warning messages that occur during the test, as well as operational results and user emotional data, and collects this information as test results.

[0465] Step 10:

[0466] The device sends the collected test results to a server, which includes information about the system's behavior and the user's emotions.

[0467] Step 11:

[0468] The server analyzes the received test results, carefully examining the results to identify system defects and areas for improvement. The analysis also includes emotional data, providing feedback based on the user's emotions.

[0469] Step 12:

[0470] The server creates a report based on the analysis results and provides it to the developer, including specific suggestions for improvement, as feedback to improve the quality of the system.

[0471] For example, if a user purchases a new smartwatch and fails to connect to Wi-Fi when starting setup, the server will predict the stress response to a Wi-Fi connection error based on past user data and emotional data, and generate this scenario. If a test is run on the device and the error message is inappropriate, the server will report the results to the developer and provide feedback to improve the message content.

[0472] ---

[0473] The above is a detailed flow of the program processing for a system that combines an emotion engine. This method enables testing that takes into account user behavior patterns and emotions, improving the quality of the system.

[0474] Example 2

[0475] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0476] Conventional system testing has the problem that it is difficult to consider the user's emotional state, making it difficult to accurately identify test obstacles and areas for improvement in the user experience. It is also difficult to cover a wide range of use cases, limiting the improvement of system quality.

[0477] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for collecting various information from a database, a preprocessing means for cleaning and normalizing the collected data, a means for training a machine learning model using the preprocessed data, and a means for generating use cases that take emotion data into consideration based on the trained model. This enables comprehensive testing of various use cases that take emotion data into consideration, thereby improving the quality of the system.

[0478] A "database" is a system for storing, managing, and retrieving information in an organized manner.

[0479] An "information gathering means" is a method or device for obtaining the required data from a database.

[0480] A "preprocessing means" is a method or device for cleaning and normalizing collected data.

[0481] A "means for training a machine learning model" is a method or apparatus for training a model based on an algorithm using pre-processed data.

[0482] A "use case generator" is a method or device for generating market scenarios based on a trained model.

[0483] A "system test execution means" is a method or device for verifying the functionality of the system based on the generated use cases.

[0484] "Test result analysis means" refers to a method or device for analyzing test results and identifying defects and areas for improvement.

[0485] "Emotion data" is information about the user's emotional state based on text input, facial expression data, tone of voice, and the like.

[0486] "Natural language processing technology" is a technology that enables computers to understand, analyze, and generate human language.

[0487] System Program Overview

[0488] This system is an AI system that collects and analyzes various information, including user sentiment data, generates use cases using machine learning models, and tests the system. The specific software used includes the Pandas library for data processing, the Scikit-learn library for machine learning, and the GPT-3 NLP model.

[0489] Data collection

[0490] The server collects the following information from the database:

[0491] Usage history

[0492] Inquiry details

[0493] Error Log

[0494] User emotional data (text input, facial expression data, tone of voice, etc.)

[0495] For example, the server uses an SQL query to select all records in the "Usage History" table from the database. Emotion data is obtained by acquiring the user's facial expression and voice data through the sensor API.

[0496] Data Preprocessing

[0497] The server cleans and normalizes the collected data, removing incomplete and duplicate data and imputing outliers and missing values. It uses the Pandas library to impute NaNs (missing values) in the data frame with the mean, detects outliers using statistical methods, and replaces values ​​beyond ±3 standard deviations with the median.

[0498] Training a machine learning model

[0499] The server uses the preprocessed data to train the machine learning model, which includes the following steps:

[0500] Creating training and test datasets

[0501] Trying different learning algorithms

[0502] Selecting the best model

[0503] The server uses the Scikit-learn library to split the dataset into a training set and a test set in a ratio of 8:2, and runs random forests and support vector machines. The resulting F1 scores are compared to select the best model.

[0504] Use Case Generation

[0505] The server generates use cases using a trained AI model, taking into account emotional data and creating specific scenarios. For example, the server uses an NLP model (e.g., GPT-3) to generate a scenario such as "emotional reactions when a user encounters a connection error when launching a new smartphone app for the first time."

[0506] An example prompt is "Generate a scenario in which you purchase a new smartwatch and the Wi-Fi connection fails when you begin setup."

[0507] Running the tests

[0508] The device receives use cases sent from the server and executes system tests, taking into account emotional data and verifying that each function works as expected. It emulates a smartphone app, generates errors in Wi-Fi connection scenarios, and monitors the user's stress level in real time. It also measures the display time of error messages and the response speed of the user interface.

[0509] Analyzing test results

[0510] The device collects test results and sends them to the server. The server analyzes the results and identifies defects and areas for improvement. Specifically, the program sends runtime log files and stress level measurement data in JSON format to the server. The server analyzes the received data and generates a heat map of defects using a Python script and the Matplotlib library. The report can be downloaded in PDF format.

[0511] In this way, AI systems based on emotion data can comprehensively test a variety of use cases and improve the quality of the tests.

[0512] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0513] Specific explanation of processing steps

[0514] Step 1: Collect data

[0515] The server collects various information from the database. The input of this step is the database, and the output is the collected information (usage history, inquiry content, error log, emotion data).

[0516] Specific behavior:

[0517] The server uses an SQL query to select all records in the "Usage History" table from the database. Emotion data is also obtained from the user's facial expressions and voice data via the sensor API.

[0518] Step 2: Clean and normalize the data

[0519] The server cleans and normalizes the collected data. The input of this step is the collected data and the output is the cleaned and normalized data.

[0520] Specific behavior:

[0521] The server uses the Pandas library to impute missing values ​​in the data frame with the mean and remove duplicates, and also uses statistical methods to detect outliers and replace values ​​beyond ±3 standard deviations with the median.

[0522] Step 3: Train the machine learning model

[0523] The server uses the preprocessed data to train a machine learning model. The input of this step is the preprocessed data, and the output is the trained model.

[0524] Specific behavior:

[0525] The server uses the Scikit-learn library to split the dataset into a training set and a test set in a ratio of 8:2. It then tries out models such as random forest and support vector machine, and selects the best model based on the F1 score.

[0526] Step 4: Generate Use Cases

[0527] The server generates use cases using the trained AI model. The inputs of this step are the trained model and emotion data, and the output is a specific use case scenario.

[0528] Specific behavior:

[0529] The server generates a scenario by inputting a prompt into an NLP model (e.g., GPT-3) such as "the emotional reaction when a user encounters a connection error when launching a new smartphone app for the first time."

[0530] Step 5: Receive use cases and test execution

[0531] The terminal receives the use case sent from the server and executes the system test. The input of this step is the use case scenario, and the output is the test result.

[0532] Specific behavior:

[0533] The device emulates smartphone apps and simulates Wi-Fi connection errors, while monitoring users' stress levels, reaction times, and other metrics in real time during the test.

[0534] Step 6: Submit and analyze test results

[0535] The terminal collects the test results and sends them to the server, which analyzes the received test results and identifies system defects and areas for improvement. The input of this step is the test results, and the output is an analyzed report.

[0536] Specific behavior:

[0537] The device sends runtime log files and stress level measurements in JSON format to a server, which uses a Python script and the Matplotlib library to generate a heat map of defects and compiles it into a PDF report.

[0538] Example of input prompt for generative AI model

[0539] Below are some example prompts to input to a generative AI model:

[0540] "Generate a scenario where you purchase a new smartwatch and fail to connect to Wi-Fi when starting setup."

[0541] The above are the specific processing steps of the system process, and by covering a variety of use cases that combine emotion data, the quality of the system can be improved.

[0542] (Application example 2)

[0543] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0544] In conventional systems, data collection and testing are primarily limited to technical aspects, and evaluation of customer emotional data and usage experience is insufficient. This limits the improvement of user satisfaction, and there is a problem that customer experience in physical stores is not sufficiently improved. In addition, it is difficult to perform simulations and propose improvement proposals using emotional data in real time.

[0545] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0546] In this invention, the server includes means for collecting various information from a data storage device, means for preprocessing the collected data, means for training a computational model using the preprocessed data, means for generating a simulation based on the trained model, means for testing the device based on the generated simulation, means for analyzing the test results and identifying device defects and areas for improvement, means for collecting and analyzing customer emotion data in real time, and means for presenting improvement proposals based on the analysis results. This enables an improved customer experience in physical stores. By acquiring and analyzing customer emotion data in real time, appropriate responses and improvements can be made immediately, thereby improving user satisfaction.

[0547] A "data storage device" is a device for storing various information.

[0548] "Preprocessing" refers to the process of cleaning and normalizing the collected data.

[0549] A "computational model" is a model for data analysis that is trained using machine learning algorithms.

[0550] "Simulation" refers to the virtual reproduction of use cases and situations that may occur in the market based on a trained model.

[0551] "Device testing" is the process of verifying whether a device or system actually operates as expected based on the generated simulation.

[0552] "Test results" refer to the measurement data and operation history obtained when a device test is executed.

[0553] "Analysis" is the process of identifying equipment defects and areas for improvement based on test results.

[0554] "Emotion data" refers to data relating to emotions acquired from the customer's facial expressions, voice, etc.

[0555] "Real-time" refers to the immediate processing and reflection of ongoing events and situations.

[0556] "Customer experience" refers to the experiences a customer has with a product or service and everything related to it.

[0557] "Improvement proposals" are proposals for improving systems and services that are based on the analysis results.

[0558] This invention relates to a system for improving customer experience in physical stores by combining and analyzing various information collected from data storage devices and customer emotion data collected in real time.

[0559] The server collects usage history, inquiry details, error logs, and customer sentiment data from the data storage device. This sentiment data is obtained from user text input, facial expressions, tone of voice, etc. The server then cleans and normalizes the collected data, filling in outliers and missing values. This stage ensures data integrity. The sentiment data is also preprocessed in the same way.

[0560] After preprocessing is complete, the server uses the preprocessed data to train a machine learning model. This training involves trying different learning algorithms and selecting the best model. Using emotion data for training improves the accuracy of predicting user behavior patterns. After training is complete, the server uses the trained machine learning model to generate simulations of possible market events. These are generated as specific scenarios, taking into account user behavior patterns and emotions at the time.

[0561] The terminal then receives the simulation generated by the server and runs a system test based on it. The terminal is equipped with a camera and a voice recognition device, which allows it to collect and analyze customer emotion data in real time. For example, a camera installed at the entrance can analyze the facial expressions of customers entering the store to determine whether they are satisfied or stressed.

[0562] Once the test is complete, the device collects the test results and sends them to the server. The server then analyzes the results and identifies any equipment defects or areas for improvement. It also generates a report based on the analysis results, presenting specific improvement proposals. This allows for rapid improvements to be made to services and layouts in physical stores.

[0563] As a concrete example, consider a system in which a camera is installed at the entrance of a store and the facial expressions of customers as they enter the store are analyzed in real time. The camera and a voice recognition device work together to acquire text data and facial expression data, which are then collected as emotion data. Based on this data, store staff can correct their customer service responses in real time, and the system can automatically suggest improvements to the store layout.

[0564] Example prompt for a generative AI model:

[0565] "Generate optimization proposals for store layout and customer service methods based on today's customer facial expression data."

[0566] This prompt is expected to enable the generative AI model to provide specific improvement suggestions based on emotional data.

[0567] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0568] Step 1:

[0569] The server collects usage history, inquiry details, error logs, and customer sentiment data from the data storage device. Specifically, the server obtains these data through API and stores them in a database. The input data is raw data, and the output is pre-processed, consistent data.

[0570] Step 2:

[0571] The server preprocesses the collected data. Preprocessing includes cleaning and normalizing the data, removing incomplete and duplicate data, and imputing outliers and missing values. Specifically, these operations are performed using the Python pandas library. The input data is raw data, and the output data is cleaned data.

[0572] Step 3:

[0573] The server trains a computational model using the preprocessed data. It tries different machine learning algorithms and selects the best model. It trains the model with particular emphasis on emotion data. Specifically, it uses libraries such as Scikit-learn and TensorFlow. The input data is the preprocessed data, and the output data is the trained model.

[0574] Step 4:

[0575] The server generates a simulation based on the trained model. This involves creating specific scenarios that take into account the user's behavioral patterns and emotions at the time. For example, it includes scenarios such as "customer satisfaction when trying a new product." The input data are the trained model and user data, and the output data is the simulation scenario.

[0576] Step 5:

[0577] The terminal receives the simulation generated by the server and executes the system test based on it. Specifically, it collects and analyzes customer emotion data in real time using a camera and a voice recognition device. The input data is the simulation scenario, and the output data is the test results.

[0578] Step 6:

[0579] The terminal collects the test results and sends them to the server. The server analyzes the received test results and identifies any equipment defects or areas for improvement. Specifically, a Python analysis tool is used for the analysis. The input data is the test results, and the output data is an analysis report.

[0580] Step 7:

[0581] The server creates a report based on the analysis results, proposing specific improvement proposals, and provides it to developers. For example, it proposes improvements to the store layout or changes to customer service methods. The input data is the analysis results, and the output data is a report of the improvement proposals.

[0582] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0583] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0584] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0585] [Third embodiment]

[0586] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0587] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0588] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0589] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0590] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0591] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0592] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0593] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0594] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0595] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0596] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0597] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0598] ---

[0599] The present invention relates to an AI system for testing a wide range of use cases. The system collects necessary information from a database, preprocesses it, trains a machine learning model, generates use cases based on the model, and finally executes tests based on these use cases.

[0600] The system configuration is as follows: First, the server collects various information from the database, such as usage history, inquiry details, and error logs. This information is also obtained from the customer support organization and other related systems.

[0601] The server then preprocesses the collected data, which involves cleaning the data, removing incomplete and duplicate data, and normalizing and standardizing the data to create a dataset suitable for analysis.

[0602] Once preprocessing is complete, the server begins training the machine learning model using the preprocessed data. Natural language processing techniques are used for training, and the model learns user behavior patterns. Multiple models are tested, and the best-performing model is selected.

[0603] After training is complete, the server uses the trained AI model to estimate and generate potential use cases in the market. These use cases are expressed as specific scenarios, such as "what happens when a user launches a new smartphone app for the first time" or "what error message should be displayed if a Wi-Fi connection fails."

[0604] The device then receives the use cases generated by the server and runs tests based on them to verify that each system function works as expected, such as displaying a message when a Wi-Fi connection error occurs and the reconnection option.

[0605] Once the test is complete, the device collects the test results and sends them to the server. The server analyzes the received test results and identifies any system defects or areas for improvement. The analysis results are compiled into a report and provided to the developer. This allows specific improvement proposals to be presented, improving the quality of the system.

[0606] As a concrete example, consider the following scenario: A user purchases a new smartwatch and fails to connect to Wi-Fi when starting setup. In this case, the server generates this scenario based on past user data and runs a test on the device. If the test results show that the error message is inappropriate, the server reports the results to the developer and provides feedback to improve the message content.

[0607] In this way, the present invention enables an AI system using diverse data to comprehensively test a variety of use cases and improve the quality of the tests.

[0608] The processing flow will be explained below.

[0609] ---

[0610] Step 1:

[0611] The server collects information such as usage history, inquiry details, and error logs from the database, including data from the customer support organization and related systems, and is configured to always retrieve the latest data.

[0612] Step 2:

[0613] The server cleans the collected data, removing incomplete and duplicate data and filling in outliers and missing values. At this stage, data integrity is ensured.

[0614] Step 3:

[0615] The server normalizes the cleaned data and standardizes data of different scales, thereby preparing the data so that the machine learning model can learn efficiently.

[0616] Step 4:

[0617] The server uses the preprocessed data to train a machine learning model, generating training and test datasets and testing different learning algorithms to select the best model.

[0618] Step 5:

[0619] The server uses the selected machine learning model to estimate possible use cases in the market, predicting user behavior patterns and generating use cases as specific scenarios.

[0620] Step 6:

[0621] The server stores the generated use cases in a database and converts them into a form that can be executed on the system under test.

[0622] Step 7:

[0623] The terminal receives the use case data provided by the server, sets up the test environment, and prepares the test according to the pre-defined distribution rules.

[0624] Step 8:

[0625] The terminal executes system tests based on the received use cases, verifying that the system functions correctly for each use case.

[0626] Step 9:

[0627] The terminal records detailed error logs, warning messages, and operation results that occur during the test, and collects test results in real time.

[0628] Step 10:

[0629] The terminals send the collected test results to a server, which allows the test results to be centrally managed.

[0630] Step 11:

[0631] The server analyzes the received test results, carefully examining the results and identifying system defects and areas for improvement.

[0632] Step 12:

[0633] The server creates a report based on the analysis results and provides it to the developers, including specific suggestions for improvement and providing feedback to help improve the quality of the system.

[0634] ---

[0635] This is the detailed flow of the system's program processing, which enables comprehensive testing of a wide range of use cases and provides a high-quality system without any defects.

[0636] Example 1

[0637] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0638] Conventional systems face the problem of being unable to comprehensively test a wide range of use cases, making it difficult to properly identify system defects and areas that need optimization. In particular, it is difficult to process and analyze a wide range of data and predict user behavior patterns, which can lead to inappropriate feedback being provided.

[0639] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0640] In this invention, the server includes means for collecting various information from a database, means for preprocessing the collected data, means for training a machine learning model using the preprocessed data, means for generating use cases that may occur in the market based on the trained model, means for executing system tests based on the generated use cases, means for analyzing the test results and identifying system defects and areas for improvement, and means for compiling the analysis results as a report and feeding them back to the developer. This makes it possible to comprehensively test various use cases, and appropriately identify and improve system defects and areas for optimization.

[0641] 1. A "database" is a collection of data that stores information systematically and can be searched and retrieved as needed.

[0642] 2. "Means of collecting information" refers to the function of obtaining data such as usage history, inquiry details, and error logs from databases and related systems.

[0643] 3. "Preprocessing means" refers to the function of cleaning, normalizing, and standardizing collected data, converting it into a state suitable for analysis.

[0644] 4. "Means for training machine learning models" refers to the function of training models based on algorithms using collected data to improve predictive accuracy.

[0645] 5. "Means for generating use cases" refers to the function of using a trained model to create specific scenarios that could occur in the market.

[0646] 6. "Means of executing tests" refers to the testing process that verifies each function of the system based on the generated use cases.

[0647] 7. "Means for analyzing test results" refers to the ability to analyze data obtained from tests and evaluate system performance and defects.

[0648] 8. "Means of providing feedback" refers to the function of compiling the analysis results into a report and presenting improvements to the system to developers and other stakeholders.

[0649] 9. "Test automation tool" means software for automatically testing each function of a system.

[0650] 10. A "dashboard" is an interface that displays analytical data and important indicators in a visually easy-to-understand manner.

[0651] This invention relates to an AI system that comprehensively tests various use cases. The system collects information from a database, preprocesses it, trains a machine learning model, generates use cases based on the model, and finally performs testing to improve the quality of the system.

[0652] The system configuration is as follows: First, the server collects information from the database and other related systems. Specifically, it obtains data such as usage history, inquiry details, and error logs from SQL Server and MongoDB. It can also obtain data in real time from the customer support organization and other related systems via API.

[0653] The server then preprocesses the collected data using Python and the pandas library. First, incomplete and duplicate data is removed, and missing values ​​are imputed as appropriate. Next, the data is normalized and standardized to generate a dataset suitable for analysis. For example, text data is tokenized, and numerical data is converted to Z-scores.

[0654] Once preprocessing is complete, the server begins training the machine learning model using the preprocessed data. TensorFlow and PyTorch are used for training, and natural language processing techniques are used to learn user behavior patterns. Multiple models are tested, and the best-performing model is selected through cross-validation.

[0655] After training is complete, the server uses the trained AI model to generate potential use cases that could occur in the market. This is expressed as a specific scenario using the generative AI model. Examples include "what happens when a user launches a new smartphone app for the first time" and "what error message should be displayed if a Wi-Fi connection fails." Examples of prompts include:

[0656] "Generate error messages that may occur when a user launches a new smartphone app for the first time."

[0657] The terminal then receives the use cases generated from the server and executes system tests based on them. Test automation tools (e.g., Selenium, Appium) are installed on the terminal, and these are used to verify whether each function of the system works as expected. For example, based on the scenario of "What happens when a Wi-Fi connection error occurs," the operation of the reconnection option and the display of error messages are tested.

[0658] Once the test is complete, the device collects the test results and sends them to a server, where they are analyzed using Python or R. The analysis includes statistical evaluations such as the frequency of error messages and system response times.

[0659] Finally, the server compiles the analyzed test results into a report. This report details any system defects or areas for improvement and is provided to developers. For example, it includes specific suggestions for improvement, such as "the reconnection function in the event of a Wi-Fi connection error does not work properly." This feedback is used to optimize the system and improve quality.

[0660] As described above, the present invention can improve the quality of a system by comprehensively testing a variety of use cases and appropriately identifying system defects and areas for improvement.

[0661] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0662] Step 1:

[0663] The server collects information from the database. The input is data such as usage history, inquiry details, and error logs from the database and related systems. The output is the collected raw data, which is obtained using SQL queries or API requests. Specifically, the past six months of inquiry details are obtained from the customer support system via API.

[0664] Step 2:

[0665] The server preprocesses the collected data. The input is the raw data collected in step 1, and the output is cleaned and normalized data. Specifically, it uses Python and the pandas library to remove incomplete and duplicate data and impute missing values ​​as appropriate. It also normalizes and standardizes the data to generate a dataset suitable for analysis. For example, it tokenizes text data and converts numerical data into Z-scores.

[0666] Step 3:

[0667] The server uses the preprocessed data to train a machine learning model. The input is the preprocessed data, and the output is a trained machine learning model. TensorFlow or PyTorch is used for training, and natural language processing techniques are used to learn user behavior patterns. Specifically, multiple models (e.g., random forests, neural networks) are tried, and the best-performing model is selected through cross-validation.

[0668] Step 4:

[0669] The server uses the trained AI model to generate use cases that could occur in the market. The input is a trained machine learning model and a prompt based on it, and the output is a specific use case scenario. For example, it generates "what happens when a user launches a new smartphone app for the first time" or "what error message should be displayed if Wi-Fi connection fails."

[0670] Step 5:

[0671] The terminal receives use cases generated from the server and executes system tests based on them. The input is the generated use case scenario, and the output is the test result. Test automation tools (e.g., Selenium, Appium) are installed on the terminal, and these are used to check whether each function of the system works as expected. For example, based on the scenario of "What happens when a Wi-Fi connection error occurs," the operation of the reconnection option and the display of error messages are tested.

[0672] Step 6:

[0673] Once the test is complete, the terminal collects the test results and sends them to the server. The input is the test results, and the output is the analysis data. The server analyzes the test results using Python or R. Specifically, statistical evaluations such as the frequency of error messages and system response speed are included.

[0674] Step 7:

[0675] The server creates a report based on the analysis of the test results. The input is the analysis data, and the output is a report. This report details any system defects or areas for improvement and is provided to developers. For example, it includes specific improvement suggestions such as "The reconnection function in the event of a Wi-Fi connection error does not work properly." This feedback is used to optimize the system and improve quality.

[0676] (Application example 1)

[0677] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0678] When operating factory robots, there is a need to efficiently perform comprehensive testing for a variety of possible use cases. Currently, manual scenario creation and test execution is the norm, which requires a significant amount of man-hours and has limitations on the quality and comprehensiveness of the tests. In addition, the process of analyzing test results and identifying system defects and areas for improvement is not efficient, which poses the issue of taking a long time to improve the quality of the entire system.

[0679] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0680] In this invention, the server includes means for collecting various information from a database, means for preprocessing the collected data, and means for training a machine learning model using the preprocessed data, which makes it possible to generate various use cases that arise in the operation of factory robots, run system tests based on the use cases, and analyze the test results to identify system defects and areas for improvement.

[0681] A "database" is a system that stores a variety of information in a structured manner and enables efficient query and data retrieval.

[0682] "Preprocessing" is the process of removing incomplete and duplicate data from collected data, and converting it into a format suitable for analysis by normalizing and standardizing it.

[0683] A "machine learning model" is an algorithm that learns from collected and preprocessed data and performs estimation and classification on unknown data.

[0684] A "use case" is a specific example that shows how a system works under specific scenarios or conditions.

[0685] A "factory robot" is a mechanical device used to automate specific tasks on a manufacturing line or in a work environment within a factory.

[0686] "Test results" are evaluation data on the success or failure of operations and performance obtained when a system test is executed.

[0687] A "failure" is a phenomenon in which a system does not function as expected or causes an error.

[0688] "Improvements" are modifications or additional functions that are necessary to improve the performance or functionality of the system.

[0689] The present invention relates to an AI system for conducting tests covering a wide range of use cases. This system can efficiently test a wide range of use cases in the operation of factory robots.

[0690] The system configuration is as follows: First, the server collects various information from the database, such as usage history, error logs, and operation history. This information may also be obtained from sensors and robot control systems within the factory.

[0691] The server then preprocesses the collected data. The preprocessing step involves cleaning the data, removing incomplete and duplicate data, and normalizing and standardizing it to create a dataset suitable for analysis. This is done using Python and the Pandas library.

[0692] Once the preprocessing is complete, the server begins training the machine learning model using the preprocessed data. Training uses natural language processing techniques to learn the behavioral patterns of the factory robots. Using deep learning frameworks such as TensorFlow, multiple models are tested and the best-performing model is selected.

[0693] After training is complete, the server uses the trained AI model to estimate and generate use cases that may occur in the operation of the factory robot. These use cases are expressed as specific scenarios. For example, they include "what to do when a factory robot misidentifies a part during production line work" and "what to do in an emergency when it collides with an obstacle."

[0694] The terminal then receives the use cases generated by the server and executes system tests based on them. The tests verify whether each function of the robot operates as expected. For example, they test the message display and re-operation option when a part recognition error occurs.

[0695] Once the test is complete, the device collects the test results and sends them to the server. The server analyzes the received test results and identifies any system defects or areas for improvement. The analysis results are compiled into a report and provided to the developer. This allows specific improvement proposals to be presented, improving the quality of the system.

[0696] As a concrete example, consider the following scenario: If a factory robot misidentifies a part during line work, resulting in a stoppage of work, the server generates this use case based on past robot operation data and runs a test on the terminal. If the test results show that the error message is inappropriate, the server reports the result to the developer and provides feedback to improve the message content.

[0697] In this way, the present invention enables an AI system using diverse data to comprehensively test a variety of use cases for factory robots, thereby improving the quality of the tests.

[0698] Example prompt sentence:

[0699] "Generate a scenario to test the behavior of a factory robot when it misidentifies a part during production line operation."

[0700] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0701] Step 1:

[0702] The server collects information such as usage history, error logs, and operational history from the database and imports it as data. The input is data obtained from sensors and robot control systems in the factory. The output is a raw data set.

[0703] Step 2:

[0704] The server preprocesses the collected data. Specifically, it cleans the data, removes incomplete or duplicate data, and normalizes and standardizes the data to convert it into a format suitable for analysis. The input is the raw dataset. The output is the preprocessed dataset.

[0705] Step 3:

[0706] The server uses the preprocessed data to train a machine learning model. It uses a deep learning framework such as TensorFlow to train the model on the data. The input is the preprocessed dataset. The output is a trained AI model.

[0707] Step 4:

[0708] The server uses a trained AI model to estimate and generate use cases that may occur in the operation of factory robots. Specifically, it runs an algorithm that generates new use cases based on past operational data. The inputs are the trained AI model and operational data. The output is a use case scenario.

[0709] Step 5:

[0710] The terminal receives the use cases generated from the server and executes system tests based on them. Specifically, the terminal controls the robot according to the operation instructions in the use cases and executes the tests. The input is the use case scenario. The output is the test result data.

[0711] Step 6:

[0712] The terminal collects the test results and sends them to the server. The input is the test result data. The output is the test result sent to the server.

[0713] Step 7:

[0714] The server analyzes the received test results and identifies system defects and areas for improvement. Specifically, it analyzes the test results and runs an algorithm to extract defects and areas for improvement. The input is the test results. The output is an analysis report.

[0715] By performing specific operations at each step, it is possible to comprehensively test a variety of use cases and improve the quality of the tests.

[0716] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0717] ---

[0718] This invention relates to an AI system for comprehensive testing of various use cases, and is characterized by its configuration that incorporates an emotion engine that recognizes user emotions. This system collects various information, including emotion data, preprocesses it, trains a machine learning model, generates use cases based on the model, and tests the system taking into account the emotion data.

[0719] The system configuration is as follows: First, the server collects information such as usage history, inquiry details, error logs, and user emotional data from the database. Emotional data is obtained from the user's text input, facial expressions detected by sensors, tone of voice, etc.

[0720] The server then cleans the collected data, removing incomplete and duplicate data and imputing outliers and missing values. At this stage, data integrity is ensured, and sentiment data is also preprocessed.

[0721] Once preprocessing is complete, the server uses the preprocessed data to train a machine learning model. It generates training and test datasets, tests different learning algorithms, and selects the best model. Sentiment data is also used for training, allowing for more accurate prediction of user behavior patterns.

[0722] After training is complete, the server uses the trained AI model to predict potential use cases that may arise in the market. It generates use cases as specific scenarios, taking into account not only the user's behavioral patterns but also their emotions at the time. Examples include "the stress response when a connection error occurs when a user launches a new smartphone app for the first time" and "the behavior of completing setup while showing positive emotions."

[0723] The device then receives the use cases generated by the server and runs tests on the system based on them. The tests verify that each function of the system works as expected. By taking into account emotional data, it is possible to identify user stress points and functions that generate high satisfaction.

[0724] Once the test is complete, the device collects the test results and sends them to the server. The server analyzes the received test results and identifies any system defects or areas for improvement. The analysis results are compiled into a report and provided to the developer. This allows specific improvement proposals to be presented, improving the quality of the system.

[0725] As a concrete example, consider the following scenario: A user purchases a new smartwatch and may experience Wi-Fi connection failure when starting setup. In this case, the server generates this scenario based on past user data and emotional data, and runs a test on the device. If the test results show that the error message is inappropriate, the server reports the results to the developer and provides feedback to improve the message content.

[0726] In this way, the present invention enables AI systems based on emotion data to comprehensively test a variety of use cases and improve the quality of the tests.

[0727] The processing flow will be explained below.

[0728] ---

[0729] Step 1:

[0730] The server collects usage history, inquiry details, error logs, and user emotional data from a database. Emotional data is collected from multiple sources, including user text input, facial expressions obtained by sensors, and tone of voice.

[0731] Step 2:

[0732] The server cleans the collected data, removing incomplete and duplicate data and imputing outliers and missing values. At this stage, sentiment data is also preprocessed to ensure consistency.

[0733] Step 3:

[0734] The server normalizes the cleaned data, standardizing data on different scales and converting it into a format suitable for analysis. Emotion data is also processed on a consistent scale.

[0735] Step 4:

[0736] The server uses the preprocessed data to train a machine learning model, generating training and test datasets and testing multiple learning algorithms to select the best model. It also trains the model on emotion data to improve its ability to predict user behavior and emotions.

[0737] Step 5:

[0738] The server uses the trained AI model to generate potential use cases in the market. It creates specific scenarios that take into account user behavior and emotions. For example, it could be a scenario where the Wi-Fi connection fails during smartwatch setup, causing frustration for the user.

[0739] Step 6:

[0740] The server stores the generated use cases in a database and converts them into a format that can be executed on the system under test. The use cases also contain emotional data to accurately evaluate the system's response.

[0741] Step 7:

[0742] The terminal receives the use case data provided by the server and sets up the test environment, which is configured to prepare the system for testing based on each use case.

[0743] Step 8:

[0744] The device tests the system based on the received use cases, verifying the system's behavior for each use case, and verifying whether the user's emotional responses are as expected.

[0745] Step 9:

[0746] The device records detailed error logs and warning messages that occur during the test, as well as operational results and user emotional data, and collects this information as test results.

[0747] Step 10:

[0748] The device sends the collected test results to a server, which includes information about the system's behavior and the user's emotions.

[0749] Step 11:

[0750] The server analyzes the received test results, carefully examining the results to identify system defects and areas for improvement. The analysis also includes emotional data, providing feedback based on the user's emotions.

[0751] Step 12:

[0752] The server creates a report based on the analysis results and provides it to the developer, including specific suggestions for improvement, as feedback to improve the quality of the system.

[0753] For example, if a user purchases a new smartwatch and fails to connect to Wi-Fi when starting setup, the server will predict the stress response to a Wi-Fi connection error based on past user data and emotional data, and generate this scenario. If a test is run on the device and the error message is inappropriate, the server will report the results to the developer and provide feedback to improve the message content.

[0754] ---

[0755] The above is a detailed flow of the program processing for a system that combines an emotion engine. This method enables testing that takes into account user behavior patterns and emotions, improving the quality of the system.

[0756] Example 2

[0757] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0758] Conventional system testing has the problem that it is difficult to consider the user's emotional state, making it difficult to accurately identify test obstacles and areas for improvement in the user experience. It is also difficult to cover a wide range of use cases, limiting the improvement of system quality.

[0759] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for collecting various information from a database, a preprocessing means for cleaning and normalizing the collected data, a means for training a machine learning model using the preprocessed data, and a means for generating use cases that take emotion data into consideration based on the trained model. This enables comprehensive testing of various use cases that take emotion data into consideration, thereby improving the quality of the system.

[0760] A "database" is a system for storing, managing, and retrieving information in an organized manner.

[0761] An "information gathering means" is a method or device for obtaining the required data from a database.

[0762] A "preprocessing means" is a method or device for cleaning and normalizing collected data.

[0763] A "means for training a machine learning model" is a method or apparatus for training a model based on an algorithm using pre-processed data.

[0764] A "use case generator" is a method or device for generating market scenarios based on a trained model.

[0765] A "system test execution means" is a method or device for verifying the functionality of the system based on the generated use cases.

[0766] "Test result analysis means" refers to a method or device for analyzing test results and identifying defects and areas for improvement.

[0767] "Emotion data" is information about the user's emotional state based on text input, facial expression data, tone of voice, and the like.

[0768] "Natural language processing technology" is a technology that enables computers to understand, analyze, and generate human language.

[0769] System Program Overview

[0770] This system is an AI system that collects and analyzes various information, including user sentiment data, generates use cases using machine learning models, and tests the system. The specific software used includes the Pandas library for data processing, the Scikit-learn library for machine learning, and the GPT-3 NLP model.

[0771] Data collection

[0772] The server collects the following information from the database:

[0773] Usage history

[0774] Inquiry details

[0775] Error Log

[0776] User emotional data (text input, facial expression data, tone of voice, etc.)

[0777] For example, the server uses an SQL query to select all records in the "Usage History" table from the database. Emotion data is obtained by acquiring the user's facial expression and voice data through the sensor API.

[0778] Data Preprocessing

[0779] The server cleans and normalizes the collected data, removing incomplete and duplicate data and imputing outliers and missing values. It uses the Pandas library to impute NaNs (missing values) in the data frame with the mean, detects outliers using statistical methods, and replaces values ​​beyond ±3 standard deviations with the median.

[0780] Training a machine learning model

[0781] The server uses the preprocessed data to train the machine learning model, which includes the following steps:

[0782] Creating training and test datasets

[0783] Trying different learning algorithms

[0784] Selecting the best model

[0785] The server uses the Scikit-learn library to split the dataset into a training set and a test set in a ratio of 8:2, and runs random forests and support vector machines. The resulting F1 scores are compared to select the best model.

[0786] Use Case Generation

[0787] The server generates use cases using a trained AI model, taking into account emotional data and creating specific scenarios. For example, the server uses an NLP model (e.g., GPT-3) to generate a scenario such as "emotional reactions when a user encounters a connection error when launching a new smartphone app for the first time."

[0788] An example prompt is "Generate a scenario in which you purchase a new smartwatch and the Wi-Fi connection fails when you begin setup."

[0789] Running the tests

[0790] The device receives use cases sent from the server and executes system tests, taking into account emotional data and verifying that each function works as expected. It emulates a smartphone app, generates errors in Wi-Fi connection scenarios, and monitors the user's stress level in real time. It also measures the display time of error messages and the response speed of the user interface.

[0791] Analyzing test results

[0792] The device collects test results and sends them to the server. The server analyzes the results and identifies defects and areas for improvement. Specifically, the program sends runtime log files and stress level measurement data in JSON format to the server. The server analyzes the received data and generates a heat map of defects using a Python script and the Matplotlib library. The report can be downloaded in PDF format.

[0793] In this way, AI systems based on emotion data can comprehensively test a variety of use cases and improve the quality of the tests.

[0794] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0795] Specific explanation of processing steps

[0796] Step 1: Collect data

[0797] The server collects various information from the database. The input of this step is the database, and the output is the collected information (usage history, inquiry content, error log, emotion data).

[0798] Specific behavior:

[0799] The server uses an SQL query to select all records in the "Usage History" table from the database. Emotion data is also obtained from the user's facial expressions and voice data via the sensor API.

[0800] Step 2: Clean and normalize the data

[0801] The server cleans and normalizes the collected data. The input of this step is the collected data and the output is the cleaned and normalized data.

[0802] Specific behavior:

[0803] The server uses the Pandas library to impute missing values ​​in the data frame with the mean and remove duplicates, and also uses statistical methods to detect outliers and replace values ​​beyond ±3 standard deviations with the median.

[0804] Step 3: Train the machine learning model

[0805] The server uses the preprocessed data to train a machine learning model. The input of this step is the preprocessed data, and the output is the trained model.

[0806] Specific behavior:

[0807] The server uses the Scikit-learn library to split the dataset into a training set and a test set in a ratio of 8:2. It then tries out models such as random forest and support vector machine, and selects the best model based on the F1 score.

[0808] Step 4: Generate Use Cases

[0809] The server generates use cases using the trained AI model. The inputs of this step are the trained model and emotion data, and the output is a specific use case scenario.

[0810] Specific behavior:

[0811] The server generates a scenario by inputting a prompt into an NLP model (e.g., GPT-3) such as "the emotional reaction when a user encounters a connection error when launching a new smartphone app for the first time."

[0812] Step 5: Receive use cases and test execution

[0813] The terminal receives the use case sent from the server and executes the system test. The input of this step is the use case scenario, and the output is the test result.

[0814] Specific behavior:

[0815] The device emulates smartphone apps and simulates Wi-Fi connection errors, while monitoring users' stress levels, reaction times, and other metrics in real time during the test.

[0816] Step 6: Submit and analyze test results

[0817] The terminal collects the test results and sends them to the server, which analyzes the received test results and identifies system defects and areas for improvement. The input of this step is the test results, and the output is an analyzed report.

[0818] Specific behavior:

[0819] The device sends runtime log files and stress level measurements in JSON format to a server, which uses a Python script and the Matplotlib library to generate a heat map of defects and compiles it into a PDF report.

[0820] Example of input prompt for generative AI model

[0821] Below are some example prompts to input to a generative AI model:

[0822] "Generate a scenario where you purchase a new smartwatch and fail to connect to Wi-Fi when starting setup."

[0823] The above are the specific processing steps of the system process, and by covering a variety of use cases that combine emotion data, the quality of the system can be improved.

[0824] (Application example 2)

[0825] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0826] In conventional systems, data collection and testing are primarily limited to technical aspects, and evaluation of customer emotional data and usage experience is insufficient. This limits the improvement of user satisfaction, and there is a problem that customer experience in physical stores is not sufficiently improved. In addition, it is difficult to perform simulations and propose improvement proposals using emotional data in real time.

[0827] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0828] In this invention, the server includes means for collecting various information from a data storage device, means for preprocessing the collected data, means for training a computational model using the preprocessed data, means for generating a simulation based on the trained model, means for testing the device based on the generated simulation, means for analyzing the test results and identifying device defects and areas for improvement, means for collecting and analyzing customer emotion data in real time, and means for presenting improvement proposals based on the analysis results. This enables an improved customer experience in physical stores. By acquiring and analyzing customer emotion data in real time, appropriate responses and improvements can be made immediately, thereby improving user satisfaction.

[0829] A "data storage device" is a device for storing various information.

[0830] "Preprocessing" refers to the process of cleaning and normalizing the collected data.

[0831] A "computational model" is a model for data analysis that is trained using machine learning algorithms.

[0832] "Simulation" refers to the virtual reproduction of use cases and situations that may occur in the market based on a trained model.

[0833] "Device testing" is the process of verifying whether a device or system actually operates as expected based on the generated simulation.

[0834] "Test results" refer to the measurement data and operation history obtained when a device test is executed.

[0835] "Analysis" is the process of identifying equipment defects and areas for improvement based on test results.

[0836] "Emotion data" refers to data relating to emotions acquired from the customer's facial expressions, voice, etc.

[0837] "Real-time" refers to the immediate processing and reflection of ongoing events and situations.

[0838] "Customer experience" refers to the experiences a customer has with a product or service and everything related to it.

[0839] "Improvement proposals" are proposals for improving systems and services that are based on the analysis results.

[0840] This invention relates to a system for improving customer experience in physical stores by combining and analyzing various information collected from data storage devices and customer emotion data collected in real time.

[0841] The server collects usage history, inquiry details, error logs, and customer sentiment data from the data storage device. This sentiment data is obtained from user text input, facial expressions, tone of voice, etc. The server then cleans and normalizes the collected data, filling in outliers and missing values. This stage ensures data integrity. The sentiment data is also preprocessed in the same way.

[0842] After preprocessing is complete, the server uses the preprocessed data to train a machine learning model. This training involves trying different learning algorithms and selecting the best model. Using emotion data for training improves the accuracy of predicting user behavior patterns. After training is complete, the server uses the trained machine learning model to generate simulations of possible market events. These are generated as specific scenarios, taking into account user behavior patterns and emotions at the time.

[0843] The terminal then receives the simulation generated by the server and runs a system test based on it. The terminal is equipped with a camera and a voice recognition device, which allows it to collect and analyze customer emotion data in real time. For example, a camera installed at the entrance can analyze the facial expressions of customers entering the store to determine whether they are satisfied or stressed.

[0844] Once the test is complete, the device collects the test results and sends them to the server. The server then analyzes the results and identifies any equipment defects or areas for improvement. It also generates a report based on the analysis results, presenting specific improvement proposals. This allows for rapid improvements to be made to services and layouts in physical stores.

[0845] As a concrete example, consider a system in which a camera is installed at the entrance of a store and the facial expressions of customers as they enter the store are analyzed in real time. The camera and a voice recognition device work together to acquire text data and facial expression data, which are then collected as emotion data. Based on this data, store staff can correct their customer service responses in real time, and the system can automatically suggest improvements to the store layout.

[0846] Example prompt for a generative AI model:

[0847] "Generate optimization proposals for store layout and customer service methods based on today's customer facial expression data."

[0848] This prompt is expected to enable the generative AI model to provide specific improvement suggestions based on emotional data.

[0849] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0850] Step 1:

[0851] The server collects usage history, inquiry details, error logs, and customer sentiment data from the data storage device. Specifically, the server obtains these data through API and stores them in a database. The input data is raw data, and the output is pre-processed, consistent data.

[0852] Step 2:

[0853] The server preprocesses the collected data. Preprocessing includes cleaning and normalizing the data, removing incomplete and duplicate data, and imputing outliers and missing values. Specifically, these operations are performed using the Python pandas library. The input data is raw data, and the output data is cleaned data.

[0854] Step 3:

[0855] The server trains a computational model using the preprocessed data. It tries different machine learning algorithms and selects the best model. It trains the model with particular emphasis on emotion data. Specifically, it uses libraries such as Scikit-learn and TensorFlow. The input data is the preprocessed data, and the output data is the trained model.

[0856] Step 4:

[0857] The server generates a simulation based on the trained model. This involves creating specific scenarios that take into account the user's behavioral patterns and emotions at the time. For example, it includes scenarios such as "customer satisfaction when trying a new product." The input data are the trained model and user data, and the output data is the simulation scenario.

[0858] Step 5:

[0859] The terminal receives the simulation generated by the server and executes the system test based on it. Specifically, it collects and analyzes customer emotion data in real time using a camera and a voice recognition device. The input data is the simulation scenario, and the output data is the test results.

[0860] Step 6:

[0861] The terminal collects the test results and sends them to the server. The server analyzes the received test results and identifies any equipment defects or areas for improvement. Specifically, a Python analysis tool is used for the analysis. The input data is the test results, and the output data is an analysis report.

[0862] Step 7:

[0863] The server creates a report based on the analysis results, proposing specific improvement proposals, and provides it to developers. For example, it proposes improvements to the store layout or changes to customer service methods. The input data is the analysis results, and the output data is a report of the improvement proposals.

[0864] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0865] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0866] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[0867] [Fourth embodiment]

[0868] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[0869] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0870] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0871] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[0872] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0873] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0874] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0875] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[0876] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0877] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0878] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0879] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0880] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[0881] ---

[0882] The present invention relates to an AI system for testing a wide range of use cases. The system collects necessary information from a database, preprocesses it, trains a machine learning model, generates use cases based on the model, and finally executes tests based on these use cases.

[0883] The system configuration is as follows: First, the server collects various information from the database, such as usage history, inquiry details, and error logs. This information is also obtained from the customer support organization and other related systems.

[0884] The server then preprocesses the collected data, which involves cleaning the data, removing incomplete and duplicate data, and normalizing and standardizing the data to create a dataset suitable for analysis.

[0885] Once preprocessing is complete, the server begins training the machine learning model using the preprocessed data. Natural language processing techniques are used for training, and the model learns user behavior patterns. Multiple models are tested, and the best-performing model is selected.

[0886] After training is complete, the server uses the trained AI model to estimate and generate potential use cases in the market. These use cases are expressed as specific scenarios, such as "what happens when a user launches a new smartphone app for the first time" or "what error message should be displayed if a Wi-Fi connection fails."

[0887] The device then receives the use cases generated by the server and runs tests based on them to verify that each system function works as expected, such as displaying a message when a Wi-Fi connection error occurs and the reconnection option.

[0888] Once the test is complete, the device collects the test results and sends them to the server. The server analyzes the received test results and identifies any system defects or areas for improvement. The analysis results are compiled into a report and provided to the developer. This allows specific improvement proposals to be presented, improving the quality of the system.

[0889] As a concrete example, consider the following scenario: A user purchases a new smartwatch and fails to connect to Wi-Fi when starting setup. In this case, the server generates this scenario based on past user data and runs a test on the device. If the test results show that the error message is inappropriate, the server reports the results to the developer and provides feedback to improve the message content.

[0890] In this way, the present invention enables an AI system using diverse data to comprehensively test a variety of use cases and improve the quality of the tests.

[0891] The processing flow will be explained below.

[0892] ---

[0893] Step 1:

[0894] The server collects information such as usage history, inquiry details, and error logs from the database, including data from the customer support organization and related systems, and is configured to always retrieve the latest data.

[0895] Step 2:

[0896] The server cleans the collected data, removing incomplete and duplicate data and filling in outliers and missing values. At this stage, data integrity is ensured.

[0897] Step 3:

[0898] The server normalizes the cleaned data and standardizes data of different scales, thereby preparing the data so that the machine learning model can learn efficiently.

[0899] Step 4:

[0900] The server uses the preprocessed data to train a machine learning model, generating training and test datasets and testing different learning algorithms to select the best model.

[0901] Step 5:

[0902] The server uses the selected machine learning model to estimate possible use cases in the market, predicting user behavior patterns and generating use cases as specific scenarios.

[0903] Step 6:

[0904] The server stores the generated use cases in a database and converts them into a form that can be executed on the system under test.

[0905] Step 7:

[0906] The terminal receives the use case data provided by the server, sets up the test environment, and prepares the test according to the pre-defined distribution rules.

[0907] Step 8:

[0908] The terminal executes system tests based on the received use cases, verifying that the system functions correctly for each use case.

[0909] Step 9:

[0910] The terminal records detailed error logs, warning messages, and operation results that occur during the test, and collects test results in real time.

[0911] Step 10:

[0912] The terminals send the collected test results to a server, which allows the test results to be centrally managed.

[0913] Step 11:

[0914] The server analyzes the received test results, carefully examining the results and identifying system defects and areas for improvement.

[0915] Step 12:

[0916] The server creates a report based on the analysis results and provides it to the developers, including specific suggestions for improvement and providing feedback to help improve the quality of the system.

[0917] ---

[0918] This is the detailed flow of the system's program processing, which enables comprehensive testing of a wide range of use cases and provides a high-quality system without any defects.

[0919] Example 1

[0920] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[0921] Conventional systems face the problem of being unable to comprehensively test a wide range of use cases, making it difficult to properly identify system defects and areas that need optimization. In particular, it is difficult to process and analyze a wide range of data and predict user behavior patterns, which can lead to inappropriate feedback being provided.

[0922] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0923] In this invention, the server includes means for collecting various information from a database, means for preprocessing the collected data, means for training a machine learning model using the preprocessed data, means for generating use cases that may occur in the market based on the trained model, means for executing system tests based on the generated use cases, means for analyzing the test results and identifying system defects and areas for improvement, and means for compiling the analysis results as a report and feeding them back to the developer. This makes it possible to comprehensively test various use cases, and appropriately identify and improve system defects and areas for optimization.

[0924] 1. A "database" is a collection of data that stores information systematically and can be searched and retrieved as needed.

[0925] 2. "Means of collecting information" refers to the function of obtaining data such as usage history, inquiry details, and error logs from databases and related systems.

[0926] 3. "Preprocessing means" refers to the function of cleaning, normalizing, and standardizing collected data, converting it into a state suitable for analysis.

[0927] 4. "Means for training machine learning models" refers to the function of training models based on algorithms using collected data to improve predictive accuracy.

[0928] 5. "Means for generating use cases" refers to the function of using a trained model to create specific scenarios that could occur in the market.

[0929] 6. "Means of executing tests" refers to the testing process that verifies each function of the system based on the generated use cases.

[0930] 7. "Means for analyzing test results" refers to the ability to analyze data obtained from tests and evaluate system performance and defects.

[0931] 8. "Means of providing feedback" refers to the function of compiling the analysis results into a report and presenting improvements to the system to developers and other stakeholders.

[0932] 9. "Test automation tool" means software for automatically testing each function of a system.

[0933] 10. A "dashboard" is an interface that displays analytical data and important indicators in a visually easy-to-understand manner.

[0934] This invention relates to an AI system that comprehensively tests various use cases. The system collects information from a database, preprocesses it, trains a machine learning model, generates use cases based on the model, and finally performs testing to improve the quality of the system.

[0935] The system configuration is as follows: First, the server collects information from the database and other related systems. Specifically, it obtains data such as usage history, inquiry details, and error logs from SQL Server and MongoDB. It can also obtain data in real time from the customer support organization and other related systems via API.

[0936] The server then preprocesses the collected data using Python and the pandas library. First, incomplete and duplicate data is removed, and missing values ​​are imputed as appropriate. Next, the data is normalized and standardized to generate a dataset suitable for analysis. For example, text data is tokenized, and numerical data is converted to Z-scores.

[0937] Once preprocessing is complete, the server begins training the machine learning model using the preprocessed data. TensorFlow and PyTorch are used for training, and natural language processing techniques are used to learn user behavior patterns. Multiple models are tested, and the best-performing model is selected through cross-validation.

[0938] After training is complete, the server uses the trained AI model to generate potential use cases that could occur in the market. This is expressed as a specific scenario using the generative AI model. Examples include "what happens when a user launches a new smartphone app for the first time" and "what error message should be displayed if a Wi-Fi connection fails." Examples of prompts include:

[0939] "Generate error messages that may occur when a user launches a new smartphone app for the first time."

[0940] The terminal then receives the use cases generated from the server and executes system tests based on them. Test automation tools (e.g., Selenium, Appium) are installed on the terminal, and these are used to verify whether each function of the system works as expected. For example, based on the scenario of "What happens when a Wi-Fi connection error occurs," the operation of the reconnection option and the display of error messages are tested.

[0941] Once the test is complete, the device collects the test results and sends them to a server, where they are analyzed using Python or R. The analysis includes statistical evaluations such as the frequency of error messages and system response times.

[0942] Finally, the server compiles the analyzed test results into a report. This report details any system defects or areas for improvement and is provided to developers. For example, it includes specific suggestions for improvement, such as "the reconnection function in the event of a Wi-Fi connection error does not work properly." This feedback is used to optimize the system and improve quality.

[0943] As described above, the present invention can improve the quality of a system by comprehensively testing a variety of use cases and appropriately identifying system defects and areas for improvement.

[0944] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0945] Step 1:

[0946] The server collects information from the database. The input is data such as usage history, inquiry details, and error logs from the database and related systems. The output is the collected raw data, which is obtained using SQL queries or API requests. Specifically, the past six months of inquiry details are obtained from the customer support system via API.

[0947] Step 2:

[0948] The server preprocesses the collected data. The input is the raw data collected in step 1, and the output is cleaned and normalized data. Specifically, it uses Python and the pandas library to remove incomplete and duplicate data and impute missing values ​​as appropriate. It also normalizes and standardizes the data to generate a dataset suitable for analysis. For example, it tokenizes text data and converts numerical data into Z-scores.

[0949] Step 3:

[0950] The server uses the preprocessed data to train a machine learning model. The input is the preprocessed data, and the output is a trained machine learning model. TensorFlow or PyTorch is used for training, and natural language processing techniques are used to learn user behavior patterns. Specifically, multiple models (e.g., random forests, neural networks) are tried, and the best-performing model is selected through cross-validation.

[0951] Step 4:

[0952] The server uses the trained AI model to generate use cases that could occur in the market. The input is a trained machine learning model and a prompt based on it, and the output is a specific use case scenario. For example, it generates "what happens when a user launches a new smartphone app for the first time" or "what error message should be displayed if Wi-Fi connection fails."

[0953] Step 5:

[0954] The terminal receives use cases generated from the server and executes system tests based on them. The input is the generated use case scenario, and the output is the test result. Test automation tools (e.g., Selenium, Appium) are installed on the terminal, and these are used to check whether each function of the system works as expected. For example, based on the scenario of "What happens when a Wi-Fi connection error occurs," the operation of the reconnection option and the display of error messages are tested.

[0955] Step 6:

[0956] Once the test is complete, the terminal collects the test results and sends them to the server. The input is the test results, and the output is the analysis data. The server analyzes the test results using Python or R. Specifically, statistical evaluations such as the frequency of error messages and system response speed are included.

[0957] Step 7:

[0958] The server creates a report based on the analysis of the test results. The input is the analysis data, and the output is a report. This report details any system defects or areas for improvement and is provided to developers. For example, it includes specific improvement suggestions such as "The reconnection function in the event of a Wi-Fi connection error does not work properly." This feedback is used to optimize the system and improve quality.

[0959] (Application example 1)

[0960] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[0961] When operating factory robots, there is a need to efficiently perform comprehensive testing for a variety of possible use cases. Currently, manual scenario creation and test execution is the norm, which requires a significant amount of man-hours and has limitations on the quality and comprehensiveness of the tests. In addition, the process of analyzing test results and identifying system defects and areas for improvement is not efficient, which poses the issue of taking a long time to improve the quality of the entire system.

[0962] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0963] In this invention, the server includes means for collecting various information from a database, means for preprocessing the collected data, and means for training a machine learning model using the preprocessed data, which makes it possible to generate various use cases that arise in the operation of factory robots, run system tests based on the use cases, and analyze the test results to identify system defects and areas for improvement.

[0964] A "database" is a system that stores a variety of information in a structured manner and enables efficient query and data retrieval.

[0965] "Preprocessing" is the process of removing incomplete and duplicate data from collected data, and converting it into a format suitable for analysis by normalizing and standardizing it.

[0966] A "machine learning model" is an algorithm that learns from collected and preprocessed data and performs estimation and classification on unknown data.

[0967] A "use case" is a specific example that shows how a system works under specific scenarios or conditions.

[0968] A "factory robot" is a mechanical device used to automate specific tasks on a manufacturing line or in a work environment within a factory.

[0969] "Test results" are evaluation data on the success or failure of operations and performance obtained when a system test is executed.

[0970] A "failure" is a phenomenon in which a system does not function as expected or causes an error.

[0971] "Improvements" are modifications or additional functions that are necessary to improve the performance or functionality of the system.

[0972] The present invention relates to an AI system for conducting tests covering a wide range of use cases. This system can efficiently test a wide range of use cases in the operation of factory robots.

[0973] The system configuration is as follows: First, the server collects various information from the database, such as usage history, error logs, and operation history. This information may also be obtained from sensors and robot control systems within the factory.

[0974] The server then preprocesses the collected data. The preprocessing step involves cleaning the data, removing incomplete and duplicate data, and normalizing and standardizing it to create a dataset suitable for analysis. This is done using Python and the Pandas library.

[0975] Once the preprocessing is complete, the server begins training the machine learning model using the preprocessed data. Training uses natural language processing techniques to learn the behavioral patterns of the factory robots. Using deep learning frameworks such as TensorFlow, multiple models are tested and the best-performing model is selected.

[0976] After training is complete, the server uses the trained AI model to estimate and generate use cases that may occur in the operation of the factory robot. These use cases are expressed as specific scenarios. For example, they include "what to do when a factory robot misidentifies a part during production line work" and "what to do in an emergency when it collides with an obstacle."

[0977] The terminal then receives the use cases generated by the server and executes system tests based on them. The tests verify whether each function of the robot operates as expected. For example, they test the message display and re-operation option when a part recognition error occurs.

[0978] Once the test is complete, the device collects the test results and sends them to the server. The server analyzes the received test results and identifies any system defects or areas for improvement. The analysis results are compiled into a report and provided to the developer. This allows specific improvement proposals to be presented, improving the quality of the system.

[0979] As a concrete example, consider the following scenario: If a factory robot misidentifies a part during line work, resulting in a stoppage of work, the server generates this use case based on past robot operation data and runs a test on the terminal. If the test results show that the error message is inappropriate, the server reports the result to the developer and provides feedback to improve the message content.

[0980] In this way, the present invention enables an AI system using diverse data to comprehensively test a variety of use cases for factory robots, thereby improving the quality of the tests.

[0981] Example prompt sentence:

[0982] "Generate a scenario to test the behavior of a factory robot when it misidentifies a part during production line operation."

[0983] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0984] Step 1:

[0985] The server collects information such as usage history, error logs, and operational history from the database and imports it as data. The input is data obtained from sensors and robot control systems in the factory. The output is a raw data set.

[0986] Step 2:

[0987] The server preprocesses the collected data. Specifically, it cleans the data, removes incomplete or duplicate data, and normalizes and standardizes the data to convert it into a format suitable for analysis. The input is the raw dataset. The output is the preprocessed dataset.

[0988] Step 3:

[0989] The server uses the preprocessed data to train a machine learning model. It uses a deep learning framework such as TensorFlow to train the model on the data. The input is the preprocessed dataset. The output is a trained AI model.

[0990] Step 4:

[0991] The server uses a trained AI model to estimate and generate use cases that may occur in the operation of factory robots. Specifically, it runs an algorithm that generates new use cases based on past operational data. The inputs are the trained AI model and operational data. The output is a use case scenario.

[0992] Step 5:

[0993] The terminal receives the use cases generated from the server and executes system tests based on them. Specifically, the terminal controls the robot according to the operation instructions in the use cases and executes the tests. The input is the use case scenario. The output is the test result data.

[0994] Step 6:

[0995] The terminal collects the test results and sends them to the server. The input is the test result data. The output is the test result sent to the server.

[0996] Step 7:

[0997] The server analyzes the received test results and identifies system defects and areas for improvement. Specifically, it analyzes the test results and runs an algorithm to extract defects and areas for improvement. The input is the test results. The output is an analysis report.

[0998] By performing specific operations at each step, it is possible to comprehensively test a variety of use cases and improve the quality of the tests.

[0999] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1000] ---

[1001] This invention relates to an AI system for comprehensive testing of various use cases, and is characterized by its configuration that incorporates an emotion engine that recognizes user emotions. This system collects various information, including emotion data, preprocesses it, trains a machine learning model, generates use cases based on the model, and tests the system taking into account the emotion data.

[1002] The system configuration is as follows: First, the server collects information such as usage history, inquiry details, error logs, and user emotional data from the database. Emotional data is obtained from the user's text input, facial expressions detected by sensors, tone of voice, etc.

[1003] The server then cleans the collected data, removing incomplete and duplicate data and imputing outliers and missing values. At this stage, data integrity is ensured, and sentiment data is also preprocessed.

[1004] Once preprocessing is complete, the server uses the preprocessed data to train a machine learning model. It generates training and test datasets, tests different learning algorithms, and selects the best model. Sentiment data is also used for training, allowing for more accurate prediction of user behavior patterns.

[1005] After training is complete, the server uses the trained AI model to predict potential use cases that may arise in the market. It generates use cases as specific scenarios, taking into account not only the user's behavioral patterns but also their emotions at the time. Examples include "the stress response when a connection error occurs when a user launches a new smartphone app for the first time" and "the behavior of completing setup while showing positive emotions."

[1006] The device then receives the use cases generated by the server and runs tests on the system based on them. The tests verify that each function of the system works as expected. By taking into account emotional data, it is possible to identify user stress points and functions that generate high satisfaction.

[1007] Once the test is complete, the device collects the test results and sends them to the server. The server analyzes the received test results and identifies any system defects or areas for improvement. The analysis results are compiled into a report and provided to the developer. This allows specific improvement proposals to be presented, improving the quality of the system.

[1008] As a concrete example, consider the following scenario: A user purchases a new smartwatch and may experience Wi-Fi connection failure when starting setup. In this case, the server generates this scenario based on past user data and emotional data, and runs a test on the device. If the test results show that the error message is inappropriate, the server reports the results to the developer and provides feedback to improve the message content.

[1009] In this way, the present invention enables AI systems based on emotion data to comprehensively test a variety of use cases and improve the quality of the tests.

[1010] The processing flow will be explained below.

[1011] ---

[1012] Step 1:

[1013] The server collects usage history, inquiry details, error logs, and user emotional data from a database. Emotional data is collected from multiple sources, including user text input, facial expressions obtained by sensors, and tone of voice.

[1014] Step 2:

[1015] The server cleans the collected data, removing incomplete and duplicate data and imputing outliers and missing values. At this stage, sentiment data is also preprocessed to ensure consistency.

[1016] Step 3:

[1017] The server normalizes the cleaned data, standardizing data on different scales and converting it into a format suitable for analysis. Emotion data is also processed on a consistent scale.

[1018] Step 4:

[1019] The server uses the preprocessed data to train a machine learning model, generating training and test datasets and testing multiple learning algorithms to select the best model. It also trains the model on emotion data to improve its ability to predict user behavior and emotions.

[1020] Step 5:

[1021] The server uses the trained AI model to generate potential use cases in the market. It creates specific scenarios that take into account user behavior and emotions. For example, it could be a scenario where the Wi-Fi connection fails during smartwatch setup, causing frustration for the user.

[1022] Step 6:

[1023] The server stores the generated use cases in a database and converts them into a format that can be executed on the system under test. The use cases also contain emotional data to accurately evaluate the system's response.

[1024] Step 7:

[1025] The terminal receives the use case data provided by the server and sets up the test environment, which is configured to prepare the system for testing based on each use case.

[1026] Step 8:

[1027] The device tests the system based on the received use cases, verifying the system's behavior for each use case, and verifying whether the user's emotional responses are as expected.

[1028] Step 9:

[1029] The device records detailed error logs and warning messages that occur during the test, as well as operational results and user emotional data, and collects this information as test results.

[1030] Step 10:

[1031] The device sends the collected test results to a server, which includes information about the system's behavior and the user's emotions.

[1032] Step 11:

[1033] The server analyzes the received test results, carefully examining the results to identify system defects and areas for improvement. The analysis also includes emotional data, providing feedback based on the user's emotions.

[1034] Step 12:

[1035] The server creates a report based on the analysis results and provides it to the developer, including specific suggestions for improvement, as feedback to improve the quality of the system.

[1036] For example, if a user purchases a new smartwatch and fails to connect to Wi-Fi when starting setup, the server will predict the stress response to a Wi-Fi connection error based on past user data and emotional data, and generate this scenario. If a test is run on the device and the error message is inappropriate, the server will report the results to the developer and provide feedback to improve the message content.

[1037] ---

[1038] The above is a detailed flow of the program processing for a system that combines an emotion engine. This method enables testing that takes into account user behavior patterns and emotions, improving the quality of the system.

[1039] Example 2

[1040] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1041] Conventional system testing has the problem that it is difficult to consider the user's emotional state, making it difficult to accurately identify test obstacles and areas for improvement in the user experience. It is also difficult to cover a wide range of use cases, limiting the improvement of system quality.

[1042] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for collecting various information from a database, a preprocessing means for cleaning and normalizing the collected data, a means for training a machine learning model using the preprocessed data, and a means for generating use cases that take emotion data into consideration based on the trained model. This enables comprehensive testing of various use cases that take emotion data into consideration, thereby improving the quality of the system.

[1043] A "database" is a system for storing, managing, and retrieving information in an organized manner.

[1044] An "information gathering means" is a method or device for obtaining the required data from a database.

[1045] A "preprocessing means" is a method or device for cleaning and normalizing collected data.

[1046] A "means for training a machine learning model" is a method or apparatus for training a model based on an algorithm using pre-processed data.

[1047] A "use case generator" is a method or device for generating market scenarios based on a trained model.

[1048] A "system test execution means" is a method or device for verifying the functionality of the system based on the generated use cases.

[1049] "Test result analysis means" refers to a method or device for analyzing test results and identifying defects and areas for improvement.

[1050] "Emotion data" is information about the user's emotional state based on text input, facial expression data, tone of voice, and the like.

[1051] "Natural language processing technology" is a technology that enables computers to understand, analyze, and generate human language.

[1052] System Program Overview

[1053] This system is an AI system that collects and analyzes various information, including user sentiment data, generates use cases using machine learning models, and tests the system. The specific software used includes the Pandas library for data processing, the Scikit-learn library for machine learning, and the GPT-3 NLP model.

[1054] Data collection

[1055] The server collects the following information from the database:

[1056] Usage history

[1057] Inquiry details

[1058] Error Log

[1059] User emotional data (text input, facial expression data, tone of voice, etc.)

[1060] For example, the server uses an SQL query to select all records in the "Usage History" table from the database. Emotion data is obtained by acquiring the user's facial expression and voice data through the sensor API.

[1061] Data Preprocessing

[1062] The server cleans and normalizes the collected data, removing incomplete and duplicate data and imputing outliers and missing values. It uses the Pandas library to impute NaNs (missing values) in the data frame with the mean, detects outliers using statistical methods, and replaces values ​​beyond ±3 standard deviations with the median.

[1063] Training a machine learning model

[1064] The server uses the preprocessed data to train the machine learning model, which includes the following steps:

[1065] Creating training and test datasets

[1066] Trying different learning algorithms

[1067] Selecting the best model

[1068] The server uses the Scikit-learn library to split the dataset into a training set and a test set in a ratio of 8:2, and runs random forests and support vector machines. The resulting F1 scores are compared to select the best model.

[1069] Use Case Generation

[1070] The server generates use cases using a trained AI model, taking into account emotional data and creating specific scenarios. For example, the server uses an NLP model (e.g., GPT-3) to generate a scenario such as "emotional reactions when a user encounters a connection error when launching a new smartphone app for the first time."

[1071] An example prompt is "Generate a scenario in which you purchase a new smartwatch and the Wi-Fi connection fails when you begin setup."

[1072] Running the tests

[1073] The device receives use cases sent from the server and executes system tests, taking into account emotional data and verifying that each function works as expected. It emulates a smartphone app, generates errors in Wi-Fi connection scenarios, and monitors the user's stress level in real time. It also measures the display time of error messages and the response speed of the user interface.

[1074] Analyzing test results

[1075] The device collects test results and sends them to the server. The server analyzes the results and identifies defects and areas for improvement. Specifically, the program sends runtime log files and stress level measurement data in JSON format to the server. The server analyzes the received data and generates a heat map of defects using a Python script and the Matplotlib library. The report can be downloaded in PDF format.

[1076] In this way, AI systems based on emotion data can comprehensively test a variety of use cases and improve the quality of the tests.

[1077] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1078] Specific explanation of processing steps

[1079] Step 1: Collect data

[1080] The server collects various information from the database. The input of this step is the database, and the output is the collected information (usage history, inquiry content, error log, emotion data).

[1081] Specific behavior:

[1082] The server uses an SQL query to select all records in the "Usage History" table from the database. Emotion data is also obtained from the user's facial expressions and voice data via the sensor API.

[1083] Step 2: Clean and normalize the data

[1084] The server cleans and normalizes the collected data. The input of this step is the collected data and the output is the cleaned and normalized data.

[1085] Specific behavior:

[1086] The server uses the Pandas library to impute missing values ​​in the data frame with the mean and remove duplicates, and also uses statistical methods to detect outliers and replace values ​​beyond ±3 standard deviations with the median.

[1087] Step 3: Train the machine learning model

[1088] The server uses the preprocessed data to train a machine learning model. The input of this step is the preprocessed data, and the output is the trained model.

[1089] Specific behavior:

[1090] The server uses the Scikit-learn library to split the dataset into a training set and a test set in a ratio of 8:2. It then tries out models such as random forest and support vector machine, and selects the best model based on the F1 score.

[1091] Step 4: Generate Use Cases

[1092] The server generates use cases using the trained AI model. The inputs of this step are the trained model and emotion data, and the output is a specific use case scenario.

[1093] Specific behavior:

[1094] The server generates a scenario by inputting a prompt into an NLP model (e.g., GPT-3) such as "the emotional reaction when a user encounters a connection error when launching a new smartphone app for the first time."

[1095] Step 5: Receive use cases and test execution

[1096] The terminal receives the use case sent from the server and executes the system test. The input of this step is the use case scenario, and the output is the test result.

[1097] Specific behavior:

[1098] The device emulates smartphone apps and simulates Wi-Fi connection errors, while monitoring users' stress levels, reaction times, and other metrics in real time during the test.

[1099] Step 6: Submit and analyze test results

[1100] The terminal collects the test results and sends them to the server, which analyzes the received test results and identifies system defects and areas for improvement. The input of this step is the test results, and the output is an analyzed report.

[1101] Specific behavior:

[1102] The device sends runtime log files and stress level measurements in JSON format to a server, which uses a Python script and the Matplotlib library to generate a heat map of defects and compiles it into a PDF report.

[1103] Example of input prompt for generative AI model

[1104] Below are some example prompts to input to a generative AI model:

[1105] "Generate a scenario where you purchase a new smartwatch and fail to connect to Wi-Fi when starting setup."

[1106] The above are the specific processing steps of the system process, and by covering a variety of use cases that combine emotion data, the quality of the system can be improved.

[1107] (Application example 2)

[1108] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1109] In conventional systems, data collection and testing are primarily limited to technical aspects, and evaluation of customer emotional data and usage experience is insufficient. This limits the improvement of user satisfaction, and there is a problem that customer experience in physical stores is not sufficiently improved. In addition, it is difficult to perform simulations and propose improvement proposals using emotional data in real time.

[1110] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1111] In this invention, the server includes means for collecting various information from a data storage device, means for preprocessing the collected data, means for training a computational model using the preprocessed data, means for generating a simulation based on the trained model, means for testing the device based on the generated simulation, means for analyzing the test results and identifying device defects and areas for improvement, means for collecting and analyzing customer emotion data in real time, and means for presenting improvement proposals based on the analysis results. This enables an improved customer experience in physical stores. By acquiring and analyzing customer emotion data in real time, appropriate responses and improvements can be made immediately, thereby improving user satisfaction.

[1112] A "data storage device" is a device for storing various information.

[1113] "Preprocessing" refers to the process of cleaning and normalizing the collected data.

[1114] A "computational model" is a model for data analysis that is trained using machine learning algorithms.

[1115] "Simulation" refers to the virtual reproduction of use cases and situations that may occur in the market based on a trained model.

[1116] "Device testing" is the process of verifying whether a device or system actually operates as expected based on the generated simulation.

[1117] "Test results" refer to the measurement data and operation history obtained when a device test is executed.

[1118] "Analysis" is the process of identifying equipment defects and areas for improvement based on test results.

[1119] "Emotion data" refers to data relating to emotions acquired from the customer's facial expressions, voice, etc.

[1120] "Real-time" refers to the immediate processing and reflection of ongoing events and situations.

[1121] "Customer experience" refers to the experiences a customer has with a product or service and everything related to it.

[1122] "Improvement proposals" are proposals for improving systems and services that are based on the analysis results.

[1123] This invention relates to a system for improving customer experience in physical stores by combining and analyzing various information collected from data storage devices and customer emotion data collected in real time.

[1124] The server collects usage history, inquiry details, error logs, and customer sentiment data from the data storage device. This sentiment data is obtained from user text input, facial expressions, tone of voice, etc. The server then cleans and normalizes the collected data, filling in outliers and missing values. This stage ensures data integrity. The sentiment data is also preprocessed in the same way.

[1125] After preprocessing is complete, the server uses the preprocessed data to train a machine learning model. This training involves trying different learning algorithms and selecting the best model. Using emotion data for training improves the accuracy of predicting user behavior patterns. After training is complete, the server uses the trained machine learning model to generate simulations of possible market events. These are generated as specific scenarios, taking into account user behavior patterns and emotions at the time.

[1126] The terminal then receives the simulation generated by the server and runs a system test based on it. The terminal is equipped with a camera and a voice recognition device, which allows it to collect and analyze customer emotion data in real time. For example, a camera installed at the entrance can analyze the facial expressions of customers entering the store to determine whether they are satisfied or stressed.

[1127] Once the test is complete, the device collects the test results and sends them to the server. The server then analyzes the results and identifies any equipment defects or areas for improvement. It also generates a report based on the analysis results, presenting specific improvement proposals. This allows for rapid improvements to be made to services and layouts in physical stores.

[1128] As a concrete example, consider a system in which a camera is installed at the entrance of a store and the facial expressions of customers as they enter the store are analyzed in real time. The camera and a voice recognition device work together to acquire text data and facial expression data, which are then collected as emotion data. Based on this data, store staff can correct their customer service responses in real time, and the system can automatically suggest improvements to the store layout.

[1129] Example prompt for a generative AI model:

[1130] "Generate optimization proposals for store layout and customer service methods based on today's customer facial expression data."

[1131] This prompt is expected to enable the generative AI model to provide specific improvement suggestions based on emotional data.

[1132] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1133] Step 1:

[1134] The server collects usage history, inquiry details, error logs, and customer sentiment data from the data storage device. Specifically, the server obtains these data through API and stores them in a database. The input data is raw data, and the output is pre-processed, consistent data.

[1135] Step 2:

[1136] The server preprocesses the collected data. Preprocessing includes cleaning and normalizing the data, removing incomplete and duplicate data, and imputing outliers and missing values. Specifically, these operations are performed using the Python pandas library. The input data is raw data, and the output data is cleaned data.

[1137] Step 3:

[1138] The server trains a computational model using the preprocessed data. It tries different machine learning algorithms and selects the best model. It trains the model with particular emphasis on emotion data. Specifically, it uses libraries such as Scikit-learn and TensorFlow. The input data is the preprocessed data, and the output data is the trained model.

[1139] Step 4:

[1140] The server generates a simulation based on the trained model. This involves creating specific scenarios that take into account the user's behavioral patterns and emotions at the time. For example, it includes scenarios such as "customer satisfaction when trying a new product." The input data are the trained model and user data, and the output data is the simulation scenario.

[1141] Step 5:

[1142] The terminal receives the simulation generated by the server and executes the system test based on it. Specifically, it collects and analyzes customer emotion data in real time using a camera and a voice recognition device. The input data is the simulation scenario, and the output data is the test results.

[1143] Step 6:

[1144] The terminal collects the test results and sends them to the server. The server analyzes the received test results and identifies any equipment defects or areas for improvement. Specifically, a Python analysis tool is used for the analysis. The input data is the test results, and the output data is an analysis report.

[1145] Step 7:

[1146] The server creates a report based on the analysis results, proposing specific improvement proposals, and provides it to developers. For example, it proposes improvements to the store layout or changes to customer service methods. The input data is the analysis results, and the output data is a report of the improvement proposals.

[1147] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1148] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1149] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1150] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1151] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1152] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1153] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1154] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1155] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1156] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1157] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1158] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1159] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1160] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1161] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1162] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1163] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1164] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1165] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1166] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1167] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1168] The following is further disclosed regarding the above embodiment.

[1169] (Claim 1)

[1170] A means of collecting diverse information from databases;

[1171] a means for pre-processing the collected data;

[1172] means for training a machine learning model using the preprocessed data;

[1173] A means for generating potential use cases in the market based on the trained model; and

[1174] a means for executing tests of the system based on the generated use cases;

[1175] A means of analyzing test results and identifying system defects and areas for improvement;

[1176] A system including:

[1177] (Claim 2)

[1178] 10. The system of claim 1, further comprising pre-processing means for cleaning and normalizing the collected data.

[1179] (Claim 3)

[1180] 10. The system of claim 1, further comprising a training means that uses natural language processing techniques to infer user behavior patterns.

[1181] "Example 1"

[1182] (Claim 1)

[1183] A means of collecting diverse information from databases;

[1184] a means for pre-processing the collected data;

[1185] means for training a machine learning model using the preprocessed data;

[1186] A means for generating potential use cases in the market based on the trained model; and

[1187] a means for executing tests of the system based on the generated use cases;

[1188] A means of analyzing test results and identifying system defects and areas for improvement;

[1189] A means of compiling the analysis results into a report and providing feedback to developers,

[1190] A system including:

[1191] (Claim 2)

[1192] 10. The system of claim 1, further comprising pre-processing means for cleaning and normalizing the collected data.

[1193] (Claim 3)

[1194] 10. The system of claim 1, further comprising a training means that uses natural language processing techniques to infer user behavior patterns.

[1195] (Claim 4)

[1196] 10. The system of claim 1, further comprising means for using a test automation tool to execute the generated use cases.

[1197] (Claim 5)

[1198] 10. The system of claim 1, further comprising means for displaying the analyzed test results in a dashboard format.

[1199] "Application Example 1"

[1200] (Claim 1)

[1201] A means of collecting diverse information from databases;

[1202] a means for pre-processing the collected data;

[1203] means for training a machine learning model using the preprocessed data;

[1204] A means for generating potential use cases in the market based on the trained model; and

[1205] a means for testing the system based on use cases that arise in the operation of the factory robot;

[1206] A means of analyzing test results and identifying system defects and areas for improvement;

[1207] A system including:

[1208] (Claim 2)

[1209] 10. The system of claim 1, further comprising pre-processing means for cleaning and normalizing the collected data.

[1210] (Claim 3)

[1211] 10. The system of claim 1, further comprising a training means that uses natural language processing techniques to infer user behavior patterns.

[1212] "Example 2: Combining Emotion Engines"

[1213] (Claim 1)

[1214] A means of collecting diverse information from databases;

[1215] a pre-processing means for cleaning and normalizing the collected data;

[1216] means for training a machine learning model using the preprocessed data;

[1217] A means for generating use cases that take emotion data into account based on the trained model;

[1218] a means for executing tests of the system based on the generated use cases;

[1219] A means of analyzing test results and identifying system defects and areas for improvement;

[1220] A system including:

[1221] (Claim 2)

[1222] 10. The system of claim 1, further comprising means for collecting emotional data such as user text input, facial expression data, and tone of voice during data collection.

[1223] (Claim 3)

[1224] 10. The system of claim 1, further comprising a training means for inferring a user's behavioral patterns and emotional responses using natural language processing techniques.

[1225] "Application example 2 when combining emotion engines"

[1226] (Claim 1)

[1227] means for collecting various information from a data storage device;

[1228] a means for pre-processing the collected data;

[1229] means for training a computational model using the preprocessed data;

[1230] means for generating a simulation based on the trained model;

[1231] means for performing testing of the device based on the generated simulation;

[1232] A means of analyzing test results and identifying equipment defects and areas for improvement;

[1233] A means of collecting and analyzing customer sentiment data in real time,

[1234] A means of proposing improvement proposals based on the analysis results,

[1235] A system including:

[1236] (Claim 2)

[1237] 10. The system of claim 1, further comprising pre-processing means for cleaning and normalizing the collected data.

[1238] (Claim 3)

[1239] 10. The system of claim 1, further comprising a training means that uses natural language processing techniques to predict user behavior patterns. [Explanation of symbols]

[1240] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of collecting diverse information from databases; a means for pre-processing the collected data; means for training a machine learning model using the preprocessed data; A means for generating potential use cases in the market based on the trained model; and a means for executing tests of the system based on the generated use cases; A means of analyzing test results and identifying system defects and areas for improvement; A system including:

2. 10. The system of claim 1, further comprising pre-processing means for cleaning and normalizing the collected data.

3. 10. The system of claim 1, further comprising a training means for using natural language processing techniques to infer user behavior patterns.

Images