System

Abstract

A system is provided.SOLUTION: The system includes a means for automating terminal setting, a means for automatically executing a plurality of times of tests, a means for periodically monitoring a long-time test, a means for analyzing log data and detecting abnormality, and a means for reporting the analysis result.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] The verification process in device development typically requires a wide range of tasks, including terminal configuration, repeated testing, long-term testing, and log analysis, which can result in excessive man-hours. This not only requires a large number of personnel, but also wastes a lot of time. Furthermore, manual verification processes are prone to human error, making it difficult to detect defects early. These issues cause delays in development schedules. Therefore, reducing man-hours in the device verification process, improving efficiency, early defect detection, and shortening development schedules are key challenges. [Means for solving the problem]

[0005] The present invention solves the above-mentioned problems by providing a system that includes a means for automating device configuration, a means for automatically executing multiple tests, a means for periodically monitoring long-term tests, a means for analyzing log data and detecting anomalies, and a means for reporting the analysis results. With this system, when a user registers a new device, the server loads the device profile and automatically performs the necessary initial configuration. The server then automatically executes multiple tests based on the test conditions, collects the resulting data, and generates a report. Furthermore, during long-term tests, the server continuously monitors the device during the test period and notifies the user whenever an anomaly occurs. In particular, for log analysis, the server periodically collects log data from the device, analyzes it using AI, and quickly detects and reports anomalies. This results in more efficient device verification processes, early detection of defects, and shortened development schedules.

[0006] The "means for automating terminal settings" is a function that allows the server to automatically perform initial settings on a terminal when a user registers a new terminal.

[0007] The "means for automatically executing multiple tests" is a function that allows the server to automatically execute multiple tests on the terminal based on test conditions specified by the user.

[0008] The "means for periodically monitoring a long-term test" is a function in which the server periodically monitors the operating status of the terminal during a long-term test and notifies the user if an abnormality occurs.

[0009] "Means for analyzing log data and detecting anomalies" refers to a function that uses AI technology to analyze log data collected by the server from terminals and detects anomalies early from the results.

[0010] "Means for reporting the analysis results" refers to the function by which the server generates a detailed report based on the results of the log analysis and provides it to the user. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0012] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0013] First, the terms used in the following description will be explained.

[0014] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0015] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0016] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0017] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0018] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0019] [First embodiment]

[0020] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0021] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0022] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0023] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0024] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0025] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0026] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0027] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0028] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0029] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0030] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0031] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0032] The present invention relates to a system for automating processes in the verification process of device development, such as terminal setting, number testing, long-term testing, log analysis, etc. Specific embodiments for carrying out the present invention will be described in detail below.

[0033] Automating device configuration

[0034] The user registers a new device in the system. When the user enters the device information and presses the registration button, the server receives this information. The server reads the device profile from the database and automatically applies the initial settings, such as network settings and security settings, to the device. Once the settings are complete, the server sends a notification to the user that the settings are complete.

[0035] Specific examples

[0036] When a user registers a new smartphone in the system, the server reads the smartphone profile and automatically configures Wi-Fi settings, security policies, and installs necessary applications. After the configuration is complete, the server notifies the user that the smartphone configuration is complete.

[0037] Automation of frequency testing

[0038] The user accesses the test execution screen and enters the test conditions (e.g., run the Wi-Fi connection test 100 times in a row). When the user presses the test start button, the server receives the test conditions and causes the device to run the test the specified number of times. The server collects data obtained during the test (number of successes, number of failures, connection time, etc.) and generates a report based on this data. The generated report is sent from the server to the user.

[0039] Specific examples

[0040] If a user configures the system to perform 100 Wi-Fi connection tests, the server will perform all 100 connection attempts, compile the results, create a report, and send it to the user.

[0041] Automating long-term testing

[0042] Once the user issues a command to start the test and sets the test period (e.g., 24 hours), the server will continue to monitor the device during that period. The server periodically checks the device's operating status (CPU usage, memory usage, network status, etc.) and immediately notifies the user if any abnormalities are detected.

[0043] Specific examples

[0044] When a user sets up a 24-hour continuous use test, the server periodically checks the device's operating status for 24 hours and immediately notifies the user if any abnormalities are detected.

[0045] Automated log analysis

[0046] The server periodically collects log data from the device, inputs this log data into an AI analysis engine, and detects abnormal patterns and errors in the log data. The server then generates a detailed report based on the analysis results and provides it to the user.

[0047] Specific examples

[0048] The server collects operation logs from the devices at a set time every day and runs them through an AI analysis engine to detect abnormal patterns and errors. Based on the results, a weekly report is generated and sent to the user by email.

[0049] These features not only enable users to significantly reduce the amount of work required in the device verification process, but also enable highly accurate and rapid defect detection and efficient response. Furthermore, it is expected to contribute to shortening development schedules. This system will be an extremely useful tool for device developers.

[0050] The processing flow will be explained below.

[0051] Automating device configuration

[0052] Step 1:

[0053] The user enters the information for the new device and presses the "Register" button on the device registration screen.

[0054] Step 2:

[0055] The server receives the terminal information sent from the user.

[0056] Step 3:

[0057] The server reads the device profile from the database.

[0058] Step 4:

[0059] The server automatically applies the device's network and security settings based on the profile loaded.

[0060] Step 5:

[0061] The server confirms the status of the configuration.

[0062] Step 6:

[0063] The server will send a notification to the user that the setup is complete.

[0064] Automation of frequency testing

[0065] Step 1:

[0066] The user accesses the test execution screen and enters the test conditions (e.g., execute the Wi-Fi connection test 100 times in succession).

[0067] Step 2:

[0068] The user presses the "Start Test" button.

[0069] Step 3:

[0070] The server receives the input test conditions.

[0071] Step 4:

[0072] The server causes the terminal to execute the test a specified number of times.

[0073] Step 5:

[0074] The server collects data obtained during the test (number of successes, number of failures, connection time, etc.).

[0075] Step 6:

[0076] The server generates reports based on the collected data.

[0077] Step 7:

[0078] The server generates a report and sends it to the user.

[0079] Automating long-term testing

[0080] Step 1:

[0081] The user accesses the long-term test execution screen and inputs the test period (e.g., 24 hours).

[0082] Step 2:

[0083] The user presses the "Start Test" button.

[0084] Step 3:

[0085] The server sets a timer for the test period.

[0086] Step 4:

[0087] The server will periodically monitor the terminal status (e.g., CPU usage, memory usage, network status) during the test period.

[0088] Step 5:

[0089] As soon as the server detects an abnormality, it will notify the user.

[0090] Step 6:

[0091] After the long test is completed, the server reports the test results to the user.

[0092] Automated log analysis

[0093] Step 1:

[0094] The server periodically collects operation logs from the terminal based on a set schedule (e.g., collects logs at midnight every day).

[0095] Step 2:

[0096] The log data collected by the server is input into an AI analysis engine.

[0097] Step 3:

[0098] The server uses an AI analysis engine to detect abnormal patterns and errors in the log data.

[0099] Step 4:

[0100] The server generates a detailed report based on the results of the AI ​​analysis engine.

[0101] Step 5:

[0102] Sending server-generated reports to users (e.g. email or dashboard notifications).

[0103] Example 1

[0104] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0105] The verification process for conventional device development requires manual terminal configuration, test execution, long-term test monitoring, and log data analysis, which consumes a significant amount of time and effort. Additionally, there is a lack of systems for quickly detecting anomalies and taking appropriate action. Under these circumstances, it is difficult to ensure device quality while promoting efficient development.

[0106] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0107] In this invention, the server includes a means for automating terminal configuration, a means for automatically executing multiple tests, a means for periodically monitoring long-term tests, a means for analyzing log data and detecting anomalies, a means for reporting the analysis results, a means for monitoring the terminal's operating status, and a means for notifying the user of completion of configuration. This enables fast and accurate initial terminal configuration, and enables efficient and highly accurate device verification through the automation and monitoring of multiple tests and long-term tests. Furthermore, anomaly detection and user notification promotes rapid response and improves overall development efficiency.

[0108] "Means for automating terminal settings" refers to a function in which a user inputs terminal information into the system, and the server automatically performs network settings, security settings, and installation of necessary applications.

[0109] The "means for automatically executing multiple tests" is a function that allows the server to automatically execute a specified test multiple times based on conditions specified by the user, and collect the results.

[0110] "Means for periodically monitoring long-term testing" refers to a function in which the server periodically monitors the operating status of the terminal (CPU usage, memory usage, network status, etc.) during a period set by the user, and notifies the user if an abnormality occurs.

[0111] "Means for analyzing log data and detecting anomalies" refers to a function that inputs log data collected periodically by the server into an AI analysis engine to detect abnormal patterns and errors.

[0112] "Means for reporting the analysis results" refers to a function in which the server generates a detailed report based on the anomaly detection results obtained by the AI ​​analysis engine and notifies or provides it to the user.

[0113] "Means for monitoring the operating status of the terminal" refers to a function by which the server periodically checks the operating status of the terminal (CPU usage, memory usage, network status, etc.) during the test period or a specific monitoring period.

[0114] The "means for notifying the user of completion of setup" is a function in which the server sends a push notification or email notification to the user when the initial setup of the device is complete.

[0115] The present invention relates to a system for automating processes in the verification process of device development, such as terminal setting, number testing, long-term testing, log analysis, etc. Specific embodiments for carrying out the present invention will be described in detail below.

[0116] Automating device configuration

[0117] When a user registers a new device in the system, the server receives the information and retrieves the device profile from the database. The server then automatically configures the device's network and security settings and installs any necessary applications. Once the configuration is complete, the server sends a notification to the user. The hardware and software used are as follows:

[0118] Hardware: Servers, terminals

[0119] Software: Database, terminal configuration script

[0120] Specific examples

[0121] A user registers a new smartphone in the system. The server reads the smartphone profile and automatically configures Wi-Fi settings, security policies, and installs required applications. After the configuration is complete, the server notifies the user that the smartphone configuration is complete.

[0122] Example prompt sentence:

[0123] Register your new smartphone and automatically apply network settings and security policies.

[0124] Automation of frequency testing

[0125] When the user enters the test conditions (e.g., run a Wi-Fi connection test 100 times in a row) and presses the test start button, the server receives the test conditions and automatically runs the test the specified number of times. The server collects data obtained during the test (number of successes, number of failures, connection time, etc.) and generates a report based on this data. The generated report is sent from the server to the user. The hardware and software used are as follows:

[0126] Hardware: Servers, terminals

[0127] Software: Test management software, data collection module

[0128] Specific examples

[0129] The user sets up 100 Wi-Fi connection tests. The server executes all 100 connection attempts, compiles the results, creates a report, and sends it to the user.

[0130] Example prompt sentence:

[0131] Automatically run 100 Wi-Fi connection tests and compile the results into a report.

[0132] Automating long-term testing

[0133] Once the user sets the test period (e.g., 24 hours) and issues the command to start, the server will continue to monitor the terminal during that period. The server periodically checks the terminal's operating status (CPU usage, memory usage, network status, etc.) and immediately notifies the user if an abnormality occurs. The hardware and software used are as follows:

[0134] Hardware: Servers, terminals

[0135] Software: Monitoring software, alert systems

[0136] Specific examples

[0137] The user sets up a 24-hour continuous use test. The server periodically checks the device's operating status during the 24-hour period, and immediately notifies the user if an abnormality is detected.

[0138] Example prompt sentence:

[0139] Conduct a 24-hour continuous use test and notify us immediately if any abnormalities are detected.

[0140] Automated log analysis

[0141] The server periodically collects log data from the device and inputs it into an AI analysis engine for analysis. The analysis detects abnormal patterns and errors, and generates a detailed report based on the results. The generated report is provided to the user. The hardware and software used are as follows:

[0142] Hardware: Servers, terminals

[0143] Software: Log collection module, AI analysis engine

[0144] Specific examples

[0145] The server collects operation logs from the devices at a set time each day and inputs them into an AI analysis engine to detect abnormal patterns and errors. Based on the results, a weekly report is generated and sent to the user by email.

[0146] Example prompt sentence:

[0147] Collect device log data on a regular daily basis and analyze it with AI to detect abnormal patterns.

[0148] Through the above process, users can significantly reduce the effort required for device verification, and can detect defects quickly and accurately and address them efficiently. This is expected to improve the overall efficiency of device development and enhance quality assurance.

[0149] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0150] Processing flow of the terminal setting automation program

[0151] Step 1:

[0152] The user enters device information and presses the registration button. The user accesses the system interface and enters new device information (e.g., device ID, device name, device model, etc.). When the user presses the "Register" button, the information is sent to the server.

[0153] Input: Basic device information (device ID, device name, device model)

[0154] Output: Sending device information to the server

[0155] Step 2:

[0156] The server receives the device information and retrieves the device profile from the database. The server receives the HTTP request and prepares the received data for internal processing. It then issues an SQL query to the database to retrieve the profile information for the corresponding device.

[0157] Input: Device information sent by the user

[0158] Output: Device profile retrieved from the database

[0159] Step 3:

[0160] The server applies the settings to the device. Based on the device profile obtained by the server, it sends commands to the device to automatically configure network settings, security settings, and install applications.

[0161] Input: Device profile retrieved from the database

[0162] Output: Command to apply the settings to the device

[0163] Step 4:

[0164] The device receives and executes the settings. Based on the setting information received from the server, the device configures the network and security settings and installs the necessary applications.

[0165] Input: Configuration information sent from the server

[0166] Output: Performing configuration tasks

[0167] Step 5:

[0168] The server will send a notification to the user that the setup is complete. After the setup is complete, the server will send a push notification or email to the user to let them know that the setup is complete.

[0169] Input: Device with completed settings

[0170] Output: User notification

[0171] Processing flow of the automated program for number testing

[0172] Step 1:

[0173] The user enters the test conditions and presses the "Start Test" button. The user accesses the test management screen, enters the test conditions (e.g., 100 Wi-Fi connection tests), and presses the "Start" button.

[0174] Input: Test conditions (Wi-Fi connection test 100 times)

[0175] Output: Send test conditions to the server

[0176] Step 2:

[0177] The server receives the test conditions, prepares the test, analyzes the received test conditions, and sets the test schedule.

[0178] Input: Test conditions submitted by the user

[0179] Output: Test schedule settings

[0180] Step 3:

[0181] The server starts the test and sends multiple test execution commands to the terminal based on the specified test conditions.

[0182] Input: Exam Schedule

[0183] Output: Test execution command to terminal

[0184] Step 4:

[0185] The terminal executes the test. Upon receiving commands from the server, the terminal executes the test a specified number of times.

[0186] Input: Test execution command sent from the server

[0187] Output: Test results (number of successes, number of failures, connection time, etc.)

[0188] Step 5:

[0189] The server collects the test results and generates a report. The server collects the data obtained during the test run and generates a report based on that data.

[0190] Input: Test result

[0191] Output: Generate a report

[0192] Step 6:

[0193] The server sends the report to the user. The generated report is emailed to the user.

[0194] Input: Report

[0195] Output: Report sent to user

[0196] Processing flow of the automated program for long-term testing

[0197] Step 1:

[0198] The user sets the test period and issues a command to start the test. The user sets the test period (e.g., 24 hours) and presses the "Start" button.

[0199] Input: Test period (24 hours)

[0200] Output: Send test period to server

[0201] Step 2:

[0202] The server receives the instruction to start the test and begins monitoring. The server then sends a command to periodically monitor the operating status of the terminal during the test period.

[0203] Input: Test period instructions

[0204] Output: Monitor command to terminal

[0205] Step 3:

[0206] The device sends its operating status to the server. Following instructions from the server, the device periodically reports its operating status (CPU usage, memory usage, network status, etc.) to the server.

[0207] Input: Surveillance order

[0208] Output: Operation status data

[0209] Step 4:

[0210] The server analyzes the operational status data and detects abnormalities. The server periodically analyzes the operational status data it receives and detects abnormalities.

[0211] Input: Operational status data

[0212] Output: Anomaly detection results

[0213] Step 5:

[0214] Notify the user when the server detects an abnormality. If an abnormality occurs, the server will immediately send a notification to the user.

[0215] Input: Anomaly detection results

[0216] Output: Anomaly notification to the user

[0217] Log analysis automation program processing flow

[0218] Step 1:

[0219] The server periodically collects log data, and obtains the log data from the device via the logging API.

[0220] Input: Device log data

[0221] Output: Send log data to the server

[0222] Step 2:

[0223] The server inputs the log data into an AI analysis engine, which then inputs the log data collected by the server into the AI ​​analysis engine to detect abnormal patterns and errors.

[0224] Input: Log data

[0225] Output: AI analysis results

[0226] Step 3:

[0227] The server generates a report based on the analysis results. A detailed report is generated based on the anomaly detection results obtained by the AI ​​analysis engine.

[0228] Input: AI analysis results

[0229] Output: Generate a report

[0230] Step 4:

[0231] The server provides the report to the user, either by emailing the generated report to the user or making it available for viewing within the system.

[0232] Input: Report

[0233] Output: Report provided to user

[0234] The above processing steps enable the automation of terminal settings, the automatic execution of multiple tests, the monitoring of long-term tests, and the analysis of log data to be performed consistently and rationally.

[0235] (Application example 1)

[0236] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0237] Conventional methods for configuring, testing, and monitoring industrial equipment and terminals require a great deal of manual work, making them inefficient and prone to human error. Initially configuring new equipment, conducting a large number of tests, and monitoring over long periods of time require significant time and effort. Furthermore, detecting and reporting anomalies is also manual, making it difficult to respond quickly. Therefore, there is a need for efficient management of industrial equipment, improved reliability, and rapid detection and response of anomalies.

[0238] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0239] In this invention, the server includes means for automating terminal configuration, means for automatically executing multiple tests, means for periodically monitoring long-term tests, means for analyzing log data and detecting abnormalities, means for reporting the analysis results, means for reading industrial equipment profiles and automatically applying necessary initial settings, and means for collecting test results and monitoring them in real time. This automates the configuration, testing, monitoring, abnormality detection, and reporting of industrial equipment and terminals, enabling efficient and reliable management.

[0240] "Means for automating terminal settings" refers to a function that automatically performs the initial settings of terminals and industrial equipment without requiring manual operation by the user.

[0241] The "means for automatically executing multiple tests" means a function that automatically executes a certain number of tests on terminals or industrial equipment based on pre-set conditions.

[0242] "Means for periodically monitoring long-term tests" refers to a function that periodically monitors the operating status of terminals and industrial equipment when conducting long-term tests, and records and analyzes the results.

[0243] The "means for analyzing log data and detecting abnormalities" refers to a function that analyzes log data collected from terminals and industrial equipment and detects abnormal operations or errors from the data.

[0244] The "means for reporting the analysis results" has a function for reporting the analysis results of the log data to the user, and notifies the user if an abnormality is detected.

[0245] "Means for reading the profile of industrial equipment and automatically applying the required initial settings" means a function that reads the profile information of industrial equipment and automatically applies the required initial settings (Wi-Fi settings, security policies, etc.) based on that information.

[0246] "Means for collecting test results and monitoring them in real time" means a means for collecting result data obtained during the test and monitoring the data in real time.

[0247] A system for realizing the present invention includes a server, industrial equipment (terminals), and user interaction. Specific embodiments will be described in detail below.

[0248] Hardware and Software

[0249] Hardware:

[0250] Server: A computer system with a powerful processor

[0251] Industrial equipment: Equipment equipped with various sensors and communication modules (e.g., robots, sensor devices)

[0252] Smart glasses: Wearable devices for displaying real-time monitoring information (e.g., Google® Glass®)

[0253] software:

[0254] Server software: Python, real-time data streaming API, database management system

[0255] Log analysis engine: Anomaly detection algorithm using AI models

[0256] User interface: Web browser or dedicated application

[0257] Details of data processing and calculation

[0258] When a user registers an industrial device, the server reads its profile from the database and automatically applies initial settings (e.g., network settings, security policies) to the industrial device. This initial setting process is performed based on the device's profile information.

[0259] When industrial equipment starts a new task, the server runs multiple tests and collects and monitors the results in real time. When the test is completed, the server analyzes the obtained data and notifies the user if an abnormality is detected. This is done using an AI analysis engine for anomaly detection.

[0260] Furthermore, for equipment undergoing long-term testing, its operation is regularly monitored, and the server detects and notifies users of any abnormalities as necessary. All log data is centrally managed on the server and provided to users as reports as needed.

[0261] Specific examples

[0262] This section explains how this system works for robots used in factories. When a user registers a new robot in the system, the server loads the robot's profile and automatically applies initial settings such as Wi-Fi settings and security policies. It then conducts multiple operational tests and 24-hour continuous operation tests, monitoring the robot in real time. During this process, the server collects log data and detects abnormalities using an AI analysis engine. Detected abnormalities are immediately notified to the user, who can view the information through the smart glasses. In this way, users can efficiently manage the robot's operations and quickly respond to any abnormalities that occur.

[0263] Prompt Sentence Examples

[0264] "Please develop a system that monitors robots in factories and notifies them when an abnormality is detected. For example, it can automatically set up Wi-Fi and security policies, and conduct repeated and long-term tests to monitor the results in real time."

[0265] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0266] Step 1:

[0267] A user registers a new industrial device in the system. As input, the user provides device profile information (e.g., device ID, network settings, security policy, etc.). The server receives this information and stores it in a database. As output, the server sends a notification to the user that the registration is complete.

[0268] Step 2:

[0269] The server reads the saved profile information and automatically applies the necessary initial settings to the industrial equipment based on that information. As input, it receives the profile information and executes Wi-Fi settings, security policies, etc. based on that information. As output, it logs that the initial settings were successfully completed.

[0270] Step 3:

[0271] The user configures tests in the system. Specifically, the user specifies multiple tests (e.g., operation tests or connection tests) or long-term tests (e.g., 24-hour continuous operation tests). As input, the user provides test conditions (number of tests and duration). The server receives the test conditions and creates a test plan. As output, a test plan is generated and saved.

[0272] Step 4:

[0273] The server executes the specified tests. It reads the test plan as input and executes each test sequentially. Specifically, for multiple tests, it performs the specified number of attempts, and for long-term tests, it periodically monitors the operation within the specified time. As output, it records the results of each test (success / failure and operation log).

[0274] Step 5:

[0275] The server analyzes the log data collected during and after the test. It receives the collected operation logs and test results as input. It uses an AI model for analysis to detect anomalies. It outputs the analysis results in a report.

[0276] Step 6:

[0277] The server reports the analysis results to the user. As input, it generates a report of anomaly detection and notifies the user. As output, the user receives the report and can get detailed information about the status of industrial equipment.

[0278] Step 7:

[0279] If an anomaly is detected, the server immediately notifies the user. As input, it generates an anomaly notification based on the analysis results and forwards the content to the user. As output, the user receives the notification on a device such as smart glasses, allowing them to quickly address the problem.

[0280] Step 8:

[0281] Users review the maintenance and settings of industrial equipment based on the reports provided by the server. As input, they refer to the information in the report and carry out the necessary procedures. As output, they can confirm the improved settings and maintenance status. This cycle ensures efficient and reliable operation of industrial equipment.

[0282] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0283] This invention relates to a system that uses AI to automate processes such as terminal configuration, number testing, long-term testing, and log analysis in the verification process of device development, and further combines it with an emotion engine that recognizes user emotions. Specific embodiments for carrying out the present invention are described in detail below.

[0284] Automating device configuration

[0285] The user registers a new device in the system. When the user enters the device information and presses the registration button, the server receives this information. The server reads the device profile from the database and automatically applies the initial settings, such as network settings and security settings, to the device. Once the settings are complete, the server sends a notification to the user that the settings are complete.

[0286] Specific examples

[0287] When a user registers a new smartphone in the system, the server reads the smartphone profile and automatically configures Wi-Fi settings, security policies, and installs necessary applications. After the configuration is complete, the server notifies the user that the smartphone configuration is complete.

[0288] Automation of frequency testing

[0289] The user accesses the test execution screen and enters the test conditions (e.g., run the Wi-Fi connection test 100 times in a row). When the user presses the test start button, the server receives the test conditions and causes the device to run the test the specified number of times. The server collects data obtained during the test (number of successes, number of failures, connection time, etc.) and generates a report based on this data. The generated report is sent from the server to the user.

[0290] Specific examples

[0291] If a user configures the system to perform 100 Wi-Fi connection tests, the server will perform all 100 connection attempts, compile the results, create a report, and send it to the user.

[0292] Automating long-term testing

[0293] Once the user issues a command to start the test and sets the test period (e.g., 24 hours), the server will continue to monitor the device during that period. The server periodically checks the device's operating status (CPU usage, memory usage, network status, etc.) and immediately notifies the user if any abnormalities are detected.

[0294] Specific examples

[0295] When a user sets up a 24-hour continuous use test, the server periodically checks the device's operating status for 24 hours and immediately notifies the user if any abnormalities are detected.

[0296] Automated log analysis

[0297] The server periodically collects log data from the device, inputs this log data into an AI analysis engine, and detects abnormal patterns and errors in the log data. The server then generates a detailed report based on the analysis results and provides it to the user.

[0298] Specific examples

[0299] The server collects operation logs from the devices at a set time every day and runs them through an AI analysis engine to detect abnormal patterns and errors. Based on the results, a weekly report is generated and sent to the user by email.

[0300] User Emotion Recognition

[0301] The system also incorporates an emotion engine that recognizes the user's emotions. This emotion engine analyzes emotional data from the user's voice and facial expressions, and uses the results to adjust the system's operations and analyze log data.

[0302] Specific examples

[0303] The emotion engine recognizes emotions such as stress and anxiety felt by users while operating the system. Based on the results, the system makes suggestions to improve the user experience and analyzes log data by adding emotion data. For example, if a specific operation causes stress over a long period of time, the system will make suggestions to improve that operation method or the system interface.

[0304] Combining these elements will enable the device verification process to be more efficient, defects to be detected and addressed early, and user experience to be improved. This system will serve as a very useful tool for developers.

[0305] The processing flow will be explained below.

[0306] Automating device configuration

[0307] Step 1:

[0308] The user enters the information for the new device and presses the "Register" button on the device registration screen.

[0309] Step 2:

[0310] The server receives the terminal information sent from the user.

[0311] Step 3:

[0312] The server reads the device profile from the database.

[0313] Step 4:

[0314] The server automatically applies the device's network and security settings based on the profile loaded.

[0315] Step 5:

[0316] The server confirms the status of the configuration.

[0317] Step 6:

[0318] The server will send a notification to the user that the setup is complete.

[0319] Automation of frequency testing

[0320] Step 1:

[0321] The user accesses the test execution screen and enters the test conditions (e.g., execute the Wi-Fi connection test 100 times in succession).

[0322] Step 2:

[0323] The user presses the "Start Test" button.

[0324] Step 3:

[0325] The server receives the input test conditions.

[0326] Step 4:

[0327] The server causes the terminal to execute the test a specified number of times.

[0328] Step 5:

[0329] The server collects data obtained during the test (number of successes, number of failures, connection time, etc.).

[0330] Step 6:

[0331] The server generates reports based on the collected data.

[0332] Step 7:

[0333] The server generates a report and sends it to the user.

[0334] Automating long-term testing

[0335] Step 1:

[0336] The user accesses the long-term test execution screen and inputs the test period (e.g., 24 hours).

[0337] Step 2:

[0338] The user presses the "Start Test" button.

[0339] Step 3:

[0340] The server sets a timer for the test period.

[0341] Step 4:

[0342] The server will periodically monitor the terminal status (e.g., CPU usage, memory usage, network status) during the test period.

[0343] Step 5:

[0344] As soon as the server detects an abnormality, it will notify the user.

[0345] Step 6:

[0346] After the long test is completed, the server reports the test results to the user.

[0347] Automated log analysis

[0348] Step 1:

[0349] The server periodically collects operation logs from the terminal based on a set schedule (e.g., collects logs at midnight every day).

[0350] Step 2:

[0351] The log data collected by the server is input into an AI analysis engine.

[0352] Step 3:

[0353] The server uses an AI analysis engine to detect abnormal patterns and errors in the log data.

[0354] Step 4:

[0355] The server generates a detailed report based on the results of the AI ​​analysis engine.

[0356] Step 5:

[0357] Sending server-generated reports to users (e.g. email or dashboard notifications).

[0358] User Emotion Recognition

[0359] Step 1:

[0360] The user provides voice and facial expression data while operating the system (e.g., using a webcam or microphone).

[0361] Step 2:

[0362] The server receives the user's voice and facial expression data using an emotion engine.

[0363] Step 3:

[0364] The server uses an emotion engine to analyze the user's emotions.

[0365] Step 4:

[0366] The server receives the emotion data and adjusts the system's behavior based on the analysis results.

[0367] Step 5:

[0368] The server integrates the analyzed emotion data with the log data and performs further detailed analysis.

[0369] Step 6:

[0370] The server generates a report based on the user's emotional data to improve the user experience.

[0371] Specific examples

[0372] Example of automating device settings

[0373] The user enters the information to register a new smartphone and presses the "Register" button. The server receives the information, reads the smartphone profile from the database, and automatically applies the necessary network and security settings. It then sends a notification to the user that the setup is complete.

[0374] Automation of frequency testing

[0375] When a user sets up 100 Wi-Fi connection tests, the server receives the test conditions and performs 100 Wi-Fi connection tests on the device. The server collects success and failure data during the test and finally generates a report and sends it to the user.

[0376] Automating long-term testing

[0377] When a user sets up a 24-hour continuous use test, the server periodically monitors the terminal's operating status for 24 hours and notifies the user if any abnormalities occur.

[0378] Automated log analysis

[0379] The server collects log data from devices at a set time each day and runs it through an AI analysis engine to detect abnormal patterns. The analysis results are generated as a detailed report and sent to the user.

[0380] User Emotion Recognition

[0381] As a user operates the system, the emotion engine analyzes the user's emotions from voice and facial expression data. Based on the analysis results, the system adjusts system operation and makes suggestions to reduce the user's stress and anxiety. The emotion data is also integrated with log data for more detailed analysis, improving the user experience.

[0382] Example 2

[0383] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0384] The verification process for device development involves extremely time-consuming processes such as terminal configuration, repeated testing, long-term testing, and log analysis, which consumes a large amount of time and human resources.Furthermore, it is difficult to improve the user experience because the system cannot be adjusted to take user emotions into account.

[0385] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0386] In this invention, the server includes means for automating terminal configuration, means for recognizing user emotions, means for automatically executing multiple tests, means for periodically monitoring long-term tests, means for analyzing log data and detecting anomalies, means for reporting the analysis results, and means for adjusting system operation based on user emotion data, thereby enabling more efficient device verification processes, early detection and handling of defects, and an improved user experience.

[0387] A "means for automating terminal settings" is a means by which a server automatically performs network settings, security settings, and installation of necessary applications based on terminal information entered by the user.

[0388] The "means for automatically executing a test multiple times" is a means for the server to automatically execute a test on the terminal a specified number of times based on test conditions specified by the user.

[0389] The "means for periodically monitoring the long-term test" refers to a means for the server to periodically monitor the operating status of the terminal during the specified test period and to immediately notify the user if an abnormality occurs.

[0390] "Means for analyzing log data and detecting anomalies" refers to the means by which the server inputs log data acquired from the terminal into an AI analysis engine to detect abnormal patterns and errors.

[0391] The "means of reporting the analysis results" refers to the means of generating a detailed report based on the analysis results of the AI ​​analysis engine and providing it to the user.

[0392] The "means for recognizing the user's emotions" refers to a means by which the system acquires the user's voice and facial expressions and analyzes them using an emotion engine to recognize the user's emotional state.

[0393] The "means for adjusting system operation based on user emotion data" refers to a means for the system to improve operability and adjust the interface based on the recognized user emotion data.

[0394] "Means for collecting log data from terminals" refers to the means by which the server periodically obtains operation logs, system logs, application logs, etc. from terminals.

[0395] The present invention relates to a system that automates multiple processes in the verification process of device development and further recognizes user emotions.

[0396] Automating device configuration

[0397] When a user registers a new device in the system, the server receives the device information and automatically applies initial settings to the device, such as network settings, security settings, and installation of required applications. The server uses an API to perform these settings remotely. For example, when a user registers a smartphone, the server automatically configures Wi-Fi settings, security policies, and installs required applications. After the configuration is complete, the server notifies the user that the configuration was successful.

[0398] Automation of frequency testing

[0399] Based on the test conditions specified by the user, the server will perform the test on the device the specified number of times. For example, if you set the Wi-Fi connection test to be performed 100 times, the server will perform 100 connection attempts and compile the results. The server will collect data such as the number of successes, the number of failures, and the connection time, and generate a detailed report to send to the user.

[0400] Automating long-term testing

[0401] Once the user sets the test period, the server will periodically monitor the device during that period. For example, if a 24-hour long test is set, the server will periodically check the device's operating status (CPU usage, memory usage, network status, etc.). If an abnormality is detected, the server will immediately notify the user.

[0402] Automated log analysis

[0403] The server periodically collects log data from devices. This data is input into an AI analysis engine to detect abnormal patterns and errors. Based on the results of the AI ​​analysis engine, the server generates a detailed report and provides it to the user. For example, the server could collect operation logs from devices at a set time every day, run them through the AI ​​analysis engine to detect abnormalities, and send a weekly report to the user by email.

[0404] User Emotion Recognition

[0405] This system incorporates an emotion engine that recognizes emotions from the user's voice and facial expressions. The emotion engine analyzes the user's emotional data and uses the results to adjust the system's operation. For example, if a user feels stressed while operating the system, the emotion is recognized and the system suggests improvements to operability and interface adjustments. Emotion data is also integrated with log data for comprehensive analysis.

[0406] Examples of specific prompts include:

[0407] Register a new device and automatically install Wi-Fi settings and necessary apps.

[0408] "Please perform 100 Wi-Fi connection tests and submit the results as a report."

[0409] "Please conduct a 24-hour continuous use test and notify us immediately if any abnormalities occur."

[0410] "Analyze the device operation logs and prepare a weekly report if any abnormalities are detected."

[0411] "Please be aware of the user's stress level and provide suggestions for improving usability."

[0412] The system of the present invention is expected to improve the efficiency of the device verification process, enable early detection of defects, and improve the user experience.

[0413] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0414] System program processing flow

[0415] Automating device configuration

[0416] Step 1:

[0417] The user enters the information for the new device and clicks the registration button.

[0418] Specific operation: The user inputs information such as the device model name, serial number, and network settings. The input data is sent to the server.

[0419] Step 2:

[0420] The server receives the terminal information and temporarily stores it in a database.

[0421] Input: Device information (model name, serial number, network settings, etc.)

[0422] Data processing: Converting device information into the appropriate format and storing it in the database

[0423] Output: Saved device information

[0424] Step 3:

[0425] The server loads the device profile.

[0426] Input: Saved device information

[0427] Data calculation: Obtain the corresponding device profile (initial setting information) from the database

[0428] Output:Device profile

[0429] Step 4:

[0430] The server applies the initial settings to the device.

[0431] Input:Device profile

[0432] Specific operation: Automatically configures network settings (e.g. Wi-Fi settings), security settings, and installs required applications.

[0433] Output: Device with initial setup completed

[0434] Step 5:

[0435] The server will notify you that the settings are complete.

[0436] Input: Device information for which initial settings have been completed

[0437] Data processing: Generate notification messages and prepare them for sending

[0438] Output: Notification to the user that the settings have been completed

[0439] Automation of frequency testing

[0440] Step 1:

[0441] The user inputs the test conditions and clicks the test start button.

[0442] Specific operation: The user registers the test conditions (e.g., execute the Wi-Fi connection test 100 times). The input data is sent to the server.

[0443] Step 2:

[0444] The server receives the test conditions and stores them in a database.

[0445] Input: Test conditions (e.g., run Wi-Fi connection test 100 times)

[0446] Data processing: Convert test conditions into an appropriate format and save it in a database

[0447] Output: Saved test conditions

[0448] Step 3:

[0449] The server instructs the terminal to execute the test.

[0450] Input: Saved test conditions

[0451] Specific operation: The server sends a test execution command to the terminal.

[0452] Output: Test execution started

[0453] Step 4:

[0454] The server collects the test data.

[0455] Input: Test execution data (number of successes, number of failures, connection time, etc.)

[0456] Data calculation: The data obtained during the test is compiled and stored in a database.

[0457] Output: Collected test data

[0458] Step 5:

[0459] The server generates the report.

[0460] Input: Collected test data

[0461] Data Processing: Generate reports based on test results

[0462] Output: Generated report

[0463] Step 6:

[0464] The server sends the report to the user.

[0465] Input: Generated report

[0466] Specific behavior: Send the report to the user via email or system notification

[0467] Output: Reports test results to the user

[0468] Automating long-term testing

[0469] Step 1:

[0470] The user sets the test period and issues an instruction to start the test.

[0471] Specific operation: The user inputs the test period (e.g., 24 hours) and sends it to the server.

[0472] Step 2:

[0473] The server monitors the device during the test period.

[0474] Input: Test period setting information

[0475] Specific operation: The server periodically monitors the device's operating status during the set period.

[0476] Output: Monitoring data

[0477] Step 3:

[0478] The server checks the operation status.

[0479] Input: Monitoring data

[0480] Data calculation: Analyze the device's operating status (e.g., CPU usage, memory usage, network status)

[0481] Output: Analysis results of the operating status

[0482] Step 4:

[0483] The server detects an anomaly.

[0484] Input: Analysis results of operating conditions

[0485] Data arithmetic: Detecting abnormal patterns and errors

[0486] Output: Anomaly detection results

[0487] Step 5:

[0488] The server notifies the user of the abnormality.

[0489] Input: Anomaly detection results

[0490] Specific behavior: Generates a notification message and sends it to the user

[0491] Output: Abnormal notification to the user

[0492] Automated log analysis

[0493] Step 1:

[0494] The server collects log data from the terminal.

[0495] Specific operation: Periodically obtain system logs, operation logs, and application logs from the terminal.

[0496] Step 2:

[0497] The server inputs the log data into an AI analysis engine.

[0498] Input: Collected log data

[0499] Data processing: Converting collected log data into a format suitable for the AI ​​analysis engine

[0500] Output: Data input into the AI ​​analysis engine

[0501] Step 3:

[0502] An AI analysis engine analyzes the log data.

[0503] Input: Data entered into the AI ​​analysis engine

[0504] Data Calculation: Analyzing abnormal patterns and errors

[0505] Output: Analysis results

[0506] Step 4:

[0507] The server receives the analysis results and generates a detailed report.

[0508] Input: Analysis results

[0509] Data processing: Generate reports based on analysis results

[0510] Output: Generated report

[0511] Step 5:

[0512] The server provides the report to the user.

[0513] Input: Generated report

[0514] Specific behavior: Send the report to the user via email or system notification

[0515] Output: Regular reports to users

[0516] User Emotion Recognition

[0517] Step 1:

[0518] The system captures the user's voice and facial expressions.

[0519] Specific operation: Collects user voice and facial expression data using a webcam and microphone.

[0520] Step 2:

[0521] The emotion engine analyzes the emotion data.

[0522] Input: Collected voice and facial expression data

[0523] Data Calculation: The emotion engine analyzes the data and determines the user's emotional state.

[0524] Output: Emotion analysis results

[0525] Step 3:

[0526] The system receives the analysis results and adjusts its operations.

[0527] Input: Sentiment analysis results

[0528] Specific actions: Based on the analysis results, we suggest improvements to usability and interface adjustments.

[0529] Output: Suggestions for improving the user experience

[0530] Step 4:

[0531] The system uses emotional data to help with log analysis.

[0532] Input: Sentiment analysis results

[0533] Data calculation: Emotion data is integrated with log data for comprehensive analysis.

[0534] Output: Comprehensive analysis results

[0535] (Application example 2)

[0536] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0537] Conventional factory robot maintenance and testing processes require a lot of manual work and time, making them inefficient and limiting their ability to detect anomalies and improve the user experience. In particular, there is a lack of means to reduce the mental burden on workers.

[0538] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for automating terminal configuration, means for automatically executing multiple tests, means for periodically monitoring long-term tests, means for analyzing log data and detecting abnormalities, means for reporting the analysis results, means for performing emotion recognition and acquiring user emotion data, and means for adjusting system operation and proposing improvements based on the emotion data. This makes it possible to automate and streamline the maintenance and verification processes of factory robots, and further reduce the mental burden on workers.

[0539] A "means for automating device settings" is a means by which, when a user registers a new device in the system, the server reads the device's profile and automatically applies initial settings such as network settings and security settings.

[0540] The "means for automatically executing multiple tests" refers to a means for the server to automatically execute a set number of tests on the corresponding terminal based on test conditions specified by the user, and to tally up the test results.

[0541] The "means for periodically monitoring a long-term test" is a means for the server to periodically monitor the operating status of the terminal during the test period set by the user, and to immediately notify the user if an abnormality occurs.

[0542] "Means for analyzing log data and detecting anomalies" refers to a means for inputting log data collected from terminals into an AI analysis engine to automatically detect abnormal patterns and errors.

[0543] The "means for reporting the analysis results" is a means for generating a detailed report based on the analysis results of the log data and providing the report to the user.

[0544] The "means for performing emotion recognition and acquiring user emotion data" refers to means including an emotion engine for analyzing emotion data from the user's voice and facial expressions and acquiring that data.

[0545] "Means for adjusting system operation and proposing improvements based on emotional data" refers to means for adjusting system operation and proposing improvements to the system's operation method or interface based on the stress or anxiety felt by the user.

[0546] The present invention aims to automate and improve the efficiency of maintenance and inspection processes for factory robots. Specific embodiments for carrying out the present invention will be described in detail below.

[0547] System Configuration

[0548] The system includes the following major components:

[0549] Server: The core device that processes and analyzes data.

[0550] Factory robot: An automated device that performs tasks on a manufacturing line.

[0551] Sensor: A device that detects the robot's operating status, temperature, sound, etc.

[0552] Emotion engine: A system that performs speech analysis (e.g., IBM Watson® Speech to Text).

[0553] AI analytics engine: A machine learning model that performs anomaly detection (e.g., TENSORFLOW (registered trademark)).

[0554] System functions and their operation

[0555] 1. Automate device configuration:

[0556] When a user registers a new factory robot in the system, the server reads the robot's profile and automatically applies initial settings such as network settings and security settings. Once the configuration is complete, the server sends a notification to the user.

[0557] 2. Automating multiple tests:

[0558] When the user inputs test conditions (e.g., repeat a specific action 100 times), the server automatically executes the test based on the conditions on the robot, collates and analyzes the results, and provides a report of the test results to the user.

[0559] 3. Regular monitoring of long-term tests:

[0560] Once the user sets the duration of the long-term test, the server periodically monitors the robot's operation during that period. If an abnormality is detected, the server immediately notifies the user.

[0561] 4. Log data analysis and anomaly detection:

[0562] The server inputs the log data collected from the robot into an AI analysis engine, which detects abnormal patterns and errors. Based on the analysis results, a detailed report is generated and provided to the user.

[0563] 5. User Emotion Recognition:

[0564] An emotion engine is built into the server to collect emotional data from the voice and facial expressions of the worker while the robot is operating. If the user feels anxious or stressed, that emotional data is reflected in the system, which then adjusts the robot's behavior and suggests improvements.

[0565] Adding specific examples

[0566] For example, if a user sets up a 24-hour continuous use test, the server will periodically check the robot's operating status for 24 hours. If an abnormality is detected, the user will be notified immediately. Also, if a worker complains that the robot is "slow," the emotion engine will detect this stress and prioritize log analysis of that area.

[0567] Example of input prompt for generative AI model

[0568] Collect operation logs of robots that operate continuously for 24 hours and use an AI analysis engine to detect abnormal patterns. Also, analyze the voice data of workers and prioritize areas where they feel stress or anxiety in the log analysis and report them.

[0569] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0570] Step 1:

[0571] The server receives a registration request from the user for a new factory robot. The information entered includes the robot's profile data (e.g., model number, work profile). The server uses this information to automatically perform initial configuration such as network settings and security settings. After configuration is complete, the server sends a notification to the user, allowing them to confirm that the robot has been successfully added to the system.

[0572] Step 2:

[0573] The user inputs multiple test conditions into the server. The test conditions include the number of times a specific operation is to be repeated. Based on the conditions, the server sends commands to the robot to execute the test and collects the test execution results. The server aggregates the collected data and calculates the number of successes and failures. Finally, the server provides the test results to the user as a report.

[0574] Step 3:

[0575] The user sets a long-term test period. During that period, the server periodically sends commands to the robot to monitor its operating status (e.g., CPU usage, temperature). The server analyzes the operating data received from the robot, and if an abnormality is detected, it immediately sends a notification email to the user. This allows the user to detect and respond to abnormalities early.

[0576] Step 4:

[0577] The server collects log data from the robots daily or periodically. This log data is input into an AI analysis engine to detect abnormal patterns and errors. The server then generates a detailed report based on the data and provides it to the user, allowing the user to quickly identify potential problems and take measures.

[0578] Step 5:

[0579] While the robot is operating, the emotion engine analyzes the worker's voice and facial expressions in real time. The input voice data is passed through the emotion engine and output as emotional data. The server uses this emotional data to detect the user's stress or anxiety, and based on the results, adjusts the system's operation and suggests improvements. This reduces the mental burden on the worker and improves work efficiency.

[0580] For example, if a user sets up a 24-hour continuous use test, the server will monitor the robot 24 hours a day and immediately notify the user if any abnormalities are detected. Also, if a worker points out that the robot is slow, the emotion engine will detect stress and prioritize analysis of related logs, helping to resolve the problem quickly.

[0581] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0582] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0583] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0584] [Second embodiment]

[0585] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0586] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0587] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0588] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0589] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0590] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0591] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0592] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0593] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0594] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0595] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0596] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0597] The present invention relates to a system for automating processes in the verification process of device development, such as terminal setting, number testing, long-term testing, log analysis, etc. Specific embodiments for carrying out the present invention will be described in detail below.

[0598] Automating device configuration

[0599] The user registers a new device in the system. When the user enters the device information and presses the registration button, the server receives this information. The server reads the device profile from the database and automatically applies the initial settings, such as network settings and security settings, to the device. Once the settings are complete, the server sends a notification to the user that the settings are complete.

[0600] Specific examples

[0601] When a user registers a new smartphone in the system, the server reads the smartphone profile and automatically configures Wi-Fi settings, security policies, and installs necessary applications. After the configuration is complete, the server notifies the user that the smartphone configuration is complete.

[0602] Automation of frequency testing

[0603] The user accesses the test execution screen and enters the test conditions (e.g., run the Wi-Fi connection test 100 times in a row). When the user presses the test start button, the server receives the test conditions and causes the device to run the test the specified number of times. The server collects data obtained during the test (number of successes, number of failures, connection time, etc.) and generates a report based on this data. The generated report is sent from the server to the user.

[0604] Specific examples

[0605] If a user configures the system to perform 100 Wi-Fi connection tests, the server will perform all 100 connection attempts, compile the results, create a report, and send it to the user.

[0606] Automating long-term testing

[0607] Once the user issues a command to start the test and sets the test period (e.g., 24 hours), the server will continue to monitor the device during that period. The server periodically checks the device's operating status (CPU usage, memory usage, network status, etc.) and immediately notifies the user if any abnormalities are detected.

[0608] Specific examples

[0609] When a user sets up a 24-hour continuous use test, the server periodically checks the device's operating status for 24 hours and immediately notifies the user if any abnormalities are detected.

[0610] Automated log analysis

[0611] The server periodically collects log data from the device, inputs this log data into an AI analysis engine, and detects abnormal patterns and errors in the log data. The server then generates a detailed report based on the analysis results and provides it to the user.

[0612] Specific examples

[0613] The server collects operation logs from the devices at a set time every day and runs them through an AI analysis engine to detect abnormal patterns and errors. Based on the results, a weekly report is generated and sent to the user by email.

[0614] These features not only enable users to significantly reduce the amount of work required in the device verification process, but also enable highly accurate and rapid defect detection and efficient response. Furthermore, it is expected to contribute to shortening development schedules. This system will be an extremely useful tool for device developers.

[0615] The processing flow will be explained below.

[0616] Automating device configuration

[0617] Step 1:

[0618] The user enters the information for the new device and presses the "Register" button on the device registration screen.

[0619] Step 2:

[0620] The server receives the terminal information sent from the user.

[0621] Step 3:

[0622] The server reads the device profile from the database.

[0623] Step 4:

[0624] The server automatically applies the device's network and security settings based on the profile loaded.

[0625] Step 5:

[0626] The server confirms the status of the configuration.

[0627] Step 6:

[0628] The server will send a notification to the user that the setup is complete.

[0629] Automation of frequency testing

[0630] Step 1:

[0631] The user accesses the test execution screen and enters the test conditions (e.g., execute the Wi-Fi connection test 100 times in succession).

[0632] Step 2:

[0633] The user presses the "Start Test" button.

[0634] Step 3:

[0635] The server receives the input test conditions.

[0636] Step 4:

[0637] The server causes the terminal to execute the test a specified number of times.

[0638] Step 5:

[0639] The server collects data obtained during the test (number of successes, number of failures, connection time, etc.).

[0640] Step 6:

[0641] The server generates reports based on the collected data.

[0642] Step 7:

[0643] The server generates a report and sends it to the user.

[0644] Automating long-term testing

[0645] Step 1:

[0646] The user accesses the long-term test execution screen and inputs the test period (e.g., 24 hours).

[0647] Step 2:

[0648] The user presses the "Start Test" button.

[0649] Step 3:

[0650] The server sets a timer for the test period.

[0651] Step 4:

[0652] The server will periodically monitor the terminal status (e.g., CPU usage, memory usage, network status) during the test period.

[0653] Step 5:

[0654] As soon as the server detects an abnormality, it will notify the user.

[0655] Step 6:

[0656] After the long test is completed, the server reports the test results to the user.

[0657] Automated log analysis

[0658] Step 1:

[0659] The server periodically collects operation logs from the terminal based on a set schedule (e.g., collects logs at midnight every day).

[0660] Step 2:

[0661] The log data collected by the server is input into an AI analysis engine.

[0662] Step 3:

[0663] The server uses an AI analysis engine to detect abnormal patterns and errors in the log data.

[0664] Step 4:

[0665] The server generates a detailed report based on the results of the AI ​​analysis engine.

[0666] Step 5:

[0667] Sending server-generated reports to users (e.g. email or dashboard notifications).

[0668] Example 1

[0669] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0670] The verification process for conventional device development requires manual terminal configuration, test execution, long-term test monitoring, and log data analysis, which consumes a significant amount of time and effort. Additionally, there is a lack of systems for quickly detecting anomalies and taking appropriate action. Under these circumstances, it is difficult to ensure device quality while promoting efficient development.

[0671] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0672] In this invention, the server includes a means for automating terminal configuration, a means for automatically executing multiple tests, a means for periodically monitoring long-term tests, a means for analyzing log data and detecting anomalies, a means for reporting the analysis results, a means for monitoring the terminal's operating status, and a means for notifying the user of completion of configuration. This enables fast and accurate initial terminal configuration, and enables efficient and highly accurate device verification through the automation and monitoring of multiple tests and long-term tests. Furthermore, anomaly detection and user notification promotes rapid response and improves overall development efficiency.

[0673] "Means for automating terminal settings" refers to a function in which a user inputs terminal information into the system, and the server automatically performs network settings, security settings, and installation of necessary applications.

[0674] The "means for automatically executing multiple tests" is a function that allows the server to automatically execute a specified test multiple times based on conditions specified by the user, and collect the results.

[0675] "Means for periodically monitoring long-term testing" refers to a function in which the server periodically monitors the operating status of the terminal (CPU usage, memory usage, network status, etc.) during a period set by the user, and notifies the user if an abnormality occurs.

[0676] "Means for analyzing log data and detecting anomalies" refers to a function that inputs log data collected periodically by the server into an AI analysis engine to detect abnormal patterns and errors.

[0677] "Means for reporting the analysis results" refers to a function in which the server generates a detailed report based on the anomaly detection results obtained by the AI ​​analysis engine and notifies or provides it to the user.

[0678] "Means for monitoring the operating status of the terminal" refers to a function by which the server periodically checks the operating status of the terminal (CPU usage, memory usage, network status, etc.) during the test period or a specific monitoring period.

[0679] The "means for notifying the user of completion of setup" is a function in which the server sends a push notification or email notification to the user when the initial setup of the device is complete.

[0680] The present invention relates to a system for automating processes in the verification process of device development, such as terminal setting, number testing, long-term testing, log analysis, etc. Specific embodiments for carrying out the present invention will be described in detail below.

[0681] Automating device configuration

[0682] When a user registers a new device in the system, the server receives the information and retrieves the device profile from the database. The server then automatically configures the device's network and security settings and installs any necessary applications. Once the configuration is complete, the server sends a notification to the user. The hardware and software used are as follows:

[0683] Hardware: Servers, terminals

[0684] Software: Database, terminal configuration script

[0685] Specific examples

[0686] A user registers a new smartphone in the system. The server reads the smartphone profile and automatically configures Wi-Fi settings, security policies, and installs required applications. After the configuration is complete, the server notifies the user that the smartphone configuration is complete.

[0687] Example prompt sentence:

[0688] Register your new smartphone and automatically apply network settings and security policies.

[0689] Automation of frequency testing

[0690] When the user enters the test conditions (e.g., run a Wi-Fi connection test 100 times in a row) and presses the test start button, the server receives the test conditions and automatically runs the test the specified number of times. The server collects data obtained during the test (number of successes, number of failures, connection time, etc.) and generates a report based on this data. The generated report is sent from the server to the user. The hardware and software used are as follows:

[0691] Hardware: Servers, terminals

[0692] Software: Test management software, data collection module

[0693] Specific examples

[0694] The user sets up 100 Wi-Fi connection tests. The server executes all 100 connection attempts, compiles the results, creates a report, and sends it to the user.

[0695] Example prompt sentence:

[0696] Automatically run 100 Wi-Fi connection tests and compile the results into a report.

[0697] Automating long-term testing

[0698] Once the user sets the test period (e.g., 24 hours) and issues the command to start, the server will continue to monitor the terminal during that period. The server periodically checks the terminal's operating status (CPU usage, memory usage, network status, etc.) and immediately notifies the user if an abnormality occurs. The hardware and software used are as follows:

[0699] Hardware: Servers, terminals

[0700] Software: Monitoring software, alert systems

[0701] Specific examples

[0702] The user sets up a 24-hour continuous use test. The server periodically checks the device's operating status during the 24-hour period, and immediately notifies the user if an abnormality is detected.

[0703] Example prompt sentence:

[0704] Conduct a 24-hour continuous use test and notify us immediately if any abnormalities are detected.

[0705] Automated log analysis

[0706] The server periodically collects log data from the device and inputs it into an AI analysis engine for analysis. The analysis detects abnormal patterns and errors, and generates a detailed report based on the results. The generated report is provided to the user. The hardware and software used are as follows:

[0707] Hardware: Servers, terminals

[0708] Software: Log collection module, AI analysis engine

[0709] Specific examples

[0710] The server collects operation logs from the devices at a set time each day and inputs them into an AI analysis engine to detect abnormal patterns and errors. Based on the results, a weekly report is generated and sent to the user by email.

[0711] Example prompt sentence:

[0712] Collect device log data on a regular daily basis and analyze it with AI to detect abnormal patterns.

[0713] Through the above process, users can significantly reduce the effort required for device verification, and can detect defects quickly and accurately and address them efficiently. This is expected to improve the overall efficiency of device development and enhance quality assurance.

[0714] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0715] Processing flow of the terminal setting automation program

[0716] Step 1:

[0717] The user enters device information and presses the registration button. The user accesses the system interface and enters new device information (e.g., device ID, device name, device model, etc.). When the user presses the "Register" button, the information is sent to the server.

[0718] Input: Basic device information (device ID, device name, device model)

[0719] Output: Sending device information to the server

[0720] Step 2:

[0721] The server receives the device information and retrieves the device profile from the database. The server receives the HTTP request and prepares the received data for internal processing. It then issues an SQL query to the database to retrieve the profile information for the corresponding device.

[0722] Input: Device information sent by the user

[0723] Output: Device profile retrieved from the database

[0724] Step 3:

[0725] The server applies the settings to the device. Based on the device profile obtained by the server, it sends commands to the device to automatically configure network settings, security settings, and install applications.

[0726] Input: Device profile retrieved from the database

[0727] Output: Command to apply the settings to the device

[0728] Step 4:

[0729] The device receives and executes the settings. Based on the setting information received from the server, the device configures the network and security settings and installs the necessary applications.

[0730] Input: Configuration information sent from the server

[0731] Output: Performing configuration tasks

[0732] Step 5:

[0733] The server will send a notification to the user that the setup is complete. After the setup is complete, the server will send a push notification or email to the user to let them know that the setup is complete.

[0734] Input: Device with completed settings

[0735] Output: User notification

[0736] Processing flow of the automated program for number testing

[0737] Step 1:

[0738] The user enters the test conditions and presses the "Start Test" button. The user accesses the test management screen, enters the test conditions (e.g., 100 Wi-Fi connection tests), and presses the "Start" button.

[0739] Input: Test conditions (Wi-Fi connection test 100 times)

[0740] Output: Send test conditions to the server

[0741] Step 2:

[0742] The server receives the test conditions, prepares the test, analyzes the received test conditions, and sets the test schedule.

[0743] Input: Test conditions submitted by the user

[0744] Output: Test schedule settings

[0745] Step 3:

[0746] The server starts the test and sends multiple test execution commands to the terminal based on the specified test conditions.

[0747] Input: Exam Schedule

[0748] Output: Test execution command to terminal

[0749] Step 4:

[0750] The terminal executes the test. Upon receiving commands from the server, the terminal executes the test a specified number of times.

[0751] Input: Test execution command sent from the server

[0752] Output: Test results (number of successes, number of failures, connection time, etc.)

[0753] Step 5:

[0754] The server collects the test results and generates a report. The server collects the data obtained during the test run and generates a report based on that data.

[0755] Input: Test result

[0756] Output: Generate a report

[0757] Step 6:

[0758] The server sends the report to the user. The generated report is emailed to the user.

[0759] Input: Report

[0760] Output: Report sent to user

[0761] Processing flow of the automated program for long-term testing

[0762] Step 1:

[0763] The user sets the test period and issues a command to start the test. The user sets the test period (e.g., 24 hours) and presses the "Start" button.

[0764] Input: Test period (24 hours)

[0765] Output: Send test period to server

[0766] Step 2:

[0767] The server receives the instruction to start the test and begins monitoring. The server then sends a command to periodically monitor the operating status of the terminal during the test period.

[0768] Input: Test period instructions

[0769] Output: Monitor command to terminal

[0770] Step 3:

[0771] The device sends its operating status to the server. Following instructions from the server, the device periodically reports its operating status (CPU usage, memory usage, network status, etc.) to the server.

[0772] Input: Surveillance order

[0773] Output: Operation status data

[0774] Step 4:

[0775] The server analyzes the operational status data and detects abnormalities. The server periodically analyzes the operational status data it receives and detects abnormalities.

[0776] Input: Operational status data

[0777] Output: Anomaly detection results

[0778] Step 5:

[0779] Notify the user when the server detects an abnormality. If an abnormality occurs, the server will immediately send a notification to the user.

[0780] Input: Anomaly detection results

[0781] Output: Anomaly notification to the user

[0782] Log analysis automation program processing flow

[0783] Step 1:

[0784] The server periodically collects log data, and obtains the log data from the device via the logging API.

[0785] Input: Device log data

[0786] Output: Send log data to the server

[0787] Step 2:

[0788] The server inputs the log data into an AI analysis engine, which then inputs the log data collected by the server into the AI ​​analysis engine to detect abnormal patterns and errors.

[0789] Input: Log data

[0790] Output: AI analysis results

[0791] Step 3:

[0792] The server generates a report based on the analysis results. A detailed report is generated based on the anomaly detection results obtained by the AI ​​analysis engine.

[0793] Input: AI analysis results

[0794] Output: Generate a report

[0795] Step 4:

[0796] The server provides the report to the user, either by emailing the generated report to the user or making it available for viewing within the system.

[0797] Input: Report

[0798] Output: Report provided to user

[0799] The above processing steps enable the automation of terminal settings, the automatic execution of multiple tests, the monitoring of long-term tests, and the analysis of log data to be performed consistently and rationally.

[0800] (Application example 1)

[0801] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0802] Conventional methods for configuring, testing, and monitoring industrial equipment and terminals require a great deal of manual work, making them inefficient and prone to human error. Initially configuring new equipment, conducting a large number of tests, and monitoring over long periods of time require significant time and effort. Furthermore, detecting and reporting anomalies is also manual, making it difficult to respond quickly. Therefore, there is a need for efficient management of industrial equipment, improved reliability, and rapid detection and response of anomalies.

[0803] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0804] In this invention, the server includes means for automating terminal configuration, means for automatically executing multiple tests, means for periodically monitoring long-term tests, means for analyzing log data and detecting abnormalities, means for reporting the analysis results, means for reading industrial equipment profiles and automatically applying necessary initial settings, and means for collecting test results and monitoring them in real time. This automates the configuration, testing, monitoring, abnormality detection, and reporting of industrial equipment and terminals, enabling efficient and reliable management.

[0805] "Means for automating terminal settings" refers to a function that automatically performs the initial settings of terminals and industrial equipment without requiring manual operation by the user.

[0806] The "means for automatically executing multiple tests" means a function that automatically executes a certain number of tests on terminals or industrial equipment based on pre-set conditions.

[0807] "Means for periodically monitoring long-term tests" refers to a function that periodically monitors the operating status of terminals and industrial equipment when conducting long-term tests, and records and analyzes the results.

[0808] The "means for analyzing log data and detecting abnormalities" refers to a function that analyzes log data collected from terminals and industrial equipment and detects abnormal operations or errors from the data.

[0809] The "means for reporting the analysis results" has a function for reporting the analysis results of the log data to the user, and notifies the user if an abnormality is detected.

[0810] "Means for reading the profile of industrial equipment and automatically applying the required initial settings" means a function that reads the profile information of industrial equipment and automatically applies the required initial settings (Wi-Fi settings, security policies, etc.) based on that information.

[0811] "Means for collecting test results and monitoring them in real time" means a means for collecting result data obtained during the test and monitoring the data in real time.

[0812] A system for realizing the present invention includes a server, industrial equipment (terminals), and user interaction. Specific embodiments will be described in detail below.

[0813] Hardware and Software

[0814] Hardware:

[0815] Server: A computer system with a powerful processor

[0816] Industrial equipment: Equipment equipped with various sensors and communication modules (e.g., robots, sensor devices)

[0817] Smart glasses: wearable devices for displaying real-time monitoring information (e.g., Google Glass)

[0818] software:

[0819] Server software: Python, real-time data streaming API, database management system

[0820] Log analysis engine: Anomaly detection algorithm using AI models

[0821] User interface: Web browser or dedicated application

[0822] Details of data processing and calculation

[0823] When a user registers an industrial device, the server reads its profile from the database and automatically applies initial settings (e.g., network settings, security policies) to the industrial device. This initial setting process is performed based on the device's profile information.

[0824] When industrial equipment starts a new task, the server runs multiple tests and collects and monitors the results in real time. When the test is completed, the server analyzes the obtained data and notifies the user if an abnormality is detected. This is done using an AI analysis engine for anomaly detection.

[0825] Furthermore, for equipment undergoing long-term testing, its operation is regularly monitored, and the server detects and notifies users of any abnormalities as necessary. All log data is centrally managed on the server and provided to users as reports as needed.

[0826] Specific examples

[0827] This section explains how this system works for robots used in factories. When a user registers a new robot in the system, the server loads the robot's profile and automatically applies initial settings such as Wi-Fi settings and security policies. It then conducts multiple operational tests and 24-hour continuous operation tests, monitoring the robot in real time. During this process, the server collects log data and detects abnormalities using an AI analysis engine. Detected abnormalities are immediately notified to the user, who can view the information through the smart glasses. In this way, users can efficiently manage the robot's operations and quickly respond to any abnormalities that occur.

[0828] Prompt Sentence Examples

[0829] "Please develop a system that monitors robots in factories and notifies them when an abnormality is detected. For example, it can automatically set up Wi-Fi and security policies, and conduct repeated and long-term tests to monitor the results in real time."

[0830] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0831] Step 1:

[0832] A user registers a new industrial device in the system. As input, the user provides device profile information (e.g., device ID, network settings, security policy, etc.). The server receives this information and stores it in a database. As output, the server sends a notification to the user that the registration is complete.

[0833] Step 2:

[0834] The server reads the saved profile information and automatically applies the necessary initial settings to the industrial equipment based on that information. As input, it receives the profile information and executes Wi-Fi settings, security policies, etc. based on that information. As output, it logs that the initial settings were successfully completed.

[0835] Step 3:

[0836] The user configures tests in the system. Specifically, the user specifies multiple tests (e.g., operation tests or connection tests) or long-term tests (e.g., 24-hour continuous operation tests). As input, the user provides test conditions (number of tests and duration). The server receives the test conditions and creates a test plan. As output, a test plan is generated and saved.

[0837] Step 4:

[0838] The server executes the specified tests. It reads the test plan as input and executes each test sequentially. Specifically, for multiple tests, it performs the specified number of attempts, and for long-term tests, it periodically monitors the operation within the specified time. As output, it records the results of each test (success / failure and operation log).

[0839] Step 5:

[0840] The server analyzes the log data collected during and after the test. It receives the collected operation logs and test results as input. It uses an AI model for analysis to detect anomalies. It outputs the analysis results in a report.

[0841] Step 6:

[0842] The server reports the analysis results to the user. As input, it generates a report of anomaly detection and notifies the user. As output, the user receives the report and can get detailed information about the status of industrial equipment.

[0843] Step 7:

[0844] If an anomaly is detected, the server immediately notifies the user. As input, it generates an anomaly notification based on the analysis results and forwards the content to the user. As output, the user receives the notification on a device such as smart glasses, allowing them to quickly address the problem.

[0845] Step 8:

[0846] Users review the maintenance and settings of industrial equipment based on the reports provided by the server. As input, they refer to the information in the report and carry out the necessary procedures. As output, they can confirm the improved settings and maintenance status. This cycle ensures efficient and reliable operation of industrial equipment.

[0847] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0848] This invention relates to a system that uses AI to automate processes such as terminal configuration, number testing, long-term testing, and log analysis in the verification process of device development, and further combines it with an emotion engine that recognizes user emotions. Specific embodiments for carrying out the present invention are described in detail below.

[0849] Automating device configuration

[0850] The user registers a new device in the system. When the user enters the device information and presses the registration button, the server receives this information. The server reads the device profile from the database and automatically applies the initial settings, such as network settings and security settings, to the device. Once the settings are complete, the server sends a notification to the user that the settings are complete.

[0851] Specific examples

[0852] When a user registers a new smartphone in the system, the server reads the smartphone profile and automatically configures Wi-Fi settings, security policies, and installs necessary applications. After the configuration is complete, the server notifies the user that the smartphone configuration is complete.

[0853] Automation of frequency testing

[0854] The user accesses the test execution screen and enters the test conditions (e.g., run the Wi-Fi connection test 100 times in a row). When the user presses the test start button, the server receives the test conditions and causes the device to run the test the specified number of times. The server collects data obtained during the test (number of successes, number of failures, connection time, etc.) and generates a report based on this data. The generated report is sent from the server to the user.

[0855] Specific examples

[0856] If a user configures the system to perform 100 Wi-Fi connection tests, the server will perform all 100 connection attempts, compile the results, create a report, and send it to the user.

[0857] Automating long-term testing

[0858] Once the user issues a command to start the test and sets the test period (e.g., 24 hours), the server will continue to monitor the device during that period. The server periodically checks the device's operating status (CPU usage, memory usage, network status, etc.) and immediately notifies the user if any abnormalities are detected.

[0859] Specific examples

[0860] When a user sets up a 24-hour continuous use test, the server periodically checks the device's operating status for 24 hours and immediately notifies the user if any abnormalities are detected.

[0861] Automated log analysis

[0862] The server periodically collects log data from the device, inputs this log data into an AI analysis engine, and detects abnormal patterns and errors in the log data. The server then generates a detailed report based on the analysis results and provides it to the user.

[0863] Specific examples

[0864] The server collects operation logs from the devices at a set time every day and runs them through an AI analysis engine to detect abnormal patterns and errors. Based on the results, a weekly report is generated and sent to the user by email.

[0865] User Emotion Recognition

[0866] The system also incorporates an emotion engine that recognizes the user's emotions. This emotion engine analyzes emotional data from the user's voice and facial expressions, and uses the results to adjust the system's operations and analyze log data.

[0867] Specific examples

[0868] The emotion engine recognizes emotions such as stress and anxiety felt by users while operating the system. Based on the results, the system makes suggestions to improve the user experience and analyzes log data by adding emotion data. For example, if a specific operation causes stress over a long period of time, the system will make suggestions to improve that operation method or the system interface.

[0869] Combining these elements will enable the device verification process to be more efficient, defects to be detected and addressed early, and user experience to be improved. This system will serve as a very useful tool for developers.

[0870] The processing flow will be explained below.

[0871] Automating device configuration

[0872] Step 1:

[0873] The user enters the information for the new device and presses the "Register" button on the device registration screen.

[0874] Step 2:

[0875] The server receives the terminal information sent from the user.

[0876] Step 3:

[0877] The server reads the device profile from the database.

[0878] Step 4:

[0879] The server automatically applies the device's network and security settings based on the profile loaded.

[0880] Step 5:

[0881] The server confirms the status of the configuration.

[0882] Step 6:

[0883] The server will send a notification to the user that the setup is complete.

[0884] Automation of frequency testing

[0885] Step 1:

[0886] The user accesses the test execution screen and enters the test conditions (e.g., execute the Wi-Fi connection test 100 times in succession).

[0887] Step 2:

[0888] The user presses the "Start Test" button.

[0889] Step 3:

[0890] The server receives the input test conditions.

[0891] Step 4:

[0892] The server causes the terminal to execute the test a specified number of times.

[0893] Step 5:

[0894] The server collects data obtained during the test (number of successes, number of failures, connection time, etc.).

[0895] Step 6:

[0896] The server generates reports based on the collected data.

[0897] Step 7:

[0898] The server generates a report and sends it to the user.

[0899] Automating long-term testing

[0900] Step 1:

[0901] The user accesses the long-term test execution screen and inputs the test period (e.g., 24 hours).

[0902] Step 2:

[0903] The user presses the "Start Test" button.

[0904] Step 3:

[0905] The server sets a timer for the test period.

[0906] Step 4:

[0907] The server will periodically monitor the terminal status (e.g., CPU usage, memory usage, network status) during the test period.

[0908] Step 5:

[0909] As soon as the server detects an abnormality, it will notify the user.

[0910] Step 6:

[0911] After the long test is completed, the server reports the test results to the user.

[0912] Automated log analysis

[0913] Step 1:

[0914] The server periodically collects operation logs from the terminal based on a set schedule (e.g., collects logs at midnight every day).

[0915] Step 2:

[0916] The log data collected by the server is input into an AI analysis engine.

[0917] Step 3:

[0918] The server uses an AI analysis engine to detect abnormal patterns and errors in the log data.

[0919] Step 4:

[0920] The server generates a detailed report based on the results of the AI ​​analysis engine.

[0921] Step 5:

[0922] Sending server-generated reports to users (e.g. email or dashboard notifications).

[0923] User Emotion Recognition

[0924] Step 1:

[0925] The user provides voice and facial expression data while operating the system (e.g., using a webcam or microphone).

[0926] Step 2:

[0927] The server receives the user's voice and facial expression data using an emotion engine.

[0928] Step 3:

[0929] The server uses an emotion engine to analyze the user's emotions.

[0930] Step 4:

[0931] The server receives the emotion data and adjusts the system's behavior based on the analysis results.

[0932] Step 5:

[0933] The server integrates the analyzed emotion data with the log data and performs further detailed analysis.

[0934] Step 6:

[0935] The server generates a report based on the user's emotional data to improve the user experience.

[0936] Specific examples

[0937] Example of automating device settings

[0938] The user enters the information to register a new smartphone and presses the "Register" button. The server receives the information, reads the smartphone profile from the database, and automatically applies the necessary network and security settings. It then sends a notification to the user that the setup is complete.

[0939] Automation of frequency testing

[0940] When a user sets up 100 Wi-Fi connection tests, the server receives the test conditions and performs 100 Wi-Fi connection tests on the device. The server collects success and failure data during the test and finally generates a report and sends it to the user.

[0941] Automating long-term testing

[0942] When a user sets up a 24-hour continuous use test, the server periodically monitors the terminal's operating status for 24 hours and notifies the user if any abnormalities occur.

[0943] Automated log analysis

[0944] The server collects log data from devices at a set time each day and runs it through an AI analysis engine to detect abnormal patterns. The analysis results are generated as a detailed report and sent to the user.

[0945] User Emotion Recognition

[0946] As a user operates the system, the emotion engine analyzes the user's emotions from voice and facial expression data. Based on the analysis results, the system adjusts system operation and makes suggestions to reduce the user's stress and anxiety. The emotion data is also integrated with log data for more detailed analysis, improving the user experience.

[0947] Example 2

[0948] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0949] The verification process for device development involves extremely time-consuming processes such as terminal configuration, repeated testing, long-term testing, and log analysis, which consumes a large amount of time and human resources.Furthermore, it is difficult to improve the user experience because the system cannot be adjusted to take user emotions into account.

[0950] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0951] In this invention, the server includes means for automating terminal configuration, means for recognizing user emotions, means for automatically executing multiple tests, means for periodically monitoring long-term tests, means for analyzing log data and detecting anomalies, means for reporting the analysis results, and means for adjusting system operation based on user emotion data, thereby enabling more efficient device verification processes, early detection and handling of defects, and an improved user experience.

[0952] A "means for automating terminal settings" is a means by which a server automatically performs network settings, security settings, and installation of necessary applications based on terminal information entered by the user.

[0953] The "means for automatically executing a test multiple times" is a means for the server to automatically execute a test on the terminal a specified number of times based on test conditions specified by the user.

[0954] The "means for periodically monitoring the long-term test" refers to a means for the server to periodically monitor the operating status of the terminal during the specified test period and to immediately notify the user if an abnormality occurs.

[0955] "Means for analyzing log data and detecting anomalies" refers to the means by which the server inputs log data acquired from the terminal into an AI analysis engine to detect abnormal patterns and errors.

[0956] The "means of reporting the analysis results" refers to the means of generating a detailed report based on the analysis results of the AI ​​analysis engine and providing it to the user.

[0957] The "means for recognizing the user's emotions" refers to a means by which the system acquires the user's voice and facial expressions and analyzes them using an emotion engine to recognize the user's emotional state.

[0958] The "means for adjusting system operation based on user emotion data" refers to a means for the system to improve operability and adjust the interface based on the recognized user emotion data.

[0959] "Means for collecting log data from terminals" refers to the means by which the server periodically obtains operation logs, system logs, application logs, etc. from terminals.

[0960] The present invention relates to a system that automates multiple processes in the verification process of device development and further recognizes user emotions.

[0961] Automating device configuration

[0962] When a user registers a new device in the system, the server receives the device information and automatically applies initial settings to the device, such as network settings, security settings, and installation of required applications. The server uses an API to perform these settings remotely. For example, when a user registers a smartphone, the server automatically configures Wi-Fi settings, security policies, and installs required applications. After the configuration is complete, the server notifies the user that the configuration was successful.

[0963] Automation of frequency testing

[0964] Based on the test conditions specified by the user, the server will perform the test on the device the specified number of times. For example, if you set the Wi-Fi connection test to be performed 100 times, the server will perform 100 connection attempts and compile the results. The server will collect data such as the number of successes, the number of failures, and the connection time, and generate a detailed report to send to the user.

[0965] Automating long-term testing

[0966] Once the user sets the test period, the server will periodically monitor the device during that period. For example, if a 24-hour long test is set, the server will periodically check the device's operating status (CPU usage, memory usage, network status, etc.). If an abnormality is detected, the server will immediately notify the user.

[0967] Automated log analysis

[0968] The server periodically collects log data from devices. This data is input into an AI analysis engine to detect abnormal patterns and errors. Based on the results of the AI ​​analysis engine, the server generates a detailed report and provides it to the user. For example, the server could collect operation logs from devices at a set time every day, run them through the AI ​​analysis engine to detect abnormalities, and send a weekly report to the user by email.

[0969] User Emotion Recognition

[0970] This system incorporates an emotion engine that recognizes emotions from the user's voice and facial expressions. The emotion engine analyzes the user's emotional data and uses the results to adjust the system's operation. For example, if a user feels stressed while operating the system, the emotion is recognized and the system suggests improvements to operability and interface adjustments. Emotion data is also integrated with log data for comprehensive analysis.

[0971] Examples of specific prompts include:

[0972] Register a new device and automatically install Wi-Fi settings and necessary apps.

[0973] "Please perform 100 Wi-Fi connection tests and submit the results as a report."

[0974] "Please conduct a 24-hour continuous use test and notify us immediately if any abnormalities occur."

[0975] "Analyze the device operation logs and prepare a weekly report if any abnormalities are detected."

[0976] "Please be aware of the user's stress level and provide suggestions for improving usability."

[0977] The system of the present invention is expected to improve the efficiency of the device verification process, enable early detection of defects, and improve the user experience.

[0978] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0979] System program processing flow

[0980] Automating device configuration

[0981] Step 1:

[0982] The user enters the information for the new device and clicks the registration button.

[0983] Specific operation: The user inputs information such as the device model name, serial number, and network settings. The input data is sent to the server.

[0984] Step 2:

[0985] The server receives the terminal information and temporarily stores it in a database.

[0986] Input: Device information (model name, serial number, network settings, etc.)

[0987] Data processing: Converting device information into the appropriate format and storing it in the database

[0988] Output: Saved device information

[0989] Step 3:

[0990] The server loads the device profile.

[0991] Input: Saved device information

[0992] Data calculation: Obtain the corresponding device profile (initial setting information) from the database

[0993] Output:Device profile

[0994] Step 4:

[0995] The server applies the initial settings to the device.

[0996] Input:Device profile

[0997] Specific operation: Automatically configures network settings (e.g. Wi-Fi settings), security settings, and installs required applications.

[0998] Output: Device with initial setup completed

[0999] Step 5:

[1000] The server will notify you that the settings are complete.

[1001] Input: Device information for which initial settings have been completed

[1002] Data processing: Generate notification messages and prepare them for sending

[1003] Output: Notification to the user that the settings have been completed

[1004] Automation of frequency testing

[1005] Step 1:

[1006] The user inputs the test conditions and clicks the test start button.

[1007] Specific operation: The user registers the test conditions (e.g., execute the Wi-Fi connection test 100 times). The input data is sent to the server.

[1008] Step 2:

[1009] The server receives the test conditions and stores them in a database.

[1010] Input: Test conditions (e.g., run Wi-Fi connection test 100 times)

[1011] Data processing: Convert test conditions into an appropriate format and save it in a database

[1012] Output: Saved test conditions

[1013] Step 3:

[1014] The server instructs the terminal to execute the test.

[1015] Input: Saved test conditions

[1016] Specific operation: The server sends a test execution command to the terminal.

[1017] Output: Test execution started

[1018] Step 4:

[1019] The server collects the test data.

[1020] Input: Test execution data (number of successes, number of failures, connection time, etc.)

[1021] Data calculation: The data obtained during the test is compiled and stored in a database.

[1022] Output: Collected test data

[1023] Step 5:

[1024] The server generates the report.

[1025] Input: Collected test data

[1026] Data Processing: Generate reports based on test results

[1027] Output: Generated report

[1028] Step 6:

[1029] The server sends the report to the user.

[1030] Input: Generated report

[1031] Specific behavior: Send the report to the user via email or system notification

[1032] Output: Reports test results to the user

[1033] Automating long-term testing

[1034] Step 1:

[1035] The user sets the test period and issues an instruction to start the test.

[1036] Specific operation: The user inputs the test period (e.g., 24 hours) and sends it to the server.

[1037] Step 2:

[1038] The server monitors the device during the test period.

[1039] Input: Test period setting information

[1040] Specific operation: The server periodically monitors the device's operating status during the set period.

[1041] Output: Monitoring data

[1042] Step 3:

[1043] The server checks the operation status.

[1044] Input: Monitoring data

[1045] Data calculation: Analyze the device's operating status (e.g., CPU usage, memory usage, network status)

[1046] Output: Analysis results of the operating status

[1047] Step 4:

[1048] The server detects an anomaly.

[1049] Input: Analysis results of operating conditions

[1050] Data arithmetic: Detecting abnormal patterns and errors

[1051] Output: Anomaly detection results

[1052] Step 5:

[1053] The server notifies the user of the abnormality.

[1054] Input: Anomaly detection results

[1055] Specific behavior: Generates a notification message and sends it to the user

[1056] Output: Abnormal notification to the user

[1057] Automated log analysis

[1058] Step 1:

[1059] The server collects log data from the terminal.

[1060] Specific operation: Periodically obtain system logs, operation logs, and application logs from the terminal.

[1061] Step 2:

[1062] The server inputs the log data into an AI analysis engine.

[1063] Input: Collected log data

[1064] Data processing: Converting collected log data into a format suitable for the AI ​​analysis engine

[1065] Output: Data input into the AI ​​analysis engine

[1066] Step 3:

[1067] An AI analysis engine analyzes the log data.

[1068] Input: Data entered into the AI ​​analysis engine

[1069] Data Calculation: Analyzing abnormal patterns and errors

[1070] Output: Analysis results

[1071] Step 4:

[1072] The server receives the analysis results and generates a detailed report.

[1073] Input: Analysis results

[1074] Data processing: Generate reports based on analysis results

[1075] Output: Generated report

[1076] Step 5:

[1077] The server provides the report to the user.

[1078] Input: Generated report

[1079] Specific behavior: Send the report to the user via email or system notification

[1080] Output: Regular reports to users

[1081] User Emotion Recognition

[1082] Step 1:

[1083] The system captures the user's voice and facial expressions.

[1084] Specific operation: Collects user voice and facial expression data using a webcam and microphone.

[1085] Step 2:

[1086] The emotion engine analyzes the emotion data.

[1087] Input: Collected voice and facial expression data

[1088] Data Calculation: The emotion engine analyzes the data and determines the user's emotional state.

[1089] Output: Emotion analysis results

[1090] Step 3:

[1091] The system receives the analysis results and adjusts its operations.

[1092] Input: Sentiment analysis results

[1093] Specific actions: Based on the analysis results, we suggest improvements to usability and interface adjustments.

[1094] Output: Suggestions for improving the user experience

[1095] Step 4:

[1096] The system uses emotional data to help with log analysis.

[1097] Input: Sentiment analysis results

[1098] Data calculation: Emotion data is integrated with log data for comprehensive analysis.

[1099] Output: Comprehensive analysis results

[1100] (Application example 2)

[1101] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1102] Conventional factory robot maintenance and testing processes require a lot of manual work and time, making them inefficient and limiting their ability to detect anomalies and improve the user experience. In particular, there is a lack of means to reduce the mental burden on workers.

[1103] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for automating terminal configuration, means for automatically executing multiple tests, means for periodically monitoring long-term tests, means for analyzing log data and detecting abnormalities, means for reporting the analysis results, means for performing emotion recognition and acquiring user emotion data, and means for adjusting system operation and proposing improvements based on the emotion data. This makes it possible to automate and streamline the maintenance and verification processes of factory robots, and further reduce the mental burden on workers.

[1104] A "means for automating device settings" is a means by which, when a user registers a new device in the system, the server reads the device's profile and automatically applies initial settings such as network settings and security settings.

[1105] The "means for automatically executing multiple tests" refers to a means for the server to automatically execute a set number of tests on the corresponding terminal based on test conditions specified by the user, and to tally up the test results.

[1106] The "means for periodically monitoring a long-term test" is a means for the server to periodically monitor the operating status of the terminal during the test period set by the user, and to immediately notify the user if an abnormality occurs.

[1107] "Means for analyzing log data and detecting anomalies" refers to a means for inputting log data collected from terminals into an AI analysis engine to automatically detect abnormal patterns and errors.

[1108] The "means for reporting the analysis results" is a means for generating a detailed report based on the analysis results of the log data and providing the report to the user.

[1109] The "means for performing emotion recognition and acquiring user emotion data" refers to means including an emotion engine for analyzing emotion data from the user's voice and facial expressions and acquiring that data.

[1110] "Means for adjusting system operation and proposing improvements based on emotional data" refers to means for adjusting system operation and proposing improvements to the system's operation method or interface based on the stress or anxiety felt by the user.

[1111] The present invention aims to automate and improve the efficiency of maintenance and inspection processes for factory robots. Specific embodiments for carrying out the present invention will be described in detail below.

[1112] System Configuration

[1113] The system includes the following major components:

[1114] Server: The core device that processes and analyzes data.

[1115] Factory robot: An automated device that performs tasks on a manufacturing line.

[1116] Sensor: A device that detects the robot's operating status, temperature, sound, etc.

[1117] Emotion engine: A system that performs speech analysis (e.g., IBM Watson Speech to Text).

[1118] AI analytics engine: Machine learning models (e.g., TensorFlow) that perform anomaly detection.

[1119] System functions and their operation

[1120] 1. Automate device configuration:

[1121] When a user registers a new factory robot in the system, the server reads the robot's profile and automatically applies initial settings such as network settings and security settings. Once the configuration is complete, the server sends a notification to the user.

[1122] 2. Automating multiple tests:

[1123] When the user inputs test conditions (e.g., repeat a specific action 100 times), the server automatically executes the test based on the conditions on the robot, collates and analyzes the results, and provides a report of the test results to the user.

[1124] 3. Regular monitoring of long-term tests:

[1125] Once the user sets the duration of the long-term test, the server periodically monitors the robot's operation during that period. If an abnormality is detected, the server immediately notifies the user.

[1126] 4. Log data analysis and anomaly detection:

[1127] The server inputs the log data collected from the robot into an AI analysis engine, which detects abnormal patterns and errors. Based on the analysis results, a detailed report is generated and provided to the user.

[1128] 5. User Emotion Recognition:

[1129] An emotion engine is built into the server to collect emotional data from the voice and facial expressions of the worker while the robot is operating. If the user feels anxious or stressed, that emotional data is reflected in the system, which then adjusts the robot's behavior and suggests improvements.

[1130] Adding specific examples

[1131] For example, if a user sets up a 24-hour continuous use test, the server will periodically check the robot's operating status for 24 hours. If an abnormality is detected, the user will be notified immediately. Also, if a worker complains that the robot is "slow," the emotion engine will detect this stress and prioritize log analysis of that area.

[1132] Example of input prompt for generative AI model

[1133] Collect operation logs of robots that operate continuously for 24 hours and use an AI analysis engine to detect abnormal patterns. Also, analyze the voice data of workers and prioritize areas where they feel stress or anxiety in the log analysis and report them.

[1134] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1135] Step 1:

[1136] The server receives a registration request from the user for a new factory robot. The information entered includes the robot's profile data (e.g., model number, work profile). The server uses this information to automatically perform initial configuration such as network settings and security settings. After configuration is complete, the server sends a notification to the user, allowing them to confirm that the robot has been successfully added to the system.

[1137] Step 2:

[1138] The user inputs multiple test conditions into the server. The test conditions include the number of times a specific operation is to be repeated. Based on the conditions, the server sends commands to the robot to execute the test and collects the test execution results. The server aggregates the collected data and calculates the number of successes and failures. Finally, the server provides the test results to the user as a report.

[1139] Step 3:

[1140] The user sets a long-term test period. During that period, the server periodically sends commands to the robot to monitor its operating status (e.g., CPU usage, temperature). The server analyzes the operating data received from the robot, and if an abnormality is detected, it immediately sends a notification email to the user. This allows the user to detect and respond to abnormalities early.

[1141] Step 4:

[1142] The server collects log data from the robots daily or periodically. This log data is input into an AI analysis engine to detect abnormal patterns and errors. The server then generates a detailed report based on the data and provides it to the user, allowing the user to quickly identify potential problems and take measures.

[1143] Step 5:

[1144] While the robot is operating, the emotion engine analyzes the worker's voice and facial expressions in real time. The input voice data is passed through the emotion engine and output as emotional data. The server uses this emotional data to detect the user's stress or anxiety, and based on the results, adjusts the system's operation and suggests improvements. This reduces the mental burden on the worker and improves work efficiency.

[1145] For example, if a user sets up a 24-hour continuous use test, the server will monitor the robot 24 hours a day and immediately notify the user if any abnormalities are detected. Also, if a worker points out that the robot is slow, the emotion engine will detect stress and prioritize analysis of related logs, helping to resolve the problem quickly.

[1146] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1147] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1148] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1149] [Third embodiment]

[1150] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1151] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1152] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1153] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1154] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1155] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1156] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1157] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1158] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1159] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1160] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1161] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1162] The present invention relates to a system for automating processes in the verification process of device development, such as terminal setting, number testing, long-term testing, log analysis, etc. Specific embodiments for carrying out the present invention will be described in detail below.

[1163] Automating device configuration

[1164] The user registers a new device in the system. When the user enters the device information and presses the registration button, the server receives this information. The server reads the device profile from the database and automatically applies the initial settings, such as network settings and security settings, to the device. Once the settings are complete, the server sends a notification to the user that the settings are complete.

[1165] Specific examples

[1166] When a user registers a new smartphone in the system, the server reads the smartphone profile and automatically configures Wi-Fi settings, security policies, and installs necessary applications. After the configuration is complete, the server notifies the user that the smartphone configuration is complete.

[1167] Automation of frequency testing

[1168] The user accesses the test execution screen and enters the test conditions (e.g., run the Wi-Fi connection test 100 times in a row). When the user presses the test start button, the server receives the test conditions and causes the device to run the test the specified number of times. The server collects data obtained during the test (number of successes, number of failures, connection time, etc.) and generates a report based on this data. The generated report is sent from the server to the user.

[1169] Specific examples

[1170] If a user configures the system to perform 100 Wi-Fi connection tests, the server will perform all 100 connection attempts, compile the results, create a report, and send it to the user.

[1171] Automating long-term testing

[1172] Once the user issues a command to start the test and sets the test period (e.g., 24 hours), the server will continue to monitor the device during that period. The server periodically checks the device's operating status (CPU usage, memory usage, network status, etc.) and immediately notifies the user if any abnormalities are detected.

[1173] Specific examples

[1174] When a user sets up a 24-hour continuous use test, the server periodically checks the device's operating status for 24 hours and immediately notifies the user if any abnormalities are detected.

[1175] Automated log analysis

[1176] The server periodically collects log data from the device, inputs this log data into an AI analysis engine, and detects abnormal patterns and errors in the log data. The server then generates a detailed report based on the analysis results and provides it to the user.

[1177] Specific examples

[1178] The server collects operation logs from the devices at a set time every day and runs them through an AI analysis engine to detect abnormal patterns and errors. Based on the results, a weekly report is generated and sent to the user by email.

[1179] These features not only enable users to significantly reduce the amount of work required in the device verification process, but also enable highly accurate and rapid defect detection and efficient response. Furthermore, it is expected to contribute to shortening development schedules. This system will be an extremely useful tool for device developers.

[1180] The processing flow will be explained below.

[1181] Automating device configuration

[1182] Step 1:

[1183] The user enters the information for the new device and presses the "Register" button on the device registration screen.

[1184] Step 2:

[1185] The server receives the terminal information sent from the user.

[1186] Step 3:

[1187] The server reads the device profile from the database.

[1188] Step 4:

[1189] The server automatically applies the device's network and security settings based on the profile loaded.

[1190] Step 5:

[1191] The server confirms the status of the configuration.

[1192] Step 6:

[1193] The server will send a notification to the user that the setup is complete.

[1194] Automation of frequency testing

[1195] Step 1:

[1196] The user accesses the test execution screen and enters the test conditions (e.g., execute the Wi-Fi connection test 100 times in succession).

[1197] Step 2:

[1198] The user presses the "Start Test" button.

[1199] Step 3:

[1200] The server receives the input test conditions.

[1201] Step 4:

[1202] The server causes the terminal to execute the test a specified number of times.

[1203] Step 5:

[1204] The server collects data obtained during the test (number of successes, number of failures, connection time, etc.).

[1205] Step 6:

[1206] The server generates reports based on the collected data.

[1207] Step 7:

[1208] The server generates a report and sends it to the user.

[1209] Automating long-term testing

[1210] Step 1:

[1211] The user accesses the long-term test execution screen and inputs the test period (e.g., 24 hours).

[1212] Step 2:

[1213] The user presses the "Start Test" button.

[1214] Step 3:

[1215] The server sets a timer for the test period.

[1216] Step 4:

[1217] The server will periodically monitor the terminal status (e.g., CPU usage, memory usage, network status) during the test period.

[1218] Step 5:

[1219] As soon as the server detects an abnormality, it will notify the user.

[1220] Step 6:

[1221] After the long test is completed, the server reports the test results to the user.

[1222] Automated log analysis

[1223] Step 1:

[1224] The server periodically collects operation logs from the terminal based on a set schedule (e.g., collects logs at midnight every day).

[1225] Step 2:

[1226] The log data collected by the server is input into an AI analysis engine.

[1227] Step 3:

[1228] The server uses an AI analysis engine to detect abnormal patterns and errors in the log data.

[1229] Step 4:

[1230] The server generates a detailed report based on the results of the AI ​​analysis engine.

[1231] Step 5:

[1232] Sending server-generated reports to users (e.g. email or dashboard notifications).

[1233] Example 1

[1234] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1235] The verification process for conventional device development requires manual terminal configuration, test execution, long-term test monitoring, and log data analysis, which consumes a significant amount of time and effort. Additionally, there is a lack of systems for quickly detecting anomalies and taking appropriate action. Under these circumstances, it is difficult to ensure device quality while promoting efficient development.

[1236] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1237] In this invention, the server includes a means for automating terminal configuration, a means for automatically executing multiple tests, a means for periodically monitoring long-term tests, a means for analyzing log data and detecting anomalies, a means for reporting the analysis results, a means for monitoring the terminal's operating status, and a means for notifying the user of completion of configuration. This enables fast and accurate initial terminal configuration, and enables efficient and highly accurate device verification through the automation and monitoring of multiple tests and long-term tests. Furthermore, anomaly detection and user notification promotes rapid response and improves overall development efficiency.

[1238] "Means for automating terminal settings" refers to a function in which a user inputs terminal information into the system, and the server automatically performs network settings, security settings, and installation of necessary applications.

[1239] The "means for automatically executing multiple tests" is a function that allows the server to automatically execute a specified test multiple times based on conditions specified by the user, and collect the results.

[1240] "Means for periodically monitoring long-term testing" refers to a function in which the server periodically monitors the operating status of the terminal (CPU usage, memory usage, network status, etc.) during a period set by the user, and notifies the user if an abnormality occurs.

[1241] "Means for analyzing log data and detecting anomalies" refers to a function that inputs log data collected periodically by the server into an AI analysis engine to detect abnormal patterns and errors.

[1242] "Means for reporting the analysis results" refers to a function in which the server generates a detailed report based on the anomaly detection results obtained by the AI ​​analysis engine and notifies or provides it to the user.

[1243] "Means for monitoring the operating status of the terminal" refers to a function by which the server periodically checks the operating status of the terminal (CPU usage, memory usage, network status, etc.) during the test period or a specific monitoring period.

[1244] The "means for notifying the user of completion of setup" is a function in which the server sends a push notification or email notification to the user when the initial setup of the device is complete.

[1245] The present invention relates to a system for automating processes in the verification process of device development, such as terminal setting, number testing, long-term testing, log analysis, etc. Specific embodiments for carrying out the present invention will be described in detail below.

[1246] Automating device configuration

[1247] When a user registers a new device in the system, the server receives the information and retrieves the device profile from the database. The server then automatically configures the device's network and security settings and installs any necessary applications. Once the configuration is complete, the server sends a notification to the user. The hardware and software used are as follows:

[1248] Hardware: Servers, terminals

[1249] Software: Database, terminal configuration script

[1250] Specific examples

[1251] A user registers a new smartphone in the system. The server reads the smartphone profile and automatically configures Wi-Fi settings, security policies, and installs required applications. After the configuration is complete, the server notifies the user that the smartphone configuration is complete.

[1252] Example prompt sentence:

[1253] Register your new smartphone and automatically apply network settings and security policies.

[1254] Automation of frequency testing

[1255] When the user enters the test conditions (e.g., run a Wi-Fi connection test 100 times in a row) and presses the test start button, the server receives the test conditions and automatically runs the test the specified number of times. The server collects data obtained during the test (number of successes, number of failures, connection time, etc.) and generates a report based on this data. The generated report is sent from the server to the user. The hardware and software used are as follows:

[1256] Hardware: Servers, terminals

[1257] Software: Test management software, data collection module

[1258] Specific examples

[1259] The user sets up 100 Wi-Fi connection tests. The server executes all 100 connection attempts, compiles the results, creates a report, and sends it to the user.

[1260] Example prompt sentence:

[1261] Automatically run 100 Wi-Fi connection tests and compile the results into a report.

[1262] Automating long-term testing

[1263] Once the user sets the test period (e.g., 24 hours) and issues the command to start, the server will continue to monitor the terminal during that period. The server periodically checks the terminal's operating status (CPU usage, memory usage, network status, etc.) and immediately notifies the user if an abnormality occurs. The hardware and software used are as follows:

[1264] Hardware: Servers, terminals

[1265] Software: Monitoring software, alert systems

[1266] Specific examples

[1267] The user sets up a 24-hour continuous use test. The server periodically checks the device's operating status during the 24-hour period, and immediately notifies the user if an abnormality is detected.

[1268] Example prompt sentence:

[1269] Conduct a 24-hour continuous use test and notify us immediately if any abnormalities are detected.

[1270] Automated log analysis

[1271] The server periodically collects log data from the device and inputs it into an AI analysis engine for analysis. The analysis detects abnormal patterns and errors, and generates a detailed report based on the results. The generated report is provided to the user. The hardware and software used are as follows:

[1272] Hardware: Servers, terminals

[1273] Software: Log collection module, AI analysis engine

[1274] Specific examples

[1275] The server collects operation logs from the devices at a set time each day and inputs them into an AI analysis engine to detect abnormal patterns and errors. Based on the results, a weekly report is generated and sent to the user by email.

[1276] Example prompt sentence:

[1277] Collect device log data on a regular daily basis and analyze it with AI to detect abnormal patterns.

[1278] Through the above process, users can significantly reduce the effort required for device verification, and can detect defects quickly and accurately and address them efficiently. This is expected to improve the overall efficiency of device development and enhance quality assurance.

[1279] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1280] Processing flow of the terminal setting automation program

[1281] Step 1:

[1282] The user enters device information and presses the registration button. The user accesses the system interface and enters new device information (e.g., device ID, device name, device model, etc.). When the user presses the "Register" button, the information is sent to the server.

[1283] Input: Basic device information (device ID, device name, device model)

[1284] Output: Sending device information to the server

[1285] Step 2:

[1286] The server receives the device information and retrieves the device profile from the database. The server receives the HTTP request and prepares the received data for internal processing. It then issues an SQL query to the database to retrieve the profile information for the corresponding device.

[1287] Input: Device information sent by the user

[1288] Output: Device profile retrieved from the database

[1289] Step 3:

[1290] The server applies the settings to the device. Based on the device profile obtained by the server, it sends commands to the device to automatically configure network settings, security settings, and install applications.

[1291] Input: Device profile retrieved from the database

[1292] Output: Command to apply the settings to the device

[1293] Step 4:

[1294] The device receives and executes the settings. Based on the setting information received from the server, the device configures the network and security settings and installs the necessary applications.

[1295] Input: Configuration information sent from the server

[1296] Output: Performing configuration tasks

[1297] Step 5:

[1298] The server will send a notification to the user that the setup is complete. After the setup is complete, the server will send a push notification or email to the user to let them know that the setup is complete.

[1299] Input: Device with completed settings

[1300] Output: User notification

[1301] Processing flow of the automated program for number testing

[1302] Step 1:

[1303] The user enters the test conditions and presses the "Start Test" button. The user accesses the test management screen, enters the test conditions (e.g., 100 Wi-Fi connection tests), and presses the "Start" button.

[1304] Input: Test conditions (Wi-Fi connection test 100 times)

[1305] Output: Send test conditions to the server

[1306] Step 2:

[1307] The server receives the test conditions, prepares the test, analyzes the received test conditions, and sets the test schedule.

[1308] Input: Test conditions submitted by the user

[1309] Output: Test schedule settings

[1310] Step 3:

[1311] The server starts the test and sends multiple test execution commands to the terminal based on the specified test conditions.

[1312] Input: Exam Schedule

[1313] Output: Test execution command to terminal

[1314] Step 4:

[1315] The terminal executes the test. Upon receiving commands from the server, the terminal executes the test a specified number of times.

[1316] Input: Test execution command sent from the server

[1317] Output: Test results (number of successes, number of failures, connection time, etc.)

[1318] Step 5:

[1319] The server collects the test results and generates a report. The server collects the data obtained during the test run and generates a report based on that data.

[1320] Input: Test result

[1321] Output: Generate a report

[1322] Step 6:

[1323] The server sends the report to the user. The generated report is emailed to the user.

[1324] Input: Report

[1325] Output: Report sent to user

[1326] Processing flow of the automated program for long-term testing

[1327] Step 1:

[1328] The user sets the test period and issues a command to start the test. The user sets the test period (e.g., 24 hours) and presses the "Start" button.

[1329] Input: Test period (24 hours)

[1330] Output: Send test period to server

[1331] Step 2:

[1332] The server receives the instruction to start the test and begins monitoring. The server then sends a command to periodically monitor the operating status of the terminal during the test period.

[1333] Input: Test period instructions

[1334] Output: Monitor command to terminal

[1335] Step 3:

[1336] The device sends its operating status to the server. Following instructions from the server, the device periodically reports its operating status (CPU usage, memory usage, network status, etc.) to the server.

[1337] Input: Surveillance order

[1338] Output: Operation status data

[1339] Step 4:

[1340] The server analyzes the operational status data and detects abnormalities. The server periodically analyzes the operational status data it receives and detects abnormalities.

[1341] Input: Operational status data

[1342] Output: Anomaly detection results

[1343] Step 5:

[1344] Notify the user when the server detects an abnormality. If an abnormality occurs, the server will immediately send a notification to the user.

[1345] Input: Anomaly detection results

[1346] Output: Anomaly notification to the user

[1347] Log analysis automation program processing flow

[1348] Step 1:

[1349] The server periodically collects log data, and obtains the log data from the device via the logging API.

[1350] Input: Device log data

[1351] Output: Send log data to the server

[1352] Step 2:

[1353] The server inputs the log data into an AI analysis engine, which then inputs the log data collected by the server into the AI ​​analysis engine to detect abnormal patterns and errors.

[1354] Input: Log data

[1355] Output: AI analysis results

[1356] Step 3:

[1357] The server generates a report based on the analysis results. A detailed report is generated based on the anomaly detection results obtained by the AI ​​analysis engine.

[1358] Input: AI analysis results

[1359] Output: Generate a report

[1360] Step 4:

[1361] The server provides the report to the user, either by emailing the generated report to the user or making it available for viewing within the system.

[1362] Input: Report

[1363] Output: Report provided to user

[1364] The above processing steps enable the automation of terminal settings, the automatic execution of multiple tests, the monitoring of long-term tests, and the analysis of log data to be performed consistently and rationally.

[1365] (Application example 1)

[1366] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1367] Conventional methods for configuring, testing, and monitoring industrial equipment and terminals require a great deal of manual work, making them inefficient and prone to human error. Initially configuring new equipment, conducting a large number of tests, and monitoring over long periods of time require significant time and effort. Furthermore, detecting and reporting anomalies is also manual, making it difficult to respond quickly. Therefore, there is a need for efficient management of industrial equipment, improved reliability, and rapid detection and response of anomalies.

[1368] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1369] In this invention, the server includes means for automating terminal configuration, means for automatically executing multiple tests, means for periodically monitoring long-term tests, means for analyzing log data and detecting abnormalities, means for reporting the analysis results, means for reading industrial equipment profiles and automatically applying necessary initial settings, and means for collecting test results and monitoring them in real time. This automates the configuration, testing, monitoring, abnormality detection, and reporting of industrial equipment and terminals, enabling efficient and reliable management.

[1370] "Means for automating terminal settings" refers to a function that automatically performs the initial settings of terminals and industrial equipment without requiring manual operation by the user.

[1371] The "means for automatically executing multiple tests" means a function that automatically executes a certain number of tests on terminals or industrial equipment based on pre-set conditions.

[1372] "Means for periodically monitoring long-term tests" refers to a function that periodically monitors the operating status of terminals and industrial equipment when conducting long-term tests, and records and analyzes the results.

[1373] The "means for analyzing log data and detecting abnormalities" refers to a function that analyzes log data collected from terminals and industrial equipment and detects abnormal operations or errors from the data.

[1374] The "means for reporting the analysis results" has a function for reporting the analysis results of the log data to the user, and notifies the user if an abnormality is detected.

[1375] "Means for reading the profile of industrial equipment and automatically applying the required initial settings" means a function that reads the profile information of industrial equipment and automatically applies the required initial settings (Wi-Fi settings, security policies, etc.) based on that information.

[1376] "Means for collecting test results and monitoring them in real time" means a means for collecting result data obtained during the test and monitoring the data in real time.

[1377] A system for realizing the present invention includes a server, industrial equipment (terminals), and user interaction. Specific embodiments will be described in detail below.

[1378] Hardware and Software

[1379] Hardware:

[1380] Server: A computer system with a powerful processor

[1381] Industrial equipment: Equipment equipped with various sensors and communication modules (e.g., robots, sensor devices)

[1382] Smart glasses: wearable devices for displaying real-time monitoring information (e.g., Google Glass)

[1383] software:

[1384] Server software: Python, real-time data streaming API, database management system

[1385] Log analysis engine: Anomaly detection algorithm using AI models

[1386] User interface: Web browser or dedicated application

[1387] Details of data processing and calculation

[1388] When a user registers an industrial device, the server reads its profile from the database and automatically applies initial settings (e.g., network settings, security policies) to the industrial device. This initial setting process is performed based on the device's profile information.

[1389] When industrial equipment starts a new task, the server runs multiple tests and collects and monitors the results in real time. When the test is completed, the server analyzes the obtained data and notifies the user if an abnormality is detected. This is done using an AI analysis engine for anomaly detection.

[1390] Furthermore, for equipment undergoing long-term testing, its operation is regularly monitored, and the server detects and notifies users of any abnormalities as necessary. All log data is centrally managed on the server and provided to users as reports as needed.

[1391] Specific examples

[1392] This section explains how this system works for robots used in factories. When a user registers a new robot in the system, the server loads the robot's profile and automatically applies initial settings such as Wi-Fi settings and security policies. It then conducts multiple operational tests and 24-hour continuous operation tests, monitoring the robot in real time. During this process, the server collects log data and detects abnormalities using an AI analysis engine. Detected abnormalities are immediately notified to the user, who can view the information through the smart glasses. In this way, users can efficiently manage the robot's operations and quickly respond to any abnormalities that occur.

[1393] Prompt Sentence Examples

[1394] "Please develop a system that monitors robots in factories and notifies them when an abnormality is detected. For example, it can automatically set up Wi-Fi and security policies, and conduct repeated and long-term tests to monitor the results in real time."

[1395] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1396] Step 1:

[1397] A user registers a new industrial device in the system. As input, the user provides device profile information (e.g., device ID, network settings, security policy, etc.). The server receives this information and stores it in a database. As output, the server sends a notification to the user that the registration is complete.

[1398] Step 2:

[1399] The server reads the saved profile information and automatically applies the necessary initial settings to the industrial equipment based on that information. As input, it receives the profile information and executes Wi-Fi settings, security policies, etc. based on that information. As output, it logs that the initial settings were successfully completed.

[1400] Step 3:

[1401] The user configures tests in the system. Specifically, the user specifies multiple tests (e.g., operation tests or connection tests) or long-term tests (e.g., 24-hour continuous operation tests). As input, the user provides test conditions (number of tests and duration). The server receives the test conditions and creates a test plan. As output, a test plan is generated and saved.

[1402] Step 4:

[1403] The server executes the specified tests. It reads the test plan as input and executes each test sequentially. Specifically, for multiple tests, it performs the specified number of attempts, and for long-term tests, it periodically monitors the operation within the specified time. As output, it records the results of each test (success / failure and operation log).

[1404] Step 5:

[1405] The server analyzes the log data collected during and after the test. It receives the collected operation logs and test results as input. It uses an AI model for analysis to detect anomalies. It outputs the analysis results in a report.

[1406] Step 6:

[1407] The server reports the analysis results to the user. As input, it generates a report of anomaly detection and notifies the user. As output, the user receives the report and can get detailed information about the status of industrial equipment.

[1408] Step 7:

[1409] If an anomaly is detected, the server immediately notifies the user. As input, it generates an anomaly notification based on the analysis results and forwards the content to the user. As output, the user receives the notification on a device such as smart glasses, allowing them to quickly address the problem.

[1410] Step 8:

[1411] Users review the maintenance and settings of industrial equipment based on the reports provided by the server. As input, they refer to the information in the report and carry out the necessary procedures. As output, they can confirm the improved settings and maintenance status. This cycle ensures efficient and reliable operation of industrial equipment.

[1412] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1413] This invention relates to a system that uses AI to automate processes such as terminal configuration, number testing, long-term testing, and log analysis in the verification process of device development, and further combines it with an emotion engine that recognizes user emotions. Specific embodiments for carrying out the present invention are described in detail below.

[1414] Automating device configuration

[1415] The user registers a new device in the system. When the user enters the device information and presses the registration button, the server receives this information. The server reads the device profile from the database and automatically applies the initial settings, such as network settings and security settings, to the device. Once the settings are complete, the server sends a notification to the user that the settings are complete.

[1416] Specific examples

[1417] When a user registers a new smartphone in the system, the server reads the smartphone profile and automatically configures Wi-Fi settings, security policies, and installs necessary applications. After the configuration is complete, the server notifies the user that the smartphone configuration is complete.

[1418] Automation of frequency testing

[1419] The user accesses the test execution screen and enters the test conditions (e.g., run the Wi-Fi connection test 100 times in a row). When the user presses the test start button, the server receives the test conditions and causes the device to run the test the specified number of times. The server collects data obtained during the test (number of successes, number of failures, connection time, etc.) and generates a report based on this data. The generated report is sent from the server to the user.

[1420] Specific examples

[1421] If a user configures the system to perform 100 Wi-Fi connection tests, the server will perform all 100 connection attempts, compile the results, create a report, and send it to the user.

[1422] Automating long-term testing

[1423] Once the user issues a command to start the test and sets the test period (e.g., 24 hours), the server will continue to monitor the device during that period. The server periodically checks the device's operating status (CPU usage, memory usage, network status, etc.) and immediately notifies the user if any abnormalities are detected.

[1424] Specific examples

[1425] When a user sets up a 24-hour continuous use test, the server periodically checks the device's operating status for 24 hours and immediately notifies the user if any abnormalities are detected.

[1426] Automated log analysis

[1427] The server periodically collects log data from the device, inputs this log data into an AI analysis engine, and detects abnormal patterns and errors in the log data. The server then generates a detailed report based on the analysis results and provides it to the user.

[1428] Specific examples

[1429] The server collects operation logs from the devices at a set time every day and runs them through an AI analysis engine to detect abnormal patterns and errors. Based on the results, a weekly report is generated and sent to the user by email.

[1430] User Emotion Recognition

[1431] The system also incorporates an emotion engine that recognizes the user's emotions. This emotion engine analyzes emotional data from the user's voice and facial expressions, and uses the results to adjust the system's operations and analyze log data.

[1432] Specific examples

[1433] The emotion engine recognizes emotions such as stress and anxiety felt by users while operating the system. Based on the results, the system makes suggestions to improve the user experience and analyzes log data by adding emotion data. For example, if a specific operation causes stress over a long period of time, the system will make suggestions to improve that operation method or the system interface.

[1434] Combining these elements will enable the device verification process to be more efficient, defects to be detected and addressed early, and user experience to be improved. This system will serve as a very useful tool for developers.

[1435] The processing flow will be explained below.

[1436] Automating device configuration

[1437] Step 1:

[1438] The user enters the information for the new device and presses the "Register" button on the device registration screen.

[1439] Step 2:

[1440] The server receives the terminal information sent from the user.

[1441] Step 3:

[1442] The server reads the device profile from the database.

[1443] Step 4:

[1444] The server automatically applies the device's network and security settings based on the profile loaded.

[1445] Step 5:

[1446] The server confirms the status of the configuration.

[1447] Step 6:

[1448] The server will send a notification to the user that the setup is complete.

[1449] Automation of frequency testing

[1450] Step 1:

[1451] The user accesses the test execution screen and enters the test conditions (e.g., execute the Wi-Fi connection test 100 times in succession).

[1452] Step 2:

[1453] The user presses the "Start Test" button.

[1454] Step 3:

[1455] The server receives the input test conditions.

[1456] Step 4:

[1457] The server causes the terminal to execute the test a specified number of times.

[1458] Step 5:

[1459] The server collects data obtained during the test (number of successes, number of failures, connection time, etc.).

[1460] Step 6:

[1461] The server generates reports based on the collected data.

[1462] Step 7:

[1463] The server generates a report and sends it to the user.

[1464] Automating long-term testing

[1465] Step 1:

[1466] The user accesses the long-term test execution screen and inputs the test period (e.g., 24 hours).

[1467] Step 2:

[1468] The user presses the "Start Test" button.

[1469] Step 3:

[1470] The server sets a timer for the test period.

[1471] Step 4:

[1472] The server will periodically monitor the terminal status (e.g., CPU usage, memory usage, network status) during the test period.

[1473] Step 5:

[1474] As soon as the server detects an abnormality, it will notify the user.

[1475] Step 6:

[1476] After the long test is completed, the server reports the test results to the user.

[1477] Automated log analysis

[1478] Step 1:

[1479] The server periodically collects operation logs from the terminal based on a set schedule (e.g., collects logs at midnight every day).

[1480] Step 2:

[1481] The log data collected by the server is input into an AI analysis engine.

[1482] Step 3:

[1483] The server uses an AI analysis engine to detect abnormal patterns and errors in the log data.

[1484] Step 4:

[1485] The server generates a detailed report based on the results of the AI ​​analysis engine.

[1486] Step 5:

[1487] Sending server-generated reports to users (e.g. email or dashboard notifications).

[1488] User Emotion Recognition

[1489] Step 1:

[1490] The user provides voice and facial expression data while operating the system (e.g., using a webcam or microphone).

[1491] Step 2:

[1492] The server receives the user's voice and facial expression data using an emotion engine.

[1493] Step 3:

[1494] The server uses an emotion engine to analyze the user's emotions.

[1495] Step 4:

[1496] The server receives the emotion data and adjusts the system's behavior based on the analysis results.

[1497] Step 5:

[1498] The server integrates the analyzed emotion data with the log data and performs further detailed analysis.

[1499] Step 6:

[1500] The server generates a report based on the user's emotional data to improve the user experience.

[1501] Specific examples

[1502] Example of automating device settings

[1503] The user enters the information to register a new smartphone and presses the "Register" button. The server receives the information, reads the smartphone profile from the database, and automatically applies the necessary network and security settings. It then sends a notification to the user that the setup is complete.

[1504] Automation of frequency testing

[1505] When a user sets up 100 Wi-Fi connection tests, the server receives the test conditions and performs 100 Wi-Fi connection tests on the device. The server collects success and failure data during the test and finally generates a report and sends it to the user.

[1506] Automating long-term testing

[1507] When a user sets up a 24-hour continuous use test, the server periodically monitors the terminal's operating status for 24 hours and notifies the user if any abnormalities occur.

[1508] Automated log analysis

[1509] The server collects log data from devices at a set time each day and runs it through an AI analysis engine to detect abnormal patterns. The analysis results are generated as a detailed report and sent to the user.

[1510] User Emotion Recognition

[1511] As a user operates the system, the emotion engine analyzes the user's emotions from voice and facial expression data. Based on the analysis results, the system adjusts system operation and makes suggestions to reduce the user's stress and anxiety. The emotion data is also integrated with log data for more detailed analysis, improving the user experience.

[1512] Example 2

[1513] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1514] The verification process for device development involves extremely time-consuming processes such as terminal configuration, repeated testing, long-term testing, and log analysis, which consumes a large amount of time and human resources.Furthermore, it is difficult to improve the user experience because the system cannot be adjusted to take user emotions into account.

[1515] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1516] In this invention, the server includes means for automating terminal configuration, means for recognizing user emotions, means for automatically executing multiple tests, means for periodically monitoring long-term tests, means for analyzing log data and detecting anomalies, means for reporting the analysis results, and means for adjusting system operation based on user emotion data, thereby enabling more efficient device verification processes, early detection and handling of defects, and an improved user experience.

[1517] A "means for automating terminal settings" is a means by which a server automatically performs network settings, security settings, and installation of necessary applications based on terminal information entered by the user.

[1518] The "means for automatically executing a test multiple times" is a means for the server to automatically execute a test on the terminal a specified number of times based on test conditions specified by the user.

[1519] The "means for periodically monitoring the long-term test" refers to a means for the server to periodically monitor the operating status of the terminal during the specified test period and to immediately notify the user if an abnormality occurs.

[1520] "Means for analyzing log data and detecting anomalies" refers to the means by which the server inputs log data acquired from the terminal into an AI analysis engine to detect abnormal patterns and errors.

[1521] The "means of reporting the analysis results" refers to the means of generating a detailed report based on the analysis results of the AI ​​analysis engine and providing it to the user.

[1522] The "means for recognizing the user's emotions" refers to a means by which the system acquires the user's voice and facial expressions and analyzes them using an emotion engine to recognize the user's emotional state.

[1523] The "means for adjusting system operation based on user emotion data" refers to a means for the system to improve operability and adjust the interface based on the recognized user emotion data.

[1524] "Means for collecting log data from terminals" refers to the means by which the server periodically obtains operation logs, system logs, application logs, etc. from terminals.

[1525] The present invention relates to a system that automates multiple processes in the verification process of device development and further recognizes user emotions.

[1526] Automating device configuration

[1527] When a user registers a new device in the system, the server receives the device information and automatically applies initial settings to the device, such as network settings, security settings, and installation of required applications. The server uses an API to perform these settings remotely. For example, when a user registers a smartphone, the server automatically configures Wi-Fi settings, security policies, and installs required applications. After the configuration is complete, the server notifies the user that the configuration was successful.

[1528] Automation of frequency testing

[1529] Based on the test conditions specified by the user, the server will perform the test on the device the specified number of times. For example, if you set the Wi-Fi connection test to be performed 100 times, the server will perform 100 connection attempts and compile the results. The server will collect data such as the number of successes, the number of failures, and the connection time, and generate a detailed report to send to the user.

[1530] Automating long-term testing

[1531] Once the user sets the test period, the server will periodically monitor the device during that period. For example, if a 24-hour long test is set, the server will periodically check the device's operating status (CPU usage, memory usage, network status, etc.). If an abnormality is detected, the server will immediately notify the user.

[1532] Automated log analysis

[1533] The server periodically collects log data from devices. This data is input into an AI analysis engine to detect abnormal patterns and errors. Based on the results of the AI ​​analysis engine, the server generates a detailed report and provides it to the user. For example, the server could collect operation logs from devices at a set time every day, run them through the AI ​​analysis engine to detect abnormalities, and send a weekly report to the user by email.

[1534] User Emotion Recognition

[1535] This system incorporates an emotion engine that recognizes emotions from the user's voice and facial expressions. The emotion engine analyzes the user's emotional data and uses the results to adjust the system's operation. For example, if a user feels stressed while operating the system, the emotion is recognized and the system suggests improvements to operability and interface adjustments. Emotion data is also integrated with log data for comprehensive analysis.

[1536] Examples of specific prompts include:

[1537] Register a new device and automatically install Wi-Fi settings and necessary apps.

[1538] "Please perform 100 Wi-Fi connection tests and submit the results as a report."

[1539] "Please conduct a 24-hour continuous use test and notify us immediately if any abnormalities occur."

[1540] "Analyze the device operation logs and prepare a weekly report if any abnormalities are detected."

[1541] "Please be aware of the user's stress level and provide suggestions for improving usability."

[1542] The system of the present invention is expected to improve the efficiency of the device verification process, enable early detection of defects, and improve the user experience.

[1543] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1544] System program processing flow

[1545] Automating device configuration

[1546] Step 1:

[1547] The user enters the information for the new device and clicks the registration button.

[1548] Specific operation: The user inputs information such as the device model name, serial number, and network settings. The input data is sent to the server.

[1549] Step 2:

[1550] The server receives the terminal information and temporarily stores it in a database.

[1551] Input: Device information (model name, serial number, network settings, etc.)

[1552] Data processing: Converting device information into the appropriate format and storing it in the database

[1553] Output: Saved device information

[1554] Step 3:

[1555] The server loads the device profile.

[1556] Input: Saved device information

[1557] Data calculation: Obtain the corresponding device profile (initial setting information) from the database

[1558] Output:Device profile

[1559] Step 4:

[1560] The server applies the initial settings to the device.

[1561] Input:Device profile

[1562] Specific operation: Automatically configures network settings (e.g. Wi-Fi settings), security settings, and installs required applications.

[1563] Output: Device with initial setup completed

[1564] Step 5:

[1565] The server will notify you that the settings are complete.

[1566] Input: Device information for which initial settings have been completed

[1567] Data processing: Generate notification messages and prepare them for sending

[1568] Output: Notification to the user that the settings have been completed

[1569] Automation of frequency testing

[1570] Step 1:

[1571] The user inputs the test conditions and clicks the test start button.

[1572] Specific operation: The user registers the test conditions (e.g., execute the Wi-Fi connection test 100 times). The input data is sent to the server.

[1573] Step 2:

[1574] The server receives the test conditions and stores them in a database.

[1575] Input: Test conditions (e.g., run Wi-Fi connection test 100 times)

[1576] Data processing: Convert test conditions into an appropriate format and save it in a database

[1577] Output: Saved test conditions

[1578] Step 3:

[1579] The server instructs the terminal to execute the test.

[1580] Input: Saved test conditions

[1581] Specific operation: The server sends a test execution command to the terminal.

[1582] Output: Test execution started

[1583] Step 4:

[1584] The server collects the test data.

[1585] Input: Test execution data (number of successes, number of failures, connection time, etc.)

[1586] Data calculation: The data obtained during the test is compiled and stored in a database.

[1587] Output: Collected test data

[1588] Step 5:

[1589] The server generates the report.

[1590] Input: Collected test data

[1591] Data Processing: Generate reports based on test results

[1592] Output: Generated report

[1593] Step 6:

[1594] The server sends the report to the user.

[1595] Input: Generated report

[1596] Specific behavior: Send the report to the user via email or system notification

[1597] Output: Reports test results to the user

[1598] Automating long-term testing

[1599] Step 1:

[1600] The user sets the test period and issues an instruction to start the test.

[1601] Specific operation: The user inputs the test period (e.g., 24 hours) and sends it to the server.

[1602] Step 2:

[1603] The server monitors the device during the test period.

[1604] Input: Test period setting information

[1605] Specific operation: The server periodically monitors the device's operating status during the set period.

[1606] Output: Monitoring data

[1607] Step 3:

[1608] The server checks the operation status.

[1609] Input: Monitoring data

[1610] Data calculation: Analyze the device's operating status (e.g., CPU usage, memory usage, network status)

[1611] Output: Analysis results of the operating status

[1612] Step 4:

[1613] The server detects an anomaly.

[1614] Input: Analysis results of operating conditions

[1615] Data arithmetic: Detecting abnormal patterns and errors

[1616] Output: Anomaly detection results

[1617] Step 5:

[1618] The server notifies the user of the abnormality.

[1619] Input: Anomaly detection results

[1620] Specific behavior: Generates a notification message and sends it to the user

[1621] Output: Abnormal notification to the user

[1622] Automated log analysis

[1623] Step 1:

[1624] The server collects log data from the terminal.

[1625] Specific operation: Periodically obtain system logs, operation logs, and application logs from the terminal.

[1626] Step 2:

[1627] The server inputs the log data into an AI analysis engine.

[1628] Input: Collected log data

[1629] Data processing: Converting collected log data into a format suitable for the AI ​​analysis engine

[1630] Output: Data input into the AI ​​analysis engine

[1631] Step 3:

[1632] An AI analysis engine analyzes the log data.

[1633] Input: Data entered into the AI ​​analysis engine

[1634] Data Calculation: Analyzing abnormal patterns and errors

[1635] Output: Analysis results

[1636] Step 4:

[1637] The server receives the analysis results and generates a detailed report.

[1638] Input: Analysis results

[1639] Data processing: Generate reports based on analysis results

[1640] Output: Generated report

[1641] Step 5:

[1642] The server provides the report to the user.

[1643] Input: Generated report

[1644] Specific behavior: Send the report to the user via email or system notification

[1645] Output: Regular reports to users

[1646] User Emotion Recognition

[1647] Step 1:

[1648] The system captures the user's voice and facial expressions.

[1649] Specific operation: Collects user voice and facial expression data using a webcam and microphone.

[1650] Step 2:

[1651] The emotion engine analyzes the emotion data.

[1652] Input: Collected voice and facial expression data

[1653] Data Calculation: The emotion engine analyzes the data and determines the user's emotional state.

[1654] Output: Emotion analysis results

[1655] Step 3:

[1656] The system receives the analysis results and adjusts its operations.

[1657] Input: Sentiment analysis results

[1658] Specific actions: Based on the analysis results, we suggest improvements to usability and interface adjustments.

[1659] Output: Suggestions for improving the user experience

[1660] Step 4:

[1661] The system uses emotional data to help with log analysis.

[1662] Input: Sentiment analysis results

[1663] Data calculation: Emotion data is integrated with log data for comprehensive analysis.

[1664] Output: Comprehensive analysis results

[1665] (Application example 2)

[1666] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1667] Conventional factory robot maintenance and testing processes require a lot of manual work and time, making them inefficient and limiting their ability to detect anomalies and improve the user experience. In particular, there is a lack of means to reduce the mental burden on workers.

[1668] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for automating terminal configuration, means for automatically executing multiple tests, means for periodically monitoring long-term tests, means for analyzing log data and detecting abnormalities, means for reporting the analysis results, means for performing emotion recognition and acquiring user emotion data, and means for adjusting system operation and proposing improvements based on the emotion data. This makes it possible to automate and streamline the maintenance and verification processes of factory robots, and further reduce the mental burden on workers.

[1669] A "means for automating device settings" is a means by which, when a user registers a new device in the system, the server reads the device's profile and automatically applies initial settings such as network settings and security settings.

[1670] The "means for automatically executing multiple tests" refers to a means for the server to automatically execute a set number of tests on the corresponding terminal based on test conditions specified by the user, and to tally up the test results.

[1671] The "means for periodically monitoring a long-term test" is a means for the server to periodically monitor the operating status of the terminal during the test period set by the user, and to immediately notify the user if an abnormality occurs.

[1672] "Means for analyzing log data and detecting anomalies" refers to a means for inputting log data collected from terminals into an AI analysis engine to automatically detect abnormal patterns and errors.

[1673] The "means for reporting the analysis results" is a means for generating a detailed report based on the analysis results of the log data and providing the report to the user.

[1674] The "means for performing emotion recognition and acquiring user emotion data" refers to means including an emotion engine for analyzing emotion data from the user's voice and facial expressions and acquiring that data.

[1675] "Means for adjusting system operation and proposing improvements based on emotional data" refers to means for adjusting system operation and proposing improvements to the system's operation method or interface based on the stress or anxiety felt by the user.

[1676] The present invention aims to automate and improve the efficiency of maintenance and inspection processes for factory robots. Specific embodiments for carrying out the present invention will be described in detail below.

[1677] System Configuration

[1678] The system includes the following major components:

[1679] Server: The core device that processes and analyzes data.

[1680] Factory robot: An automated device that performs tasks on a manufacturing line.

[1681] Sensor: A device that detects the robot's operating status, temperature, sound, etc.

[1682] Emotion engine: A system that performs speech analysis (e.g., IBM Watson Speech to Text).

[1683] AI analytics engine: Machine learning models (e.g., TensorFlow) that perform anomaly detection.

[1684] System functions and their operation

[1685] 1. Automate device configuration:

[1686] When a user registers a new factory robot in the system, the server reads the robot's profile and automatically applies initial settings such as network settings and security settings. Once the configuration is complete, the server sends a notification to the user.

[1687] 2. Automating multiple tests:

[1688] When the user inputs test conditions (e.g., repeat a specific action 100 times), the server automatically executes the test based on the conditions on the robot, collates and analyzes the results, and provides a report of the test results to the user.

[1689] 3. Regular monitoring of long-term tests:

[1690] Once the user sets the duration of the long-term test, the server periodically monitors the robot's operation during that period. If an abnormality is detected, the server immediately notifies the user.

[1691] 4. Log data analysis and anomaly detection:

[1692] The server inputs the log data collected from the robot into an AI analysis engine, which detects abnormal patterns and errors. Based on the analysis results, a detailed report is generated and provided to the user.

[1693] 5. User Emotion Recognition:

[1694] An emotion engine is built into the server to collect emotional data from the voice and facial expressions of the worker while the robot is operating. If the user feels anxious or stressed, that emotional data is reflected in the system, which then adjusts the robot's behavior and suggests improvements.

[1695] Adding specific examples

[1696] For example, if a user sets up a 24-hour continuous use test, the server will periodically check the robot's operating status for 24 hours. If an abnormality is detected, the user will be notified immediately. Also, if a worker complains that the robot is "slow," the emotion engine will detect this stress and prioritize log analysis of that area.

[1697] Example of input prompt for generative AI model

[1698] Collect operation logs of robots that operate continuously for 24 hours and use an AI analysis engine to detect abnormal patterns. Also, analyze the voice data of workers and prioritize areas where they feel stress or anxiety in the log analysis and report them.

[1699] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1700] Step 1:

[1701] The server receives a registration request from the user for a new factory robot. The information entered includes the robot's profile data (e.g., model number, work profile). The server uses this information to automatically perform initial configuration such as network settings and security settings. After configuration is complete, the server sends a notification to the user, allowing them to confirm that the robot has been successfully added to the system.

[1702] Step 2:

[1703] The user inputs multiple test conditions into the server. The test conditions include the number of times a specific operation is to be repeated. Based on the conditions, the server sends commands to the robot to execute the test and collects the test execution results. The server aggregates the collected data and calculates the number of successes and failures. Finally, the server provides the test results to the user as a report.

[1704] Step 3:

[1705] The user sets a long-term test period. During that period, the server periodically sends commands to the robot to monitor its operating status (e.g., CPU usage, temperature). The server analyzes the operating data received from the robot, and if an abnormality is detected, it immediately sends a notification email to the user. This allows the user to detect and respond to abnormalities early.

[1706] Step 4:

[1707] The server collects log data from the robots daily or periodically. This log data is input into an AI analysis engine to detect abnormal patterns and errors. The server then generates a detailed report based on the data and provides it to the user, allowing the user to quickly identify potential problems and take measures.

[1708] Step 5:

[1709] While the robot is operating, the emotion engine analyzes the worker's voice and facial expressions in real time. The input voice data is passed through the emotion engine and output as emotional data. The server uses this emotional data to detect the user's stress or anxiety, and based on the results, adjusts the system's operation and suggests improvements. This reduces the mental burden on the worker and improves work efficiency.

[1710] For example, if a user sets up a 24-hour continuous use test, the server will monitor the robot 24 hours a day and immediately notify the user if any abnormalities are detected. Also, if a worker points out that the robot is slow, the emotion engine will detect stress and prioritize analysis of related logs, helping to resolve the problem quickly.

[1711] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1712] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1713] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1714] [Fourth embodiment]

[1715] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1716] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1717] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1718] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1719] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1720] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1721] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1722] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1723] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1724] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1725] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1726] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1727] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1728] The present invention relates to a system for automating processes in the verification process of device development, such as terminal setting, number testing, long-term testing, log analysis, etc. Specific embodiments for carrying out the present invention will be described in detail below.

[1729] Automating device configuration

[1730] The user registers a new device in the system. When the user enters the device information and presses the registration button, the server receives this information. The server reads the device profile from the database and automatically applies the initial settings, such as network settings and security settings, to the device. Once the settings are complete, the server sends a notification to the user that the settings are complete.

[1731] Specific examples

[1732] When a user registers a new smartphone in the system, the server reads the smartphone profile and automatically configures Wi-Fi settings, security policies, and installs necessary applications. After the configuration is complete, the server notifies the user that the smartphone configuration is complete.

[1733] Automation of frequency testing

[1734] The user accesses the test execution screen and enters the test conditions (e.g., run the Wi-Fi connection test 100 times in a row). When the user presses the test start button, the server receives the test conditions and causes the device to run the test the specified number of times. The server collects data obtained during the test (number of successes, number of failures, connection time, etc.) and generates a report based on this data. The generated report is sent from the server to the user.

[1735] Specific examples

[1736] If a user configures the system to perform 100 Wi-Fi connection tests, the server will perform all 100 connection attempts, compile the results, create a report, and send it to the user.

[1737] Automating long-term testing

[1738] Once the user issues a command to start the test and sets the test period (e.g., 24 hours), the server will continue to monitor the device during that period. The server periodically checks the device's operating status (CPU usage, memory usage, network status, etc.) and immediately notifies the user if any abnormalities are detected.

[1739] Specific examples

[1740] When a user sets up a 24-hour continuous use test, the server periodically checks the device's operating status for 24 hours and immediately notifies the user if any abnormalities are detected.

[1741] Automated log analysis

[1742] The server periodically collects log data from the device, inputs this log data into an AI analysis engine, and detects abnormal patterns and errors in the log data. The server then generates a detailed report based on the analysis results and provides it to the user.

[1743] Specific examples

[1744] The server collects operation logs from the devices at a set time every day and runs them through an AI analysis engine to detect abnormal patterns and errors. Based on the results, a weekly report is generated and sent to the user by email.

[1745] These features not only enable users to significantly reduce the amount of work required in the device verification process, but also enable highly accurate and rapid defect detection and efficient response. Furthermore, it is expected to contribute to shortening development schedules. This system will be an extremely useful tool for device developers.

[1746] The processing flow will be explained below.

[1747] Automating device configuration

[1748] Step 1:

[1749] The user enters the information for the new device and presses the "Register" button on the device registration screen.

[1750] Step 2:

[1751] The server receives the terminal information sent from the user.

[1752] Step 3:

[1753] The server reads the device profile from the database.

[1754] Step 4:

[1755] The server automatically applies the device's network and security settings based on the profile loaded.

[1756] Step 5:

[1757] The server confirms the status of the configuration.

[1758] Step 6:

[1759] The server will send a notification to the user that the setup is complete.

[1760] Automation of frequency testing

[1761] Step 1:

[1762] The user accesses the test execution screen and enters the test conditions (e.g., execute the Wi-Fi connection test 100 times in succession).

[1763] Step 2:

[1764] The user presses the "Start Test" button.

[1765] Step 3:

[1766] The server receives the input test conditions.

[1767] Step 4:

[1768] The server causes the terminal to execute the test a specified number of times.

[1769] Step 5:

[1770] The server collects data obtained during the test (number of successes, number of failures, connection time, etc.).

[1771] Step 6:

[1772] The server generates reports based on the collected data.

[1773] Step 7:

[1774] The server generates a report and sends it to the user.

[1775] Automating long-term testing

[1776] Step 1:

[1777] The user accesses the long-term test execution screen and inputs the test period (e.g., 24 hours).

[1778] Step 2:

[1779] The user presses the "Start Test" button.

[1780] Step 3:

[1781] The server sets a timer for the test period.

[1782] Step 4:

[1783] The server will periodically monitor the terminal status (e.g., CPU usage, memory usage, network status) during the test period.

[1784] Step 5:

[1785] As soon as the server detects an abnormality, it will notify the user.

[1786] Step 6:

[1787] After the long test is completed, the server reports the test results to the user.

[1788] Automated log analysis

[1789] Step 1:

[1790] The server periodically collects operation logs from the terminal based on a set schedule (e.g., collects logs at midnight every day).

[1791] Step 2:

[1792] The log data collected by the server is input into an AI analysis engine.

[1793] Step 3:

[1794] The server uses an AI analysis engine to detect abnormal patterns and errors in the log data.

[1795] Step 4:

[1796] The server generates a detailed report based on the results of the AI ​​analysis engine.

[1797] Step 5:

[1798] Sending server-generated reports to users (e.g. email or dashboard notifications).

[1799] Example 1

[1800] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1801] The verification process for conventional device development requires manual terminal configuration, test execution, long-term test monitoring, and log data analysis, which consumes a significant amount of time and effort. Additionally, there is a lack of systems for quickly detecting anomalies and taking appropriate action. Under these circumstances, it is difficult to ensure device quality while promoting efficient development.

[1802] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1803] In this invention, the server includes a means for automating terminal configuration, a means for automatically executing multiple tests, a means for periodically monitoring long-term tests, a means for analyzing log data and detecting anomalies, a means for reporting the analysis results, a means for monitoring the terminal's operating status, and a means for notifying the user of completion of configuration. This enables fast and accurate initial terminal configuration, and enables efficient and highly accurate device verification through the automation and monitoring of multiple tests and long-term tests. Furthermore, anomaly detection and user notification promotes rapid response and improves overall development efficiency.

[1804] "Means for automating terminal settings" refers to a function in which a user inputs terminal information into the system, and the server automatically performs network settings, security settings, and installation of necessary applications.

[1805] The "means for automatically executing multiple tests" is a function that allows the server to automatically execute a specified test multiple times based on conditions specified by the user, and collect the results.

[1806] "Means for periodically monitoring long-term testing" refers to a function in which the server periodically monitors the operating status of the terminal (CPU usage, memory usage, network status, etc.) during a period set by the user, and notifies the user if an abnormality occurs.

[1807] "Means for analyzing log data and detecting anomalies" refers to a function that inputs log data collected periodically by the server into an AI analysis engine to detect abnormal patterns and errors.

[1808] "Means for reporting the analysis results" refers to a function in which the server generates a detailed report based on the anomaly detection results obtained by the AI ​​analysis engine and notifies or provides it to the user.

[1809] "Means for monitoring the operating status of the terminal" refers to a function by which the server periodically checks the operating status of the terminal (CPU usage, memory usage, network status, etc.) during the test period or a specific monitoring period.

[1810] The "means for notifying the user of completion of setup" is a function in which the server sends a push notification or email notification to the user when the initial setup of the device is complete.

[1811] The present invention relates to a system for automating processes in the verification process of device development, such as terminal setting, number testing, long-term testing, log analysis, etc. Specific embodiments for carrying out the present invention will be described in detail below.

[1812] Automating device configuration

[1813] When a user registers a new device in the system, the server receives the information and retrieves the device profile from the database. The server then automatically configures the device's network and security settings and installs any necessary applications. Once the configuration is complete, the server sends a notification to the user. The hardware and software used are as follows:

[1814] Hardware: Servers, terminals

[1815] Software: Database, terminal configuration script

[1816] Specific examples

[1817] A user registers a new smartphone in the system. The server reads the smartphone profile and automatically configures Wi-Fi settings, security policies, and installs required applications. After the configuration is complete, the server notifies the user that the smartphone configuration is complete.

[1818] Example prompt sentence:

[1819] Register your new smartphone and automatically apply network settings and security policies.

[1820] Automation of frequency testing

[1821] When the user enters the test conditions (e.g., run a Wi-Fi connection test 100 times in a row) and presses the test start button, the server receives the test conditions and automatically runs the test the specified number of times. The server collects data obtained during the test (number of successes, number of failures, connection time, etc.) and generates a report based on this data. The generated report is sent from the server to the user. The hardware and software used are as follows:

[1822] Hardware: Servers, terminals

[1823] Software: Test management software, data collection module

[1824] Specific examples

[1825] The user sets up 100 Wi-Fi connection tests. The server executes all 100 connection attempts, compiles the results, creates a report, and sends it to the user.

[1826] Example prompt sentence:

[1827] Automatically run 100 Wi-Fi connection tests and compile the results into a report.

[1828] Automating long-term testing

[1829] Once the user sets the test period (e.g., 24 hours) and issues the command to start, the server will continue to monitor the terminal during that period. The server periodically checks the terminal's operating status (CPU usage, memory usage, network status, etc.) and immediately notifies the user if an abnormality occurs. The hardware and software used are as follows:

[1830] Hardware: Servers, terminals

[1831] Software: Monitoring software, alert systems

[1832] Specific examples

[1833] The user sets up a 24-hour continuous use test. The server periodically checks the device's operating status during the 24-hour period, and immediately notifies the user if an abnormality is detected.

[1834] Example prompt sentence:

[1835] Conduct a 24-hour continuous use test and notify us immediately if any abnormalities are detected.

[1836] Automated log analysis

[1837] The server periodically collects log data from the device and inputs it into an AI analysis engine for analysis. The analysis detects abnormal patterns and errors, and generates a detailed report based on the results. The generated report is provided to the user. The hardware and software used are as follows:

[1838] Hardware: Servers, terminals

[1839] Software: Log collection module, AI analysis engine

[1840] Specific examples

[1841] The server collects operation logs from the devices at a set time each day and inputs them into an AI analysis engine to detect abnormal patterns and errors. Based on the results, a weekly report is generated and sent to the user by email.

[1842] Example prompt sentence:

[1843] Collect device log data on a regular daily basis and analyze it with AI to detect abnormal patterns.

[1844] Through the above process, users can significantly reduce the effort required for device verification, and can detect defects quickly and accurately and address them efficiently. This is expected to improve the overall efficiency of device development and enhance quality assurance.

[1845] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1846] Processing flow of the terminal setting automation program

[1847] Step 1:

[1848] The user enters device information and presses the registration button. The user accesses the system interface and enters new device information (e.g., device ID, device name, device model, etc.). When the user presses the "Register" button, the information is sent to the server.

[1849] Input: Basic device information (device ID, device name, device model)

[1850] Output: Sending device information to the server

[1851] Step 2:

[1852] The server receives the device information and retrieves the device profile from the database. The server receives the HTTP request and prepares the received data for internal processing. It then issues an SQL query to the database to retrieve the profile information for the corresponding device.

[1853] Input: Device information sent by the user

[1854] Output: Device profile retrieved from the database

[1855] Step 3:

[1856] The server applies the settings to the device. Based on the device profile obtained by the server, it sends commands to the device to automatically configure network settings, security settings, and install applications.

[1857] Input: Device profile retrieved from the database

[1858] Output: Command to apply the settings to the device

[1859] Step 4:

[1860] The device receives and executes the settings. Based on the setting information received from the server, the device configures the network and security settings and installs the necessary applications.

[1861] Input: Configuration information sent from the server

[1862] Output: Performing configuration tasks

[1863] Step 5:

[1864] The server will send a notification to the user that the setup is complete. After the setup is complete, the server will send a push notification or email to the user to let them know that the setup is complete.

[1865] Input: Device with completed settings

[1866] Output: User notification

[1867] Processing flow of the automated program for number testing

[1868] Step 1:

[1869] The user enters the test conditions and presses the "Start Test" button. The user accesses the test management screen, enters the test conditions (e.g., 100 Wi-Fi connection tests), and presses the "Start" button.

[1870] Input: Test conditions (Wi-Fi connection test 100 times)

[1871] Output: Send test conditions to the server

[1872] Step 2:

[1873] The server receives the test conditions, prepares the test, analyzes the received test conditions, and sets the test schedule.

[1874] Input: Test conditions submitted by the user

[1875] Output: Test schedule settings

[1876] Step 3:

[1877] The server starts the test and sends multiple test execution commands to the terminal based on the specified test conditions.

[1878] Input: Exam Schedule

[1879] Output: Test execution command to terminal

[1880] Step 4:

[1881] The terminal executes the test. Upon receiving commands from the server, the terminal executes the test a specified number of times.

[1882] Input: Test execution command sent from the server

[1883] Output: Test results (number of successes, number of failures, connection time, etc.)

[1884] Step 5:

[1885] The server collects the test results and generates a report. The server collects the data obtained during the test run and generates a report based on that data.

[1886] Input: Test result

[1887] Output: Generate a report

[1888] Step 6:

[1889] The server sends the report to the user. The generated report is emailed to the user.

[1890] Input: Report

[1891] Output: Report sent to user

[1892] Processing flow of the automated program for long-term testing

[1893] Step 1:

[1894] The user sets the test period and issues a command to start the test. The user sets the test period (e.g., 24 hours) and presses the "Start" button.

[1895] Input: Test period (24 hours)

[1896] Output: Send test period to server

[1897] Step 2:

[1898] The server receives the instruction to start the test and begins monitoring. The server then sends a command to periodically monitor the operating status of the terminal during the test period.

[1899] Input: Test period instructions

[1900] Output: Monitor command to terminal

[1901] Step 3:

[1902] The device sends its operating status to the server. Following instructions from the server, the device periodically reports its operating status (CPU usage, memory usage, network status, etc.) to the server.

[1903] Input: Surveillance order

[1904] Output: Operation status data

[1905] Step 4:

[1906] The server analyzes the operational status data and detects abnormalities. The server periodically analyzes the operational status data it receives and detects abnormalities.

[1907] Input: Operational status data

[1908] Output: Anomaly detection results

[1909] Step 5:

[1910] Notify the user when the server detects an abnormality. If an abnormality occurs, the server will immediately send a notification to the user.

[1911] Input: Anomaly detection results

[1912] Output: Anomaly notification to the user

[1913] Log analysis automation program processing flow

[1914] Step 1:

[1915] The server periodically collects log data, and obtains the log data from the device via the logging API.

[1916] Input: Device log data

[1917] Output: Send log data to the server

[1918] Step 2:

[1919] The server inputs the log data into an AI analysis engine, which then inputs the log data collected by the server into the AI ​​analysis engine to detect abnormal patterns and errors.

[1920] Input: Log data

[1921] Output: AI analysis results

[1922] Step 3:

[1923] The server generates a report based on the analysis results. A detailed report is generated based on the anomaly detection results obtained by the AI ​​analysis engine.

[1924] Input: AI analysis results

[1925] Output: Generate a report

[1926] Step 4:

[1927] The server provides the report to the user, either by emailing the generated report to the user or making it available for viewing within the system.

[1928] Input: Report

[1929] Output: Report provided to user

[1930] The above processing steps enable the automation of terminal settings, the automatic execution of multiple tests, the monitoring of long-term tests, and the analysis of log data to be performed consistently and rationally.

[1931] (Application example 1)

[1932] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1933] Conventional methods for configuring, testing, and monitoring industrial equipment and terminals require a great deal of manual work, making them inefficient and prone to human error. Initially configuring new equipment, conducting a large number of tests, and monitoring over long periods of time require significant time and effort. Furthermore, detecting and reporting anomalies is also manual, making it difficult to respond quickly. Therefore, there is a need for efficient management of industrial equipment, improved reliability, and rapid detection and response of anomalies.

[1934] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1935] In this invention, the server includes means for automating terminal configuration, means for automatically executing multiple tests, means for periodically monitoring long-term tests, means for analyzing log data and detecting abnormalities, means for reporting the analysis results, means for reading industrial equipment profiles and automatically applying necessary initial settings, and means for collecting test results and monitoring them in real time. This automates the configuration, testing, monitoring, abnormality detection, and reporting of industrial equipment and terminals, enabling efficient and reliable management.

[1936] "Means for automating terminal settings" refers to a function that automatically performs the initial settings of terminals and industrial equipment without requiring manual operation by the user.

[1937] The "means for automatically executing multiple tests" means a function that automatically executes a certain number of tests on terminals or industrial equipment based on pre-set conditions.

[1938] "Means for periodically monitoring long-term tests" refers to a function that periodically monitors the operating status of terminals and industrial equipment when conducting long-term tests, and records and analyzes the results.

[1939] The "means for analyzing log data and detecting abnormalities" refers to a function that analyzes log data collected from terminals and industrial equipment and detects abnormal operations or errors from the data.

[1940] The "means for reporting the analysis results" has a function for reporting the analysis results of the log data to the user, and notifies the user if an abnormality is detected.

[1941] "Means for reading the profile of industrial equipment and automatically applying the required initial settings" means a function that reads the profile information of industrial equipment and automatically applies the required initial settings (Wi-Fi settings, security policies, etc.) based on that information.

[1942] "Means for collecting test results and monitoring them in real time" means a means for collecting result data obtained during the test and monitoring the data in real time.

[1943] A system for realizing the present invention includes a server, industrial equipment (terminals), and user interaction. Specific embodiments will be described in detail below.

[1944] Hardware and Software

[1945] Hardware:

[1946] Server: A computer system with a powerful processor

[1947] Industrial equipment: Equipment equipped with various sensors and communication modules (e.g., robots, sensor devices)

[1948] Smart glasses: wearable devices for displaying real-time monitoring information (e.g., Google Glass)

[1949] software:

[1950] Server software: Python, real-time data streaming API, database management system

[1951] Log analysis engine: Anomaly detection algorithm using AI models

[1952] User interface: Web browser or dedicated application

[1953] Details of data processing and calculation

[1954] When a user registers an industrial device, the server reads its profile from the database and automatically applies initial settings (e.g., network settings, security policies) to the industrial device. This initial setting process is performed based on the device's profile information.

[1955] When industrial equipment starts a new task, the server runs multiple tests and collects and monitors the results in real time. When the test is completed, the server analyzes the obtained data and notifies the user if an abnormality is detected. This is done using an AI analysis engine for anomaly detection.

[1956] Furthermore, for equipment undergoing long-term testing, its operation is regularly monitored, and the server detects and notifies users of any abnormalities as necessary. All log data is centrally managed on the server and provided to users as reports as needed.

[1957] Specific examples

[1958] This section explains how this system works for robots used in factories. When a user registers a new robot in the system, the server loads the robot's profile and automatically applies initial settings such as Wi-Fi settings and security policies. It then conducts multiple operational tests and 24-hour continuous operation tests, monitoring the robot in real time. During this process, the server collects log data and detects abnormalities using an AI analysis engine. Detected abnormalities are immediately notified to the user, who can view the information through the smart glasses. In this way, users can efficiently manage the robot's operations and quickly respond to any abnormalities that occur.

[1959] Prompt Sentence Examples

[1960] "Please develop a system that monitors robots in factories and notifies them when an abnormality is detected. For example, it can automatically set up Wi-Fi and security policies, and conduct repeated and long-term tests to monitor the results in real time."

[1961] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1962] Step 1:

[1963] A user registers a new industrial device in the system. As input, the user provides device profile information (e.g., device ID, network settings, security policy, etc.). The server receives this information and stores it in a database. As output, the server sends a notification to the user that the registration is complete.

[1964] Step 2:

[1965] The server reads the saved profile information and automatically applies the necessary initial settings to the industrial equipment based on that information. As input, it receives the profile information and executes Wi-Fi settings, security policies, etc. based on that information. As output, it logs that the initial settings were successfully completed.

[1966] Step 3:

[1967] The user configures tests in the system. Specifically, the user specifies multiple tests (e.g., operation tests or connection tests) or long-term tests (e.g., 24-hour continuous operation tests). As input, the user provides test conditions (number of tests and duration). The server receives the test conditions and creates a test plan. As output, a test plan is generated and saved.

[1968] Step 4:

[1969] The server executes the specified tests. It reads the test plan as input and executes each test sequentially. Specifically, for multiple tests, it performs the specified number of attempts, and for long-term tests, it periodically monitors the operation within the specified time. As output, it records the results of each test (success / failure and operation log).

[1970] Step 5:

[1971] The server analyzes the log data collected during and after the test. It receives the collected operation logs and test results as input. It uses an AI model for analysis to detect anomalies. It outputs the analysis results in a report.

[1972] Step 6:

[1973] The server reports the analysis results to the user. As input, it generates a report of anomaly detection and notifies the user. As output, the user receives the report and can get detailed information about the status of industrial equipment.

[1974] Step 7:

[1975] If an anomaly is detected, the server immediately notifies the user. As input, it generates an anomaly notification based on the analysis results and forwards the content to the user. As output, the user receives the notification on a device such as smart glasses, allowing them to quickly address the problem.

[1976] Step 8:

[1977] Users review the maintenance and settings of industrial equipment based on the reports provided by the server. As input, they refer to the information in the report and carry out the necessary procedures. As output, they can confirm the improved settings and maintenance status. This cycle ensures efficient and reliable operation of industrial equipment.

[1978] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1979] This invention relates to a system that uses AI to automate processes such as terminal configuration, number testing, long-term testing, and log analysis in the verification process of device development, and further combines it with an emotion engine that recognizes user emotions. Specific embodiments for carrying out the present invention are described in detail below.

[1980] Automating device configuration

[1981] The user registers a new device in the system. When the user enters the device information and presses the registration button, the server receives this information. The server reads the device profile from the database and automatically applies the initial settings, such as network settings and security settings, to the device. Once the settings are complete, the server sends a notification to the user that the settings are complete.

[1982] Specific examples

[1983] When a user registers a new smartphone in the system, the server reads the smartphone profile and automatically configures Wi-Fi settings, security policies, and installs necessary applications. After the configuration is complete, the server notifies the user that the smartphone configuration is complete.

[1984] Automation of frequency testing

[1985] The user accesses the test execution screen and enters the test conditions (e.g., run the Wi-Fi connection test 100 times in a row). When the user presses the test start button, the server receives the test conditions and causes the device to run the test the specified number of times. The server collects data obtained during the test (number of successes, number of failures, connection time, etc.) and generates a report based on this data. The generated report is sent from the server to the user.

[1986] Specific examples

[1987] If a user configures the system to perform 100 Wi-Fi connection tests, the server will perform all 100 connection attempts, compile the results, create a report, and send it to the user.

[1988] Automating long-term testing

[1989] Once the user issues a command to start the test and sets the test period (e.g., 24 hours), the server will continue to monitor the device during that period. The server periodically checks the device's operating status (CPU usage, memory usage, network status, etc.) and immediately notifies the user if any abnormalities are detected.

[1990] Specific examples

[1991] When a user sets up a 24-hour continuous use test, the server periodically checks the device's operating status for 24 hours and immediately notifies the user if any abnormalities are detected.

[1992] Automated log analysis

[1993] The server periodically collects log data from the device, inputs this log data into an AI analysis engine, and detects abnormal patterns and errors in the log data. The server then generates a detailed report based on the analysis results and provides it to the user.

[1994] Specific examples

[1995] The server collects operation logs from the devices at a set time every day and runs them through an AI analysis engine to detect abnormal patterns and errors. Based on the results, a weekly report is generated and sent to the user by email.

[1996] User Emotion Recognition

[1997] The system also incorporates an emotion engine that recognizes the user's emotions. This emotion engine analyzes emotional data from the user's voice and facial expressions, and uses the results to adjust the system's operations and analyze log data.

[1998] Specific examples

[1999] The emotion engine recognizes emotions such as stress and anxiety felt by users while operating the system. Based on the results, the system makes suggestions to improve the user experience and analyzes log data by adding emotion data. For example, if a specific operation causes stress over a long period of time, the system will make suggestions to improve that operation method or the system interface.

[2000] Combining these elements will enable the device verification process to be more efficient, defects to be detected and addressed early, and user experience to be improved. This system will serve as a very useful tool for developers.

[2001] The processing flow will be explained below.

[2002] Automating device configuration

[2003] Step 1:

[2004] The user enters the information for the new device and presses the "Register" button on the device registration screen.

[2005] Step 2:

[2006] The server receives the terminal information sent from the user.

[2007] Step 3:

[2008] The server reads the device profile from the database.

[2009] Step 4:

[2010] The server automatically applies the device's network and security settings based on the profile loaded.

[2011] Step 5:

[2012] The server confirms the status of the configuration.

[2013] Step 6:

[2014] The server will send a notification to the user that the setup is complete.

[2015] Automation of frequency testing

[2016] Step 1:

[2017] The user accesses the test execution screen and enters the test conditions (e.g., execute the Wi-Fi connection test 100 times in succession).

[2018] Step 2:

[2019] The user presses the "Start Test" button.

[2020] Step 3:

[2021] The server receives the input test conditions.

[2022] Step 4:

[2023] The server causes the terminal to execute the test a specified number of times.

[2024] Step 5:

[2025] The server collects data obtained during the test (number of successes, number of failures, connection time, etc.).

[2026] Step 6:

[2027] The server generates reports based on the collected data.

[2028] Step 7:

[2029] The server generates a report and sends it to the user.

[2030] Automating long-term testing

[2031] Step 1:

[2032] The user accesses the long-term test execution screen and inputs the test period (e.g., 24 hours).

[2033] Step 2:

[2034] The user presses the "Start Test" button.

[2035] Step 3:

[2036] The server sets a timer for the test period.

[2037] Step 4:

[2038] The server will periodically monitor the terminal status (e.g., CPU usage, memory usage, network status) during the test period.

[2039] Step 5:

[2040] As soon as the server detects an abnormality, it will notify the user.

[2041] Step 6:

[2042] After the long test is completed, the server reports the test results to the user.

[2043] Automated log analysis

[2044] Step 1:

[2045] The server periodically collects operation logs from the terminal based on a set schedule (e.g., collects logs at midnight every day).

[2046] Step 2:

[2047] The log data collected by the server is input into an AI analysis engine.

[2048] Step 3:

[2049] The server uses an AI analysis engine to detect abnormal patterns and errors in the log data.

[2050] Step 4:

[2051] The server generates a detailed report based on the results of the AI ​​analysis engine.

[2052] Step 5:

[2053] Sending server-generated reports to users (e.g. email or dashboard notifications).

[2054] User Emotion Recognition

[2055] Step 1:

[2056] The user provides voice and facial expression data while operating the system (e.g., using a webcam or microphone).

[2057] Step 2:

[2058] The server receives the user's voice and facial expression data using an emotion engine.

[2059] Step 3:

[2060] The server uses an emotion engine to analyze the user's emotions.

[2061] Step 4:

[2062] The server receives the emotion data and adjusts the system's behavior based on the analysis results.

[2063] Step 5:

[2064] The server integrates the analyzed emotion data with the log data and performs further detailed analysis.

[2065] Step 6:

[2066] The server generates a report based on the user's emotional data to improve the user experience.

[2067] Specific examples

[2068] Example of automating device settings

[2069] The user enters the information to register a new smartphone and presses the "Register" button. The server receives the information, reads the smartphone profile from the database, and automatically applies the necessary network and security settings. It then sends a notification to the user that the setup is complete.

[2070] Automation of frequency testing

[2071] When a user sets up 100 Wi-Fi connection tests, the server receives the test conditions and performs 100 Wi-Fi connection tests on the device. The server collects success and failure data during the test and finally generates a report and sends it to the user.

[2072] Automating long-term testing

[2073] When a user sets up a 24-hour continuous use test, the server periodically monitors the terminal's operating status for 24 hours and notifies the user if any abnormalities occur.

[2074] Automated log analysis

[2075] The server collects log data from devices at a set time each day and runs it through an AI analysis engine to detect abnormal patterns. The analysis results are generated as a detailed report and sent to the user.

[2076] User Emotion Recognition

[2077] As a user operates the system, the emotion engine analyzes the user's emotions from voice and facial expression data. Based on the analysis results, the system adjusts system operation and makes suggestions to reduce the user's stress and anxiety. The emotion data is also integrated with log data for more detailed analysis, improving the user experience.

[2078] Example 2

[2079] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2080] The verification process for device development involves extremely time-consuming processes such as terminal configuration, repeated testing, long-term testing, and log analysis, which consumes a large amount of time and human resources.Furthermore, it is difficult to improve the user experience because the system cannot be adjusted to take user emotions into account.

[2081] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2082] In this invention, the server includes means for automating terminal configuration, means for recognizing user emotions, means for automatically executing multiple tests, means for periodically monitoring long-term tests, means for analyzing log data and detecting anomalies, means for reporting the analysis results, and means for adjusting system operation based on user emotion data, thereby enabling more efficient device verification processes, early detection and handling of defects, and an improved user experience.

[2083] A "means for automating terminal settings" is a means by which a server automatically performs network settings, security settings, and installation of necessary applications based on terminal information entered by the user.

[2084] The "means for automatically executing a test multiple times" is a means for the server to automatically execute a test on the terminal a specified number of times based on test conditions specified by the user.

[2085] The "means for periodically monitoring the long-term test" refers to a means for the server to periodically monitor the operating status of the terminal during the specified test period and to immediately notify the user if an abnormality occurs.

[2086] "Means for analyzing log data and detecting anomalies" refers to the means by which the server inputs log data acquired from the terminal into an AI analysis engine to detect abnormal patterns and errors.

[2087] The "means of reporting the analysis results" refers to the means of generating a detailed report based on the analysis results of the AI ​​analysis engine and providing it to the user.

[2088] The "means for recognizing the user's emotions" refers to a means by which the system acquires the user's voice and facial expressions and analyzes them using an emotion engine to recognize the user's emotional state.

[2089] The "means for adjusting system operation based on user emotion data" refers to a means for the system to improve operability and adjust the interface based on the recognized user emotion data.

[2090] "Means for collecting log data from terminals" refers to the means by which the server periodically obtains operation logs, system logs, application logs, etc. from terminals.

[2091] The present invention relates to a system that automates multiple processes in the verification process of device development and further recognizes user emotions.

[2092] Automating device configuration

[2093] When a user registers a new device in the system, the server receives the device information and automatically applies initial settings to the device, such as network settings, security settings, and installation of required applications. The server uses an API to perform these settings remotely. For example, when a user registers a smartphone, the server automatically configures Wi-Fi settings, security policies, and installs required applications. After the configuration is complete, the server notifies the user that the configuration was successful.

[2094] Automation of frequency testing

[2095] Based on the test conditions specified by the user, the server will perform the test on the device the specified number of times. For example, if you set the Wi-Fi connection test to be performed 100 times, the server will perform 100 connection attempts and compile the results. The server will collect data such as the number of successes, the number of failures, and the connection time, and generate a detailed report to send to the user.

[2096] Automating long-term testing

[2097] Once the user sets the test period, the server will periodically monitor the device during that period. For example, if a 24-hour long test is set, the server will periodically check the device's operating status (CPU usage, memory usage, network status, etc.). If an abnormality is detected, the server will immediately notify the user.

[2098] Automated log analysis

[2099] The server periodically collects log data from devices. This data is input into an AI analysis engine to detect abnormal patterns and errors. Based on the results of the AI ​​analysis engine, the server generates a detailed report and provides it to the user. For example, the server could collect operation logs from devices at a set time every day, run them through the AI ​​analysis engine to detect abnormalities, and send a weekly report to the user by email.

[2100] User Emotion Recognition

[2101] This system incorporates an emotion engine that recognizes emotions from the user's voice and facial expressions. The emotion engine analyzes the user's emotional data and uses the results to adjust the system's operation. For example, if a user feels stressed while operating the system, the emotion is recognized and the system suggests improvements to operability and interface adjustments. Emotion data is also integrated with log data for comprehensive analysis.

[2102] Examples of specific prompts include:

[2103] Register a new device and automatically install Wi-Fi settings and necessary apps.

[2104] "Please perform 100 Wi-Fi connection tests and submit the results as a report."

[2105] "Please conduct a 24-hour continuous use test and notify us immediately if any abnormalities occur."

[2106] "Analyze the device operation logs and prepare a weekly report if any abnormalities are detected."

[2107] "Please be aware of the user's stress level and provide suggestions for improving usability."

[2108] The system of the present invention is expected to improve the efficiency of the device verification process, enable early detection of defects, and improve the user experience.

[2109] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2110] System program processing flow

[2111] Automating device configuration

[2112] Step 1:

[2113] The user enters the information for the new device and clicks the registration button.

[2114] Specific operation: The user inputs information such as the device model name, serial number, and network settings. The input data is sent to the server.

[2115] Step 2:

[2116] The server receives the terminal information and temporarily stores it in a database.

[2117] Input: Device information (model name, serial number, network settings, etc.)

[2118] Data processing: Converting device information into the appropriate format and storing it in the database

[2119] Output: Saved device information

[2120] Step 3:

[2121] The server loads the device profile.

[2122] Input: Saved device information

[2123] Data calculation: Obtain the corresponding device profile (initial setting information) from the database

[2124] Output:Device profile

[2125] Step 4:

[2126] The server applies the initial settings to the device.

[2127] Input:Device profile

[2128] Specific operation: Automatically configures network settings (e.g. Wi-Fi settings), security settings, and installs required applications.

[2129] Output: Device with initial setup completed

[2130] Step 5:

[2131] The server will notify you that the settings are complete.

[2132] Input: Device information for which initial settings have been completed

[2133] Data processing: Generate notification messages and prepare them for sending

[2134] Output: Notification to the user that the settings have been completed

[2135] Automation of frequency testing

[2136] Step 1:

[2137] The user inputs the test conditions and clicks the test start button.

[2138] Specific operation: The user registers the test conditions (e.g., execute the Wi-Fi connection test 100 times). The input data is sent to the server.

[2139] Step 2:

[2140] The server receives the test conditions and stores them in a database.

[2141] Input: Test conditions (e.g., run Wi-Fi connection test 100 times)

[2142] Data processing: Convert test conditions into an appropriate format and save it in a database

[2143] Output: Saved test conditions

[2144] Step 3:

[2145] The server instructs the terminal to execute the test.

[2146] Input: Saved test conditions

[2147] Specific operation: The server sends a test execution command to the terminal.

[2148] Output: Test execution started

[2149] Step 4:

[2150] The server collects the test data.

[2151] Input: Test execution data (number of successes, number of failures, connection time, etc.)

[2152] Data calculation: The data obtained during the test is compiled and stored in a database.

[2153] Output: Collected test data

[2154] Step 5:

[2155] The server generates the report.

[2156] Input: Collected test data

[2157] Data Processing: Generate reports based on test results

[2158] Output: Generated report

[2159] Step 6:

[2160] The server sends the report to the user.

[2161] Input: Generated report

[2162] Specific behavior: Send the report to the user via email or system notification

[2163] Output: Reports test results to the user

[2164] Automating long-term testing

[2165] Step 1:

[2166] The user sets the test period and issues an instruction to start the test.

[2167] Specific operation: The user inputs the test period (e.g., 24 hours) and sends it to the server.

[2168] Step 2:

[2169] The server monitors the device during the test period.

[2170] Input: Test period setting information

[2171] Specific operation: The server periodically monitors the device's operating status during the set period.

[2172] Output: Monitoring data

[2173] Step 3:

[2174] The server checks the operation status.

[2175] Input: Monitoring data

[2176] Data calculation: Analyze the device's operating status (e.g., CPU usage, memory usage, network status)

[2177] Output: Analysis results of the operating status

[2178] Step 4:

[2179] The server detects an anomaly.

[2180] Input: Analysis results of operating conditions

[2181] Data arithmetic: Detecting abnormal patterns and errors

[2182] Output: Anomaly detection results

[2183] Step 5:

[2184] The server notifies the user of the abnormality.

[2185] Input: Anomaly detection results

[2186] Specific behavior: Generates a notification message and sends it to the user

[2187] Output: Abnormal notification to the user

[2188] Automated log analysis

[2189] Step 1:

[2190] The server collects log data from the terminal.

[2191] Specific operation: Periodically obtain system logs, operation logs, and application logs from the terminal.

[2192] Step 2:

[2193] The server inputs the log data into an AI analysis engine.

[2194] Input: Collected log data

[2195] Data processing: Converting collected log data into a format suitable for the AI ​​analysis engine

[2196] Output: Data input into the AI ​​analysis engine

[2197] Step 3:

[2198] An AI analysis engine analyzes the log data.

[2199] Input: Data entered into the AI ​​analysis engine

[2200] Data Calculation: Analyzing abnormal patterns and errors

[2201] Output: Analysis results

[2202] Step 4:

[2203] The server receives the analysis results and generates a detailed report.

[2204] Input: Analysis results

[2205] Data processing: Generate reports based on analysis results

[2206] Output: Generated report

[2207] Step 5:

[2208] The server provides the report to the user.

[2209] Input: Generated report

[2210] Specific behavior: Send the report to the user via email or system notification

[2211] Output: Regular reports to users

[2212] User Emotion Recognition

[2213] Step 1:

[2214] The system captures the user's voice and facial expressions.

[2215] Specific operation: Collects user voice and facial expression data using a webcam and microphone.

[2216] Step 2:

[2217] The emotion engine analyzes the emotion data.

[2218] Input: Collected voice and facial expression data

[2219] Data Calculation: The emotion engine analyzes the data and determines the user's emotional state.

[2220] Output: Emotion analysis results

[2221] Step 3:

[2222] The system receives the analysis results and adjusts its operations.

[2223] Input: Sentiment analysis results

[2224] Specific actions: Based on the analysis results, we suggest improvements to usability and interface adjustments.

[2225] Output: Suggestions for improving the user experience

[2226] Step 4:

[2227] The system uses emotional data to help with log analysis.

[2228] Input: Sentiment analysis results

[2229] Data calculation: Emotion data is integrated with log data for comprehensive analysis.

[2230] Output: Comprehensive analysis results

[2231] (Application example 2)

[2232] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2233] Conventional factory robot maintenance and testing processes require a lot of manual work and time, making them inefficient and limiting their ability to detect anomalies and improve the user experience. In particular, there is a lack of means to reduce the mental burden on workers.

[2234] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for automating terminal configuration, means for automatically executing multiple tests, means for periodically monitoring long-term tests, means for analyzing log data and detecting abnormalities, means for reporting the analysis results, means for performing emotion recognition and acquiring user emotion data, and means for adjusting system operation and proposing improvements based on the emotion data. This makes it possible to automate and streamline the maintenance and verification processes of factory robots, and further reduce the mental burden on workers.

[2235] A "means for automating device settings" is a means by which, when a user registers a new device in the system, the server reads the device's profile and automatically applies initial settings such as network settings and security settings.

[2236] The "means for automatically executing multiple tests" refers to a means for the server to automatically execute a set number of tests on the corresponding terminal based on test conditions specified by the user, and to tally up the test results.

[2237] The "means for periodically monitoring a long-term test" is a means for the server to periodically monitor the operating status of the terminal during the test period set by the user, and to immediately notify the user if an abnormality occurs.

[2238] "Means for analyzing log data and detecting anomalies" refers to a means for inputting log data collected from terminals into an AI analysis engine to automatically detect abnormal patterns and errors.

[2239] The "means for reporting the analysis results" is a means for generating a detailed report based on the analysis results of the log data and providing the report to the user.

[2240] The "means for performing emotion recognition and acquiring user emotion data" refers to means including an emotion engine for analyzing emotion data from the user's voice and facial expressions and acquiring that data.

[2241] "Means for adjusting system operation and proposing improvements based on emotional data" refers to means for adjusting system operation and proposing improvements to the system's operation method or interface based on the stress or anxiety felt by the user.

[2242] The present invention aims to automate and improve the efficiency of maintenance and inspection processes for factory robots. Specific embodiments for carrying out the present invention will be described in detail below.

[2243] System Configuration

[2244] The system includes the following major components:

[2245] Server: The core device that processes and analyzes data.

[2246] Factory robot: An automated device that performs tasks on a manufacturing line.

[2247] Sensor: A device that detects the robot's operating status, temperature, sound, etc.

[2248] Emotion engine: A system that performs speech analysis (e.g., IBM Watson Speech to Text).

[2249] AI analytics engine: Machine learning models (e.g., TensorFlow) that perform anomaly detection.

[2250] System functions and their operation

[2251] 1. Automate device configuration:

[2252] When a user registers a new factory robot in the system, the server reads the robot's profile and automatically applies initial settings such as network settings and security settings. Once the configuration is complete, the server sends a notification to the user.

[2253] 2. Automating multiple tests:

[2254] When the user inputs test conditions (e.g., repeat a specific action 100 times), the server automatically executes the test based on the conditions on the robot, collates and analyzes the results, and provides a report of the test results to the user.

[2255] 3. Regular monitoring of long-term tests:

[2256] Once the user sets the duration of the long-term test, the server periodically monitors the robot's operation during that period. If an abnormality is detected, the server immediately notifies the user.

[2257] 4. Log data analysis and anomaly detection:

[2258] The server inputs the log data collected from the robot into an AI analysis engine, which detects abnormal patterns and errors. Based on the analysis results, a detailed report is generated and provided to the user.

[2259] 5. User Emotion Recognition:

[2260] An emotion engine is built into the server to collect emotional data from the voice and facial expressions of the worker while the robot is operating. If the user feels anxious or stressed, that emotional data is reflected in the system, which then adjusts the robot's behavior and suggests improvements.

[2261] Adding specific examples

[2262] For example, if a user sets up a 24-hour continuous use test, the server will periodically check the robot's operating status for 24 hours. If an abnormality is detected, the user will be notified immediately. Also, if a worker complains that the robot is "slow," the emotion engine will detect this stress and prioritize log analysis of that area.

[2263] Example of input prompt for generative AI model

[2264] Collect operation logs of robots that operate continuously for 24 hours and use an AI analysis engine to detect abnormal patterns. Also, analyze the voice data of workers and prioritize areas where they feel stress or anxiety in the log analysis and report them.

[2265] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2266] Step 1:

[2267] The server receives a registration request from the user for a new factory robot. The information entered includes the robot's profile data (e.g., model number, work profile). The server uses this information to automatically perform initial configuration such as network settings and security settings. After configuration is complete, the server sends a notification to the user, allowing them to confirm that the robot has been successfully added to the system.

[2268] Step 2:

[2269] The user inputs multiple test conditions into the server. The test conditions include the number of times a specific operation is to be repeated. Based on the conditions, the server sends commands to the robot to execute the test and collects the test execution results. The server aggregates the collected data and calculates the number of successes and failures. Finally, the server provides the test results to the user as a report.

[2270] Step 3:

[2271] The user sets a long-term test period. During that period, the server periodically sends commands to the robot to monitor its operating status (e.g., CPU usage, temperature). The server analyzes the operating data received from the robot, and if an abnormality is detected, it immediately sends a notification email to the user. This allows the user to detect and respond to abnormalities early.

[2272] Step 4:

[2273] The server collects log data from the robots daily or periodically. This log data is input into an AI analysis engine to detect abnormal patterns and errors. The server then generates a detailed report based on the data and provides it to the user, allowing the user to quickly identify potential problems and take measures.

[2274] Step 5:

[2275] While the robot is operating, the emotion engine analyzes the worker's voice and facial expressions in real time. The input voice data is passed through the emotion engine and output as emotional data. The server uses this emotional data to detect the user's stress or anxiety, and based on the results, adjusts the system's operation and suggests improvements. This reduces the mental burden on the worker and improves work efficiency.

[2276] For example, if a user sets up a 24-hour continuous use test, the server will monitor the robot 24 hours a day and immediately notify the user if any abnormalities are detected. Also, if a worker points out that the robot is slow, the emotion engine will detect stress and prioritize analysis of related logs, helping to resolve the problem quickly.

[2277] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2278] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2279] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2280] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2281] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2282] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2283] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2284] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2285] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2286] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2287] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2288] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2289] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2290] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2291] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[2292] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[2293] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[2294] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[2295] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[2296] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[2297] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[2298] The following is further disclosed regarding the above embodiment.

[2299] (Claim 1)

[2300] A means to automate terminal configuration,

[2301] means for automatically performing multiple tests;

[2302] a means of periodically monitoring the long-term test;

[2303] A means for analyzing the log data and detecting anomalies;

[2304] a means for reporting the results of that analysis;

[2305] A system including:

[2306] (Claim 2)

[2307] 10. The system of claim 1, further comprising means for collecting log data from the terminal.

[2308] (Claim 3)

[2309] 2. The system according to claim 1, further comprising means for notifying a user when an abnormality is detected.

[2310] "Example 1"

[2311] (Claim 1)

[2312] A means to automate terminal configuration,

[2313] means for automatically performing multiple tests;

[2314] a means of periodically monitoring the long-term test;

[2315] A means for analyzing the log data and detecting anomalies;

[2316] a means for reporting the results of that analysis;

[2317] A means for monitoring the operation status of the terminal;

[2318] a means for notifying the user of the completion of the setting;

[2319] A system including:

[2320] (Claim 2)

[2321] 10. The system of claim 1, further comprising means for collecting log data from the terminal.

[2322] (Claim 3)

[2323] 2. The system accordi...

Claims

1. A means to automate terminal configuration, means for automatically performing multiple tests; a means of periodically monitoring the long-term test; A means for analyzing log data and detecting anomalies; a means for reporting the results of that analysis; A system including:

2. The system of claim 1 further comprising means for collecting log data from the terminal.

3. 2. The system according to claim 1, further comprising means for notifying a user when an abnormality is detected.

Images