system

Abstract

We provide the system. [Solution] A means of collecting real-time data from a terminal, A means of inputting the collected data into a generating AI model and analyzing it, A means for integrating the analyzed results and generating interactive visual data, A means of transmitting and displaying the generated visual data on a terminal, A means of making data manipulable through a user interface, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In recent business environments, many organizations require appropriately managing the balance of the workloads of team members and visualizing the efficient progress of tasks. However, conventional methods have problems in that it is difficult to collect data in real time and integrate various data formats, and it is impossible to intuitively grasp the waves of the workload. For this reason, each member cannot review tasks at an appropriate timing, which has an adverse effect on the overall productivity.

Means for Solving the Problems

[0005] To solve the above problems, the present invention provides a system that collects real-time data from a terminal and analyzes it by inputting it into a generated AI model. By generating data to visualize the fluctuations in workload and the progress of tasks obtained as a result of the analysis, and transmitting and displaying it on the terminal, the system enables users to interactively manipulate the data and understand the content of their work more quickly and accurately. This makes it possible to improve the efficiency and productivity of operations.

[0006] "Real-time data" refers to data that is collected and processed simultaneously with its generation, and is immediately available without delay.

[0007] A "terminal" is an electronic device used by a user to input data or receive information, and includes computers, smartphones, tablets, and other similar devices.

[0008] A "generative AI model" is an algorithm or mathematical model that learns from vast amounts of data and performs reasoning and predictions based on human knowledge and experience.

[0009] "Analysis" is the process used to verify, evaluate, and understand collected data, with the aim of identifying patterns and trends in the data.

[0010] "Visual data" refers to data that represents information in a visual format, such as information presented in the form of diagrams, graphs, and charts.

[0011] A "user interface" is the means of interaction that allows a user to operate a system and receive information from it, and includes the graphical display and various input devices. [Brief explanation of the drawing]

[0012] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2, which combines an emotion engine. [Modes for carrying out the invention]

[0013] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0014] First, the terms used in the following description will be explained.

[0015] In the following embodiments, a processor with a reference numeral (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0016] In the following embodiments, a RAM (Random Access Memory) with a reference numeral is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0017] In the following embodiments, a storage with a reference numeral is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0018] In the following embodiments, a communication I / F (Interface) with a reference numeral is an interface including a communication processor and an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark), etc.

[0019] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0020] [First Embodiment]

[0021] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0022] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0023] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0024] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0025] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0026] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0027] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0028] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0029] As shown in Figure 2, in the data processing device 12, specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0030] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0031] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0032] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0033] In order to implement the system of this invention, the server, terminal, and user must cooperate with each other. Their respective roles are described below.

[0034] server

[0035] The server is the central component responsible for data processing and management across the entire system. It utilizes generative AI models to analyze real-time data received from terminals. Specifically, the server centrally collects data from terminals and stores it in a database. The generative AI model then analyzes fluctuations in workload and task progress, generating visual data based on the results. The server sends these results to the user's terminal, displaying them on the interface.

[0036] terminal

[0037] The terminal is a device for users to input data and view system output. Users can input their workload and task progress through the interface on the terminal. Audio and image data formats are also supported as needed. The terminal displays visual data sent from the server using the user interface, enabling user interaction.

[0038] User

[0039] Users are the primary actors who leverage the system's output to efficiently manage tasks. They can instantly check their workload and task progress through their terminals, and use this information to reassign or adjust tasks accordingly. Users can also utilize filtering and search functions to display only information of interest. Furthermore, system improvements can be facilitated through user feedback.

[0040] Specific example

[0041] For example, consider a project team leader who wants to check the workload of team members during a busy period. The leader uses a terminal to input the team's overall work information. The server receives this data and analyzes it in real time using a generated AI model. As a result, the fluctuations in each member's workload and the progress of their tasks are visualized and instantly displayed on the leader's interface from the terminal. Based on this information, the leader can make decisions about reallocating or adjusting tasks. This series of actions contributes to improving the user's work efficiency.

[0042] Thus, the present invention enables the various components of the system to work together to improve the efficiency and productivity of the user's daily work.

[0043] The following describes the processing flow.

[0044] Step 1:

[0045] The terminal acquires information on workload and task progress entered by the user. This information is accepted in text, audio, and image formats, and appropriate preprocessing is performed for each format (for example, converting audio to text).

[0046] Step 2:

[0047] The terminal converts the acquired data into a format that can be sent to the server. For example, it converts it to JSON or XML format and sends it to the server using a secure communication protocol (e.g., HTTPS).

[0048] Step 3:

[0049] The server receives data from terminals in real time and stores it in a database for centralized management. During storage, validation and error checking are performed to maintain data integrity.

[0050] Step 4:

[0051] The server inputs new data from the database into a generative AI model for analysis. The AI ​​model analyzes patterns of workload and task progress, and outputs statistical predictions.

[0052] Step 5:

[0053] The server generates interactive visual data (graphs and charts) based on the analysis results output by the AI ​​model. This visual data is designed in a format that is easy for the user to see and understand.

[0054] Step 6:

[0055] The server sends the generated visual data to the terminal. On the terminal, the received visual data is displayed on the interface, providing an environment where the user can intuitively manipulate the data.

[0056] Step 7:

[0057] Users interact with data using a dashboard displayed on their device. This interaction includes filtering data, viewing detailed information, and specifying time periods.

[0058] Step 8:

[0059] Users can reassign tasks and adjust their workload based on the information obtained from the dashboard. This leads to improved work efficiency and a better balance of workload within the team.

[0060] (Example 1)

[0061] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0062] The challenge is to provide a system that streamlines the process from information input to analysis, visualization, and presentation to the user, effectively manages the progress and workload of tasks in real time, and enables users to make appropriate decisions quickly.

[0063] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0064] In this invention, the server includes means for acquiring real-time information from a device, means for inputting the acquired information into a generating AI model and analyzing it, and means for creating a visual representation based on the analysis results. This allows for the real-time visualization of workload and progress, enabling users to instantly grasp the situation and make appropriate decisions.

[0065] "Real-time information" refers to data that is collected and processed immediately, reflecting changes in current situations and events in real time.

[0066] A "device" is an electronic device used by users to input data or receive results.

[0067] A "generative AI model" is an artificial intelligence algorithm that analyzes data and generates results or patterns that are tailored to a specific purpose.

[0068] "Analysis" is the process of understanding given data and finding meaning and patterns that are appropriate to a specific perspective or purpose.

[0069] "Visual representation" refers to presenting data content and analysis results visually in forms such as graphs and charts, and is used to aid in understanding information.

[0070] A "human-machine interface" is a means or design for users to interact with machines or systems, enabling effective information input and output.

[0071] This invention is implemented through the collaboration of the server, terminal, and user components. The server, as the heart of the system, acquires, analyzes, and visualizes data. First, the server acquires real-time information from the devices and inputs that data into a generative AI model. The generative AI model used here is designed to analyze workload and task progress.

[0072] The server utilizes a generative AI model to analyze the acquired data. During this process, it is given a prompt message such as, "Analyze and visualize the business data." The analysis results are generated as a visual representation, provided in the form of graphs, charts, and other visual formats. The visualized data is then transferred from the server to the terminal, making it available for user use.

[0073] The terminal serves as an interface for users to input data and receive analysis results. Users can input business data into the terminal in voice, text, or image format. The terminal interface is optimized for users to input information and review visual data from the server through a human-machine interface.

[0074] For example, a user might select "Enter the number of tasks completed this week and save" via their device to manage project progress. The server then uses this information to generate a prompt, "Analyze and visualize the progress," which is passed to an AI model, and the results are provided to the user. This entire process allows the user to quickly grasp a wealth of information, enabling efficient work operations.

[0075] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0076] Step 1:

[0077] The server acquires real-time information from devices. The input is business data provided by the user through their terminal. The server formats this data into the correct format and stores it in a database. This process is necessary for centralized data management and supports subsequent analysis.

[0078] Step 2:

[0079] The terminal transmits data in the form of voice, text, or images input by the user to the server. While the input can vary, the output arrives at the server as structured digital data. The terminal provides preview templates through a user interface to assist with accurate information input, allowing the user to verify the data.

[0080] Step 3:

[0081] The server inputs the acquired data into a generating AI model to analyze workload and task progress. In this process, the input data is passed to the AI ​​model along with the prompt message, "Analyze and visualize the business data." The generating AI model uses various algorithms to process the data and obtain results. As a result of the analysis, specific patterns and trends are extracted.

[0082] Step 4:

[0083] The server creates visual representations based on the analysis results of the generated AI model. The analysis results serve as input, and the output is visualized data. This visual data is created in various formats (e.g., graphs and charts) to make it easy for the user to understand.

[0084] Step 5:

[0085] The server transfers the visual representation to the terminal. The output visual data arrives at the terminal as input from the server. The terminal receives this data and displays it intuitively on the user interface. For example, the user can use zoom in and out functions to examine the data in order to compare data or understand trends.

[0086] Step 6:

[0087] Users adjust their work and make decisions based on the visual data acquired through their devices. Based on the visual data received as input, users determine the prioritization of new tasks and the reallocation of resources. This process enables improved work efficiency and effective project management.

[0088] (Application Example 1)

[0089] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0090] Monitoring and managing the operating status and workload of robots in factories is crucial for maintaining efficient production. However, early detection and rapid response to minor and major anomalies are difficult, potentially delaying work progress. Therefore, there is a need for a system that visualizes robot operating status, quickly detects anomalies, and reallocates tasks.

[0091] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0092] In this invention, the server includes means for collecting real-time information from the device, means for inputting the collected data into a generating AI model and analyzing it, and means for integrating the analyzed results and generating interactive visual information. This makes it possible to visualize the operating status and workload of robots in a factory, quickly detect anomalies, and reallocate tasks.

[0093] "Real-time information" refers to data that allows you to instantly acquire and verify the latest status and changes on the spot.

[0094] A "device" is a machine or mechanism designed to perform a specific function.

[0095] "Collection" is the act of gathering and accumulating necessary information.

[0096] A "generative AI model" is an artificial intelligence algorithm that learns from data and performs tasks such as prediction and classification.

[0097] "Analysis" is the process of breaking down and examining data to reveal its structure and relationships.

[0098] "Integration" is the act of combining multiple pieces of information or data to create a unified whole.

[0099] "Interactive visual information" refers to visual data that is provided in a way that allows users to interact with and use it interactively.

[0100] A "terminal" is a computer or device used by a user for direct operation.

[0101] A "user interface" is the part of a computer or system that allows a user to interact with it.

[0102] "Robot operating status" refers to information that shows what tasks or actions robots in a factory are currently performing.

[0103] "Workload" is a concept that represents the amount of work or processing required for a particular task.

[0104] "Visualization" is a technique that makes data and information visible through graphs, charts, and other visual means.

[0105] "Task reallocation" is the act of readjusting the assignment of tasks or duties.

[0106] "Abnormal" refers to a state or operation that deviates from the normal condition or behavior.

[0107] To implement this invention, it is necessary to build a system in which a server, terminals, and users work together. The server collects real-time information on robots in the factory and analyzes this information using a generated AI model. Here, data is received from sensors mounted on each robot, and data analysis is performed using Python and TENSORFLOW® to understand the robot's operating status and workload. The analysis results are visually integrated as a web application using Django. This visual data is transmitted to terminals in an interactive format, and operators can access it via smartphones or tablets.

[0108] The device provides users with a user interface that allows them to manipulate information. This interface is developed with React Native and is designed to be intuitive and easy for users to use. Through this interface, users can make quick decisions and reallocate tasks between robots as needed.

[0109] As a concrete example, suppose an abnormal vibration occurs in one of the robots on a manufacturing line. In this case, the server uses a generated AI model to detect the anomaly. The analyzed data is visualized and displayed on the terminal in real time, enabling operators to quickly detect the anomaly and formulate countermeasures.

[0110] An example of a prompt message used is, "Based on the vibration data of robot R123, detect abnormal load patterns and visually report them to the operator." This prompt ensures that the generated AI model operates correctly, enabling a mechanism to immediately detect abnormalities in the robot.

[0111] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0112] Step 1:

[0113] The server collects real-time information from sensors mounted on each robot. The input is data from the robot's sensors, and the raw data is stored as output in temporary storage. This data includes information on the robot's position, operating speed, vibration, and load.

[0114] Step 2:

[0115] The server inputs the collected raw data into a generating AI model for analysis. The raw robot data obtained in Step 1 is used as input. The generating AI model is built using TensorFlow and performs data feature extraction and anomaly pattern detection. The output consists of analysis results regarding the operating status and workload of each robot.

[0116] Step 3:

[0117] The server integrates the analysis results and generates interactive visual information. The input is the analysis results output in step 2, which are then integrated into a web application using Django for visualization. The output consists of graphs and charts that the user can view on their device.

[0118] Step 4:

[0119] The terminal receives visual information transmitted from the server and displays it through the user interface. The input is the visual data generated in step 3, and the output is an interactive dashboard displayed on the terminal's screen. Here, the user can manipulate the data and view details.

[0120] Step 5:

[0121] The user reassigns tasks to the robot based on the information displayed on the terminal. The input is the visual information obtained in step 4 and the user's judgment, and the output is the new instructions for the robot. If necessary, prompts can be used to request the generating AI model to investigate anomalies in detail.

[0122] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0123] This invention provides a system that visualizes the user's workload and task progress in real time, and further recognizes and incorporates the user's emotions into the system, thereby achieving more precise work management. The following describes each element.

[0124] server

[0125] The server plays a central role in the system, receiving and integrating real-time and sentiment data transmitted from terminals. The received data is input into a generative AI model for analysis. This results in the generation of visual data that takes into account workload fluctuations, task progress, and even user emotions. The server then transmits this visual data to terminals and controls its display on the user interface.

[0126] terminal

[0127] The terminal is a device for users to input business information and emotional data. The information entered by the user is acquired as text, audio, or images. In particular, an emotion engine is used to extract the user's emotional state from the audio data. The terminal sends the acquired information to a server and provides a user interface for displaying the visual data received from the server.

[0128] User

[0129] Users input their daily workload and task status into the system, and use the resulting data to improve work efficiency. Users can operate a dashboard displayed on their terminal to filter information for specific periods or team members. Furthermore, they can reduce stress and proceed with their work by referring to the system's task reallocation suggestions based on user sentiment.

[0130] Specific example

[0131] For example, if a user is behind schedule on a project, the system sends daily emotional inputs to the server via voice data. The server analyzes this data using an emotion engine and suggests that the user may be experiencing stress. As a result, alerts that take the user's workload into account are displayed on the dashboard, allowing the project leader to re-evaluate task priorities and allocate resources appropriately.

[0132] Thus, the present invention enables not only direct management of workload but also advanced work management that takes into account the emotional state of users, thereby contributing to the realization of a better work environment.

[0133] The following describes the processing flow.

[0134] Step 1:

[0135] Users input information about their workload and task progress into the terminal. They can also express emotions using voice input as needed. The voice is converted into digital data by the terminal.

[0136] Step 2:

[0137] The terminal converts the user's input data into a format and sends it to the server. This data includes text data, voice data, and indicators of emotion.

[0138] Step 3:

[0139] The server records the data received from the terminal into a database. During this process, it performs checks to ensure data integrity.

[0140] Step 4:

[0141] The server inputs stored data into a generating AI model to analyze workload and task progress. Furthermore, it inputs voice data into an emotion engine to analyze the user's emotional state.

[0142] Step 5:

[0143] The server integrates the analysis results of the generative AI model and the emotion engine to generate visual data. This data visually represents workload, task progress, and emotional state.

[0144] Step 6:

[0145] The server sends the generated visual data to the terminal. On the terminal, the data is displayed on a dashboard and can be interactively manipulated through the user interface.

[0146] Step 7:

[0147] Users can use the dashboard on their device to check their workload, task progress, and sentiment analysis results. They can use the filtering function to narrow down the information as needed.

[0148] Step 8:

[0149] Based on the system's suggestions derived from emotional data, users can adjust their work priorities and optimize task allocation. This allows users to work more efficiently while reducing stress.

[0150] (Example 2)

[0151] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0152] In the workplace, it is essential to accurately understand and efficiently manage workloads, task progress, and employee emotional states. In particular, integrating and visualizing this information in real time is crucial for managers to make appropriate decisions. However, current systems struggle to process and integrate this information quickly and effectively, resulting in insufficient work efficiency and adequate employee stress management.

[0153] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0154] In this invention, the server includes a device for aggregating time-series information from data processing devices, a device for inputting and analyzing the aggregated information into AI technology, and a device for integrating the analyzed output and generating visualization information. This enables accurate, real-time understanding of workload, task progress, and user sentiment, allowing for rapid and effective business management.

[0155] A "data processing device" is a device that allows users to input data, stores information, and transmits it to a server.

[0156] "Time-series information" refers to data collected at regular time intervals, representing the user's work situation and emotional state.

[0157] "Generative AI technology" refers to artificial intelligence technology used to analyze input data and identify specific predictions or patterns.

[0158] "Visualized information" refers to information that represents analyzed data in visual forms such as graphs and charts, and is presented in a way that is easy for users to understand intuitively.

[0159] A "user interface" is a screen that provides an interface for users to manipulate information and obtain necessary data.

[0160] This invention provides a system that visualizes a user's workload and task progress in real time, and optimizes work management by recognizing the user's emotions. This system operates using a combination of a server and terminals. The following describes its embodiments.

[0161] The terminal is a device for users to input business information and emotional data. The terminal is responsible for text input, voice recording, and image data capture, and transmits this data to the server. For voice data, an emotion engine is used to identify the user's emotions from the voice and analyze them as numerical data.

[0162] The server is the central hardware of this system. The server receives data sent from terminals and inputs it into a generative AI model. The generative AI model is built using cloud services such as Microsoft® Azure® and Google® Cloud Platform. This allows for a comprehensive analysis of workload, task progress, and user emotional states. The server integrates and visualizes these analysis results, sending them to terminals as visually easy-to-understand data.

[0163] Users can use a dashboard displayed on their device to monitor their work status in real time. The dashboard allows them to filter information by specific time periods or project members, enabling them to adjust their work accordingly. Furthermore, users can reallocate tasks and improve work efficiency based on suggestions from the system.

[0164] As a concrete example, if a user is behind schedule on a project, they input their daily emotions as voice data. The server analyzes this data using an emotion engine and displays warnings based on their stress level on a dashboard. Based on these results, the project leader can re-evaluate task priorities and allocate resources appropriately.

[0165] An example of a prompt message would be, "Please integrate task management and user sentiment data from the project to provide appropriate suggestions based on my stress level."

[0166] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0167] Step 1:

[0168] The terminal collects work information and emotional states entered by the user. Specifically, the terminal acquires data through text input, voice recording, and image capture. The input voice data is sent to the emotion engine and converted into numerical data representing the emotional state. The output consists of work information and numerical emotional data.

[0169] Step 2:

[0170] The terminal transmits collected business information and sentiment data to the server in real time. It is crucial to maintain data integrity by using a secure communication protocol. The input is the data collected in step 1, and the output is the transmission status confirming that the data successfully reached the server.

[0171] Step 3:

[0172] The server inputs data received from the terminal into the generative AI model. Specifically, this involves converting the data format into a format the model can process. The generative AI model resides in the cloud and performs advanced calculations to integrate and analyze multiple data streams. The input consists of received business information and sentiment data, while the output consists of data patterns and predictive information as a result of the analysis.

[0173] Step 4:

[0174] The server generates integrated visualization data based on the analyzed results. Specifically, it represents the data as graphs and charts, converting it into a format that is easy for users to intuitively understand. The input is the analysis results obtained in step 3, and the output is a visualized information package.

[0175] Step 5:

[0176] The server sends the generated visualization data to the terminal. At this time, it verifies that the data transfer is of high quality and prepares the terminal for display. The input is the visualized information, and the output is the display status that can be viewed on the terminal.

[0177] Step 6:

[0178] Users can monitor their work progress and emotional state through a dashboard displayed on their device, and adjust tasks as needed. Specifically, users can use the interface's filtering function to focus on specific information. The input is the data displayed on the dashboard, and the output is the user's work decisions and action plans for the next steps.

[0179] (Application Example 2)

[0180] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0181] In factory environments, it is crucial to understand workers' workload and mental stress in real time and create an efficient work environment. However, existing systems have not adequately considered workers' emotions or real-time task progress, making it difficult to adjust task priorities appropriately. Therefore, there is a need to improve work efficiency while simultaneously reducing worker stress.

[0182] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0183] In this invention, the server includes means for collecting real-time data from terminals, means for inputting the collected data into a generating AI model and analyzing it, means for integrating the analyzed results and generating interactive visual data, means for evaluating emotional states from acquired audio and image data and calculating workload, and means for making suggestions to adjust the priority of work tasks based on the generated emotional state and workload information. This makes it possible to grasp the emotions and workload of workers in real time and adjust task priorities appropriately.

[0184] "Real-time data" refers to data that instantly reflects ongoing situations and events.

[0185] "Terminal" refers to input and output devices used as part of a system.

[0186] A "generative AI model" is an artificial intelligence algorithm that generates new information or predictions based on input data.

[0187] "Analysis" is the process of processing data to reveal the information and patterns contained within it.

[0188] "Visual data" refers to information presented in a visual format and used to facilitate user understanding.

[0189] "Emotional state" refers to data that indicates an individual's emotional situation or psychological state.

[0190] "Workload" refers to the amount of effort and resources required to perform a task.

[0191] "Task priority" is an indicator that shows the importance and urgency of the tasks that need to be done.

[0192] A "proposal" refers to advice or recommendations made based on specific data or conditions.

[0193] The system for implementing this invention mainly consists of three elements: a server, a terminal, and a user. The server has the function of receiving and collecting real-time data from the terminal within the factory. This data includes audio and images. The server inputs this data into a generating AI model to evaluate emotional states and workload. A common tool as a speech recognition library is used for speech recognition, and an emotion analysis engine is used for emotion analysis. This integrates the analyzed information and generates interactive visual data that can be visually presented to the user.

[0194] The terminal is a device, such as smart glasses worn by the worker, that displays visual data transmitted from the server. This allows the worker to understand their emotional state and workload assessment in real time. To improve work efficiency, the server suggests adjusting the priority of work tasks based on the emotional state and workload information. This suggestion is displayed on the terminal via an interactive user interface, helping the worker to progress through tasks in the most optimal way.

[0195] For example, if a worker is experiencing excessive stress, the system will analyze the data and display a suggestion on the screen indicating that a short break or time to refresh is needed. An example of a prompt message would be, "Based on the progress of your current task and the sentiment analysis, please determine whether a break is necessary."

[0196] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0197] Step 1:

[0198] The user puts on smart glasses and begins working. The device captures the worker's voice and facial expressions in real time. This data is saved as audio and image files for use in the next analysis step.

[0199] Step 2:

[0200] The device converts the acquired audio data into text data using a speech recognition library. This text data is then prepared for input into the sentiment analysis engine. As a result, verbal instructions and situational descriptions are interpreted as textual information.

[0201] Step 3:

[0202] The server receives text and image data sent from the terminal. The received text data is input into the sentiment analysis engine, which outputs an emotion score to evaluate the user's emotional state. For image data, image analysis technology is used to extract additional emotional information from facial expressions.

[0203] Step 4:

[0204] The server integrates sentiment scores and image analysis results to assess the current workload. Based on this assessment, a generative AI model is used to calculate task priorities. This generative AI model refers to historical data and successful examples from similar situations to suggest the optimal plan.

[0205] Step 5:

[0206] The server visualizes integrated emotional states, workload, and task priorities, generating interactive visual data. This visual data includes specific task suggestions and break recommendations for the worker. This data is then sent to the terminal via the user interface in the next step.

[0207] Step 6:

[0208] When the terminal receives visual data transmitted from the server, it displays specific advice and suggestions to the user to improve work efficiency. The user can then view this information through the display on their glasses and adjust their work accordingly.

[0209] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0210] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0211] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0212] [Second Embodiment]

[0213] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0214] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0215] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0216] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0217] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0218] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0219] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0220] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0221] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0222] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0223] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0224] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0225] In order to implement the system of this invention, the server, terminal, and user must cooperate with each other. Their respective roles are described below.

[0226] server

[0227] The server is the central component responsible for data processing and management across the entire system. It utilizes generative AI models to analyze real-time data received from terminals. Specifically, the server centrally collects data from terminals and stores it in a database. The generative AI model then analyzes fluctuations in workload and task progress, generating visual data based on the results. The server sends these results to the user's terminal, displaying them on the interface.

[0228] terminal

[0229] The terminal is a device for users to input data and view system output. Users can input their workload and task progress through the interface on the terminal. Audio and image data formats are also supported as needed. The terminal displays visual data sent from the server using the user interface, enabling user interaction.

[0230] User

[0231] Users are the primary actors who leverage the system's output to efficiently manage tasks. They can instantly check their workload and task progress through their terminals, and use this information to reassign or adjust tasks accordingly. Users can also utilize filtering and search functions to display only information of interest. Furthermore, system improvements can be facilitated through user feedback.

[0232] Specific example

[0233] For example, consider a project team leader who wants to check the workload of team members during a busy period. The leader uses a terminal to input the team's overall work information. The server receives this data and analyzes it in real time using a generated AI model. As a result, the fluctuations in each member's workload and the progress of their tasks are visualized and instantly displayed on the leader's interface from the terminal. Based on this information, the leader can make decisions about reallocating or adjusting tasks. This series of actions contributes to improving the user's work efficiency.

[0234] Thus, the present invention enables the various components of the system to work together to improve the efficiency and productivity of the user's daily work.

[0235] The following describes the processing flow.

[0236] Step 1:

[0237] The terminal acquires information on workload and task progress entered by the user. This information is accepted in text, audio, and image formats, and appropriate preprocessing is performed for each format (for example, converting audio to text).

[0238] Step 2:

[0239] The terminal converts the acquired data into a format that can be sent to the server. For example, it converts it to JSON or XML format and sends it to the server using a secure communication protocol (e.g., HTTPS).

[0240] Step 3:

[0241] The server receives data from terminals in real time and stores it in a database for centralized management. During storage, validation and error checking are performed to maintain data integrity.

[0242] Step 4:

[0243] The server inputs new data from the database into a generative AI model for analysis. The AI ​​model analyzes patterns of workload and task progress, and outputs statistical predictions.

[0244] Step 5:

[0245] The server generates interactive visual data (graphs and charts) based on the analysis results output by the AI ​​model. This visual data is designed in a format that is easy for the user to see and understand.

[0246] Step 6:

[0247] The server sends the generated visual data to the terminal. On the terminal, the received visual data is displayed on the interface, providing an environment where the user can intuitively manipulate the data.

[0248] Step 7:

[0249] Users interact with data using a dashboard displayed on their device. This interaction includes filtering data, viewing detailed information, and specifying time periods.

[0250] Step 8:

[0251] Users can reassign tasks and adjust their workload based on the information obtained from the dashboard. This leads to improved work efficiency and a better balance of workload within the team.

[0252] (Example 1)

[0253] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0254] The challenge is to provide a system that streamlines the process from information input to analysis, visualization, and presentation to the user, effectively manages the progress and workload of tasks in real time, and enables users to make appropriate decisions quickly.

[0255] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0256] In this invention, the server includes means for acquiring real-time information from a device, means for inputting the acquired information into a generating AI model and analyzing it, and means for creating a visual representation based on the analysis results. This allows for the real-time visualization of workload and progress, enabling users to instantly grasp the situation and make appropriate decisions.

[0257] "Real-time information" refers to data that is collected and processed immediately, reflecting changes in current situations and events in real time.

[0258] A "device" is an electronic device used by users to input data or receive results.

[0259] A "generative AI model" is an artificial intelligence algorithm that analyzes data and generates results or patterns that are tailored to a specific purpose.

[0260] "Analysis" is the process of understanding given data and finding meaning and patterns that are appropriate to a specific perspective or purpose.

[0261] "Visual representation" refers to presenting data content and analysis results visually in forms such as graphs and charts, and is used to aid in understanding the information.

[0262] A "human-machine interface" is a means or design for users to interact with machines or systems, enabling effective information input and output.

[0263] This invention is implemented through the collaboration of the server, terminal, and user components. The server, as the heart of the system, acquires, analyzes, and visualizes data. First, the server acquires real-time information from the devices and inputs that data into a generative AI model. The generative AI model used here is designed to analyze workload and task progress.

[0264] The server utilizes a generative AI model to analyze the acquired data. During this process, it is given a prompt message such as, "Analyze and visualize the business data." The analysis results are generated as a visual representation, provided in the form of graphs, charts, and other visual formats. The visualized data is then transferred from the server to the terminal, making it available for user use.

[0265] The terminal serves as an interface for users to input data and receive analysis results. Users can input business data into the terminal in voice, text, or image format. The terminal interface is optimized for users to input information and review visual data from the server through a human-machine interface.

[0266] For example, a user might select "Enter the number of tasks completed this week and save" via their device to manage project progress. The server then uses this information to generate a prompt, "Analyze and visualize the progress," which is passed to an AI model, and the results are provided to the user. This entire process allows the user to quickly grasp a wealth of information, enabling efficient work operations.

[0267] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0268] Step 1:

[0269] The server acquires real-time information from devices. The input is business data provided by the user through their terminal. The server formats this data into the correct format and stores it in a database. This process is necessary for centralized data management and supports subsequent analysis.

[0270] Step 2:

[0271] The terminal transmits data in the form of voice, text, or images input by the user to the server. While the input can vary, the output arrives at the server as structured digital data. The terminal provides preview templates through a user interface to assist with accurate information input, allowing the user to verify the data.

[0272] Step 3:

[0273] The server inputs the acquired data into a generating AI model to analyze workload and task progress. In this process, the input data is passed to the AI ​​model along with the prompt message, "Analyze and visualize the business data." The generating AI model uses various algorithms to process the data and obtain results. As a result of the analysis, specific patterns and trends are extracted.

[0274] Step 4:

[0275] The server creates visual representations based on the analysis results of the generated AI model. The analysis results serve as input, and the output is visualized data. This visual data is created in various formats (e.g., graphs and charts) to make it easy for the user to understand.

[0276] Step 5:

[0277] The server transfers the visual representation to the terminal. The output visual data reaches the terminal as input from the server. The terminal receives this data and intuitively displays it on the user interface. For example, the user uses the zoom-in / zoom-out function to check the data for data comparison and trend understanding.

[0278] Step 6:

[0279] The user makes business adjustments and decisions based on the acquired visual data through the terminal. Based on the visual data obtained as input, the user determines the priority of new tasks and the redistribution of resources. This process enables the improvement of business efficiency and effective project management.

[0280] (Application Example 1)

[0281] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0282] Monitoring and managing the operating status and workload of robots in a factory is important for maintaining efficient production. However, it is difficult to detect early and respond quickly to various abnormal situations, which may delay the progress of the business. Therefore, there is a demand for a system that visualizes the operating status of robots and quickly detects abnormalities to perform task redistribution.

[0283] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0284] In this invention, the server includes means for collecting real-time information from the device, means for inputting the collected data into the generated AI model for analysis, and means for integrating the analyzed results to generate interactive visual information. Thereby, it becomes possible to visualize the operating status and workload of robots in a factory, quickly detect abnormalities, and perform task redistribution.

[0285] "Real-time information" refers to data that can be obtained and confirmed immediately on the spot regarding the latest status and changes.

[0286] "Device" refers to a machine or mechanism designed to perform a specific function.

[0287] "Collection" refers to the act of gathering necessary information together.

[0288] "Generative AI model" refers to an artificial intelligence algorithm that learns from data and performs tasks such as prediction and classification.

[0289] "Analysis" refers to the work of decomposing and examining data to clarify its structure and relationships.

[0290] "Integration" refers to the act of combining multiple pieces of information or data into one.

[0291] "Interactive visual information" refers to visual data provided in a form that can be interactively used by the user through operation.

[0292] "Terminal" refers to a computer or device used by the user for direct operation.

[0293] "User interface" refers to the operation part for the user to interact with a computer or system.

[0294] "Operating status of the robot" refers to information indicating what kind of work and operations the robot in the factory is currently performing.

[0295] "Business load" refers to the concept representing the amount of work or processing volume required for a specific business.

[0296] "Visualization" refers to the technology of presenting data and information in a visible form such as graphs and charts.

[0297] "Task reallocation" is the act of readjusting the assignment of tasks or duties.

[0298] "Abnormal" refers to a state or operation that deviates from the normal condition or behavior.

[0299] To implement this invention, it is necessary to build a system in which a server, terminals, and users work together. The server collects real-time information on robots in the factory and analyzes this information using a generative AI model. Here, data is received from sensors mounted on each robot, and data analysis is performed using Python and TensorFlow to understand the robot's operating status and workload. The analysis results are visually integrated as a web application using Django. This visual data is sent to terminals in an interactive format, and operators can access it via smartphones or tablets.

[0300] The device provides users with a user interface that allows them to manipulate information. This interface is developed with React Native and is designed to be intuitive and easy for users to use. Through this interface, users can make quick decisions and reallocate tasks between robots as needed.

[0301] As a concrete example, suppose an abnormal vibration occurs in one of the robots on a manufacturing line. In this case, the server uses a generated AI model to detect the anomaly. The analyzed data is visualized and displayed on the terminal in real time, enabling operators to quickly detect the anomaly and formulate countermeasures.

[0302] An example of a prompt message used is, "Based on the vibration data of robot R123, detect abnormal load patterns and visually report them to the operator." This prompt ensures that the generated AI model operates correctly, enabling a mechanism to immediately detect abnormalities in the robot.

[0303] The flow of the specific process in Application Example 1 will be described using FIG. 12.

[0304] Step 1:

[0305] The server collects real-time information from the sensors installed on each robot. The input is the data from the robot sensors, and the raw data as it is is stored in the temporary storage as the output. This data includes information on the position, operating speed, vibration, and load of the robot.

[0306] Step 2:

[0307] The server inputs the collected raw data into the generated AI model for analysis. The raw data of the robot obtained in Step 1 is used as the input. The generated AI model is constructed using TensorFlow and performs feature extraction of the data and detection of abnormal patterns. The output is the analysis result regarding the operating status and workload of each robot.

[0308] Step 3:

[0309] The server integrates the analysis results and generates interactive visual information. The input is the analysis result output in Step 2, and this is visualized by incorporating it into a web application using Django. As the output, graphs and charts that can be viewed by the user on the terminal are generated.

[0310] Step 4:

[0311] The terminal receives the visual information sent from the server and displays it via the user interface. The input is the visual data generated in Step 3, and the output is an interactive dashboard displayed on the screen of the terminal. Here, the user can manipulate the data and view the details.

[0312] Step 5:

[0313] The user reassigns tasks to the robot based on the information displayed on the terminal. The input is the visual information obtained in step 4 and the user's judgment, and the output is the new instructions for the robot. If necessary, prompts can be used to request the generating AI model to investigate anomalies in detail.

[0314] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0315] This invention provides a system that visualizes the user's workload and task progress in real time, and further recognizes and incorporates the user's emotions into the system, thereby achieving more precise work management. The following describes each element.

[0316] server

[0317] The server plays a central role in the system, receiving and integrating real-time and sentiment data transmitted from terminals. The received data is input into a generative AI model for analysis. This results in the generation of visual data that takes into account workload fluctuations, task progress, and even user emotions. The server then transmits this visual data to terminals and controls its display on the user interface.

[0318] terminal

[0319] The terminal is a device for users to input business information and emotional data. The information entered by the user is acquired as text, audio, or images. In particular, an emotion engine is used to extract the user's emotional state from the audio data. The terminal sends the acquired information to a server and provides a user interface for displaying the visual data received from the server.

[0320] User

[0321] Users input their daily workload and task status into the system, and use the resulting data to improve work efficiency. Users can operate a dashboard displayed on their terminal to filter information for specific periods or team members. Furthermore, they can reduce stress and proceed with their work by referring to the system's task reallocation suggestions based on user sentiment.

[0322] Specific example

[0323] For example, if a user is behind schedule on a project, the system sends daily emotional inputs to the server via voice data. The server analyzes this data using an emotion engine and suggests that the user may be experiencing stress. As a result, alerts that take the user's workload into account are displayed on the dashboard, allowing the project leader to re-evaluate task priorities and allocate resources appropriately.

[0324] Thus, the present invention enables not only direct management of workload but also advanced work management that takes into account the emotional state of users, thereby contributing to the realization of a better work environment.

[0325] The following describes the processing flow.

[0326] Step 1:

[0327] Users input information about their workload and task progress into the terminal. They can also express emotions using voice input as needed. The voice is converted into digital data by the terminal.

[0328] Step 2:

[0329] The terminal converts the user's input data into a format and sends it to the server. This data includes text data, voice data, and indicators of emotion.

[0330] Step 3:

[0331] The server records the data received from the terminal into a database. During this process, it performs checks to ensure data integrity.

[0332] Step 4:

[0333] The server inputs stored data into a generating AI model to analyze workload and task progress. Furthermore, it inputs voice data into an emotion engine to analyze the user's emotional state.

[0334] Step 5:

[0335] The server integrates the analysis results of the generative AI model and the emotion engine to generate visual data. This data visually represents workload, task progress, and emotional state.

[0336] Step 6:

[0337] The server sends the generated visual data to the terminal. On the terminal, the data is displayed on a dashboard and can be interactively manipulated through the user interface.

[0338] Step 7:

[0339] Users can use the dashboard on their device to check their workload, task progress, and sentiment analysis results. They can use the filtering function to narrow down the information as needed.

[0340] Step 8:

[0341] Based on the system's suggestions derived from emotional data, users can adjust their work priorities and optimize task allocation. This allows users to work more efficiently while reducing stress.

[0342] (Example 2)

[0343] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0344] In the workplace, it is essential to accurately understand and efficiently manage workloads, task progress, and employee emotional states. In particular, integrating and visualizing this information in real time is crucial for managers to make appropriate decisions. However, current systems struggle to process and integrate this information quickly and effectively, resulting in insufficient work efficiency and adequate employee stress management.

[0345] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0346] In this invention, the server includes a device for aggregating time-series information from data processing devices, a device for inputting and analyzing the aggregated information into AI technology, and a device for integrating the analyzed output and generating visualization information. This enables accurate, real-time understanding of workload, task progress, and user sentiment, allowing for rapid and effective business management.

[0347] A "data processing device" is a device that allows users to input data, stores information, and transmits it to a server.

[0348] "Time-series information" refers to data collected at regular time intervals, representing the user's work situation and emotional state.

[0349] "Generative AI technology" refers to artificial intelligence technology used to analyze input data and identify specific predictions or patterns.

[0350] "Visualized information" refers to information that represents analyzed data in visual forms such as graphs and charts, and is presented in a way that is easy for users to understand intuitively.

[0351] A "user interface" is a screen that provides an interface for users to manipulate information and obtain necessary data.

[0352] This invention provides a system that visualizes a user's workload and task progress in real time, and optimizes work management by recognizing the user's emotions. This system operates using a combination of a server and terminals. The following describes its embodiments.

[0353] The terminal is a device for users to input business information and emotional data. The terminal is responsible for text input, voice recording, and image data capture, and transmits this data to the server. For voice data, an emotion engine is used to identify the user's emotions from the voice and analyze them as numerical data.

[0354] The server is the central hardware of this system. The server receives data sent from terminals and inputs it into a generative AI model. The generative AI model is built using cloud services such as Microsoft Azure and Google Cloud Platform. This allows for a comprehensive analysis of workload, task progress, and user emotional states. The server integrates and visualizes these analysis results, sending them to terminals as visually easy-to-understand data.

[0355] Users can use a dashboard displayed on their device to monitor their work status in real time. The dashboard allows them to filter information by specific time periods or project members, enabling them to adjust their work accordingly. Furthermore, users can reallocate tasks and improve work efficiency based on suggestions from the system.

[0356] As a concrete example, if a user is behind schedule on a project, they input their daily emotions as voice data. The server analyzes this data using an emotion engine and displays warnings based on their stress level on a dashboard. Based on these results, the project leader can re-evaluate task priorities and allocate resources appropriately.

[0357] An example of a prompt message would be, "Please integrate task management and user sentiment data from the project to provide appropriate suggestions based on my stress level."

[0358] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0359] Step 1:

[0360] The terminal collects work information and emotional states entered by the user. Specifically, the terminal acquires data through text input, voice recording, and image capture. The input voice data is sent to the emotion engine and converted into numerical data representing the emotional state. The output consists of work information and numerical emotional data.

[0361] Step 2:

[0362] The terminal transmits collected business information and sentiment data to the server in real time. It is crucial to maintain data integrity by using a secure communication protocol. The input is the data collected in step 1, and the output is the transmission status confirming that the data successfully reached the server.

[0363] Step 3:

[0364] The server inputs data received from the terminal into the generative AI model. Specifically, this involves converting the data format into a format the model can process. The generative AI model resides in the cloud and performs advanced calculations to integrate and analyze multiple data streams. The input consists of received business information and sentiment data, while the output consists of data patterns and predictive information as a result of the analysis.

[0365] Step 4:

[0366] The server generates integrated visualization data based on the analyzed results. Specifically, it represents the data as graphs and charts, converting it into a format that is easy for users to intuitively understand. The input is the analysis results obtained in step 3, and the output is a visualized information package.

[0367] Step 5:

[0368] The server sends the generated visualization data to the terminal. At this time, it verifies that the data transfer is of high quality and prepares the terminal for display. The input is the visualized information, and the output is the display status that can be viewed on the terminal.

[0369] Step 6:

[0370] Users can monitor their work progress and emotional state through a dashboard displayed on their device, and adjust tasks as needed. Specifically, users can use the interface's filtering function to focus on specific information. The input is the data displayed on the dashboard, and the output is the user's work decisions and action plans for the next steps.

[0371] (Application Example 2)

[0372] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0373] In factory environments, it is crucial to understand workers' workload and mental stress in real time and create an efficient work environment. However, existing systems have not adequately considered workers' emotions or real-time task progress, making it difficult to adjust task priorities appropriately. Therefore, there is a need to improve work efficiency while simultaneously reducing worker stress.

[0374] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0375] In this invention, the server includes means for collecting real-time data from terminals, means for inputting the collected data into a generating AI model and analyzing it, means for integrating the analyzed results and generating interactive visual data, means for evaluating emotional states from acquired audio and image data and calculating workload, and means for making suggestions to adjust the priority of work tasks based on the generated emotional state and workload information. This makes it possible to grasp the emotions and workload of workers in real time and adjust task priorities appropriately.

[0376] "Real-time data" refers to data that instantly reflects ongoing situations and events.

[0377] "Terminal" refers to input and output devices used as part of a system.

[0378] A "generative AI model" is an artificial intelligence algorithm that generates new information or predictions based on input data.

[0379] "Analysis" is the process of processing data to reveal the information and patterns contained within it.

[0380] "Visual data" refers to information presented in a visual format and used to facilitate user understanding.

[0381] "Emotional state" refers to data that indicates an individual's emotional situation or psychological state.

[0382] "Workload" refers to the amount of effort and resources required to perform a task.

[0383] "Task priority" is an indicator that shows the importance and urgency of the tasks that need to be done.

[0384] A "proposal" refers to advice or recommendations made based on specific data or conditions.

[0385] The system for implementing this invention mainly consists of three elements: a server, a terminal, and a user. The server has the function of receiving and collecting real-time data from the terminal within the factory. This data includes audio and images. The server inputs this data into a generating AI model to evaluate emotional states and workload. A common tool as a speech recognition library is used for speech recognition, and an emotion analysis engine is used for emotion analysis. This integrates the analyzed information and generates interactive visual data that can be visually presented to the user.

[0386] The terminal is a device, such as smart glasses worn by the worker, that displays visual data transmitted from the server. This allows the worker to understand their emotional state and workload assessment in real time. To improve work efficiency, the server suggests adjusting the priority of work tasks based on the emotional state and workload information. This suggestion is displayed on the terminal via an interactive user interface, helping the worker to progress through tasks in the most optimal way.

[0387] For example, if a worker is experiencing excessive stress, the system will analyze the data and display a suggestion on the screen indicating that a short break or time to refresh is needed. An example of a prompt message would be, "Based on the progress of your current task and the sentiment analysis, please determine whether a break is necessary."

[0388] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0389] Step 1:

[0390] The user puts on smart glasses and begins working. The device captures the worker's voice and facial expressions in real time. This data is saved as audio and image files for use in the next analysis step.

[0391] Step 2:

[0392] The device converts the acquired audio data into text data using a speech recognition library. This text data is then prepared for input into the sentiment analysis engine. As a result, verbal instructions and situational descriptions are interpreted as textual information.

[0393] Step 3:

[0394] The server receives text and image data sent from the terminal. The received text data is input into the sentiment analysis engine, which outputs an emotion score to evaluate the user's emotional state. For image data, image analysis technology is used to extract additional emotional information from facial expressions.

[0395] Step 4:

[0396] The server integrates sentiment scores and image analysis results to assess the current workload. Based on this assessment, a generative AI model is used to calculate task priorities. This generative AI model refers to historical data and successful examples from similar situations to suggest the optimal plan.

[0397] Step 5:

[0398] The server visualizes integrated emotional states, workload, and task priorities, generating interactive visual data. This visual data includes specific task suggestions and break recommendations for the worker. This data is then sent to the terminal via the user interface in the next step.

[0399] Step 6:

[0400] When the terminal receives visual data transmitted from the server, it displays specific advice and suggestions to the user to improve work efficiency. The user can then view this information through the display on their glasses and adjust their work accordingly.

[0401] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0402] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0403] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0404] [Third Embodiment]

[0405] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0406] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0407] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0408] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0409] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0410] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0411] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0412] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0413] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0414] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0415] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0416] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0417] In order to implement the system of this invention, the server, terminal, and user must cooperate with each other. Their respective roles are described below.

[0418] server

[0419] The server is the central component responsible for data processing and management across the entire system. It utilizes generative AI models to analyze real-time data received from terminals. Specifically, the server centrally collects data from terminals and stores it in a database. The generative AI model then analyzes fluctuations in workload and task progress, generating visual data based on the results. The server sends these results to the user's terminal, displaying them on the interface.

[0420] terminal

[0421] The terminal is a device for users to input data and view system output. Users can input their workload and task progress through the interface on the terminal. Audio and image data formats are also supported as needed. The terminal displays visual data sent from the server using the user interface, enabling user interaction.

[0422] User

[0423] Users are the primary actors who leverage the system's output to efficiently manage tasks. They can instantly check their workload and task progress through their terminals, and use this information to reassign or adjust tasks accordingly. Users can also utilize filtering and search functions to display only information of interest. Furthermore, system improvements can be facilitated through user feedback.

[0424] Specific example

[0425] For example, consider a project team leader who wants to check the workload of team members during a busy period. The leader uses a terminal to input the team's overall work information. The server receives this data and analyzes it in real time using a generated AI model. As a result, the fluctuations in each member's workload and the progress of their tasks are visualized and instantly displayed on the leader's interface from the terminal. Based on this information, the leader can make decisions about reallocating or adjusting tasks. This series of actions contributes to improving the user's work efficiency.

[0426] Thus, the present invention enables the various components of the system to work together to improve the efficiency and productivity of the user's daily work.

[0427] The following describes the processing flow.

[0428] Step 1:

[0429] The terminal acquires information on workload and task progress entered by the user. This information is accepted in text, audio, and image formats, and appropriate preprocessing is performed for each format (for example, converting audio to text).

[0430] Step 2:

[0431] The terminal converts the acquired data into a format that can be sent to the server. For example, it converts it to JSON or XML format and sends it to the server using a secure communication protocol (e.g., HTTPS).

[0432] Step 3:

[0433] The server receives data from terminals in real time and stores it in a database for centralized management. During storage, validation and error checking are performed to maintain data integrity.

[0434] Step 4:

[0435] The server inputs new data from the database into a generative AI model for analysis. The AI ​​model analyzes patterns of workload and task progress, and outputs statistical predictions.

[0436] Step 5:

[0437] The server generates interactive visual data (graphs and charts) based on the analysis results output by the AI ​​model. This visual data is designed in a format that is easy for the user to see and understand.

[0438] Step 6:

[0439] The server sends the generated visual data to the terminal. On the terminal, the received visual data is displayed on the interface, providing an environment where the user can intuitively manipulate the data.

[0440] Step 7:

[0441] Users interact with data using a dashboard displayed on their device. This interaction includes filtering data, viewing detailed information, and specifying time periods.

[0442] Step 8:

[0443] Users can reassign tasks and adjust their workload based on the information obtained from the dashboard. This leads to improved work efficiency and a better balance of workload within the team.

[0444] (Example 1)

[0445] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0446] The challenge is to provide a system that streamlines the process from information input to analysis, visualization, and presentation to the user, effectively manages the progress and workload of tasks in real time, and enables users to make appropriate decisions quickly.

[0447] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0448] In this invention, the server includes means for acquiring real-time information from a device, means for inputting the acquired information into a generating AI model and analyzing it, and means for creating a visual representation based on the analysis results. This allows for the real-time visualization of workload and progress, enabling users to instantly grasp the situation and make appropriate decisions.

[0449] "Real-time information" refers to data that is collected and processed immediately, reflecting changes in current situations and events in real time.

[0450] A "device" is an electronic device used by users to input data or receive results.

[0451] A "generative AI model" is an artificial intelligence algorithm that analyzes data and generates results or patterns that are tailored to a specific purpose.

[0452] "Analysis" is the process of understanding given data and finding meaning and patterns that are appropriate to a specific perspective or purpose.

[0453] "Visual representation" refers to presenting data content and analysis results visually in forms such as graphs and charts, and is used to aid in understanding the information.

[0454] A "human-machine interface" is a means or design for users to interact with machines or systems, enabling effective information input and output.

[0455] This invention is implemented through the collaboration of the server, terminal, and user components. The server, as the heart of the system, acquires, analyzes, and visualizes data. First, the server acquires real-time information from the devices and inputs that data into a generative AI model. The generative AI model used here is designed to analyze workload and task progress.

[0456] The server utilizes a generative AI model to analyze the acquired data. During this process, it is given a prompt message such as, "Analyze and visualize the business data." The analysis results are generated as a visual representation, provided in the form of graphs, charts, and other visual formats. The visualized data is then transferred from the server to the terminal, making it available for user use.

[0457] The terminal serves as an interface for users to input data and receive analysis results. Users can input business data into the terminal in voice, text, or image format. The terminal interface is optimized for users to input information and review visual data from the server through a human-machine interface.

[0458] For example, a user might select "Enter the number of tasks completed this week and save" via their device to manage project progress. The server then uses this information to generate a prompt, "Analyze and visualize the progress," which is passed to an AI model, and the results are provided to the user. This entire process allows the user to quickly grasp a wealth of information, enabling efficient work operations.

[0459] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0460] Step 1:

[0461] The server acquires real-time information from devices. The input is business data provided by the user through their terminal. The server formats this data into the correct format and stores it in a database. This process is necessary for centralized data management and supports subsequent analysis.

[0462] Step 2:

[0463] The terminal transmits data in the form of voice, text, or images input by the user to the server. While the input can vary, the output arrives at the server as structured digital data. The terminal provides preview templates through a user interface to assist with accurate information input, allowing the user to verify the data.

[0464] Step 3:

[0465] The server inputs the acquired data into a generating AI model to analyze workload and task progress. In this process, the input data is passed to the AI ​​model along with the prompt message, "Analyze and visualize the business data." The generating AI model uses various algorithms to process the data and obtain results. As a result of the analysis, specific patterns and trends are extracted.

[0466] Step 4:

[0467] The server creates visual representations based on the analysis results of the generated AI model. The analysis results serve as input, and the output is visualized data. This visual data is created in various formats (e.g., graphs and charts) to make it easy for the user to understand.

[0468] Step 5:

[0469] The server transfers the visual representation to the terminal. The output visual data arrives at the terminal as input from the server. The terminal receives this data and displays it intuitively on the user interface. For example, the user can use zoom in and out functions to examine the data in order to compare data or understand trends.

[0470] Step 6:

[0471] Users adjust their work and make decisions based on the visual data acquired through their devices. Based on the visual data received as input, users determine the prioritization of new tasks and the reallocation of resources. This process enables improved work efficiency and effective project management.

[0472] (Application Example 1)

[0473] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0474] Monitoring and managing the operating status and workload of robots in factories is crucial for maintaining efficient production. However, early detection and rapid response to minor and major anomalies are difficult, potentially delaying work progress. Therefore, there is a need for a system that visualizes robot operating status, quickly detects anomalies, and reallocates tasks.

[0475] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0476] In this invention, the server includes means for collecting real-time information from the device, means for inputting the collected data into a generating AI model and analyzing it, and means for integrating the analyzed results and generating interactive visual information. This makes it possible to visualize the operating status and workload of robots in a factory, quickly detect anomalies, and reallocate tasks.

[0477] "Real-time information" refers to data that allows you to instantly acquire and verify the latest status and changes on the spot.

[0478] A "device" is a machine or mechanism designed to perform a specific function.

[0479] "Collection" is the act of gathering and accumulating necessary information.

[0480] A "generative AI model" is an artificial intelligence algorithm that learns from data and performs tasks such as prediction and classification.

[0481] "Analysis" is the process of breaking down and examining data to reveal its structure and relationships.

[0482] "Integration" is the act of combining multiple pieces of information or data to create a unified whole.

[0483] "Interactive visual information" refers to visual data that is provided in a way that allows users to interact with and use it interactively.

[0484] A "terminal" is a computer or device used by a user for direct operation.

[0485] A "user interface" is the part of a computer or system that allows a user to interact with it.

[0486] "Robot operating status" refers to information that shows what tasks or actions robots in a factory are currently performing.

[0487] "Workload" is a concept that represents the amount of work or processing required for a particular task.

[0488] "Visualization" is a technique that makes data and information visible through graphs, charts, and other visual means.

[0489] "Task reallocation" is the act of readjusting the assignment of tasks or duties.

[0490] "Abnormal" refers to a state or operation that deviates from the normal condition or behavior.

[0491] To implement this invention, it is necessary to build a system in which a server, terminals, and users work together. The server collects real-time information on robots in the factory and analyzes this information using a generative AI model. Here, data is received from sensors mounted on each robot, and data analysis is performed using Python and TensorFlow to understand the robot's operating status and workload. The analysis results are visually integrated as a web application using Django. This visual data is sent to terminals in an interactive format, and operators can access it via smartphones or tablets.

[0492] The device provides users with a user interface that allows them to manipulate information. This interface is developed with React Native and is designed to be intuitive and easy for users to use. Through this interface, users can make quick decisions and reallocate tasks between robots as needed.

[0493] As a concrete example, suppose an abnormal vibration occurs in one of the robots on a manufacturing line. In this case, the server uses a generated AI model to detect the anomaly. The analyzed data is visualized and displayed on the terminal in real time, enabling operators to quickly detect the anomaly and formulate countermeasures.

[0494] An example of a prompt message used is, "Based on the vibration data of robot R123, detect abnormal load patterns and visually report them to the operator." This prompt ensures that the generated AI model operates correctly, enabling a mechanism to immediately detect abnormalities in the robot.

[0495] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0496] Step 1:

[0497] The server collects real-time information from sensors mounted on each robot. The input is data from the robot's sensors, and the raw data is stored as output in temporary storage. This data includes information on the robot's position, operating speed, vibration, and load.

[0498] Step 2:

[0499] The server inputs the collected raw data into a generating AI model for analysis. The raw robot data obtained in Step 1 is used as input. The generating AI model is built using TensorFlow and performs data feature extraction and anomaly pattern detection. The output consists of analysis results regarding the operating status and workload of each robot.

[0500] Step 3:

[0501] The server integrates the analysis results and generates interactive visual information. The input is the analysis results output in step 2, which are then integrated into a web application using Django for visualization. The output consists of graphs and charts that the user can view on their device.

[0502] Step 4:

[0503] The terminal receives visual information transmitted from the server and displays it through the user interface. The input is the visual data generated in step 3, and the output is an interactive dashboard displayed on the terminal's screen. Here, the user can manipulate the data and view details.

[0504] Step 5:

[0505] The user reassigns tasks to the robot based on the information displayed on the terminal. The input is the visual information obtained in step 4 and the user's judgment, and the output is the new instructions for the robot. If necessary, prompts can be used to request the generating AI model to investigate anomalies in detail.

[0506] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0507] This invention provides a system that visualizes the user's workload and task progress in real time, and further recognizes and incorporates the user's emotions into the system, thereby achieving more precise work management. The following describes each element.

[0508] server

[0509] The server plays a central role in the system, receiving and integrating real-time and sentiment data transmitted from terminals. The received data is input into a generative AI model for analysis. This results in the generation of visual data that takes into account workload fluctuations, task progress, and even user emotions. The server then transmits this visual data to terminals and controls its display on the user interface.

[0510] terminal

[0511] The terminal is a device for users to input business information and emotional data. The information entered by the user is acquired as text, audio, or images. In particular, an emotion engine is used to extract the user's emotional state from the audio data. The terminal sends the acquired information to a server and provides a user interface for displaying the visual data received from the server.

[0512] User

[0513] Users input their daily workload and task status into the system, and use the resulting data to improve work efficiency. Users can operate a dashboard displayed on their terminal to filter information for specific periods or team members. Furthermore, they can reduce stress and proceed with their work by referring to the system's task reallocation suggestions based on user sentiment.

[0514] Specific example

[0515] For example, if a user is behind schedule on a project, the system sends daily emotional inputs to the server via voice data. The server analyzes this data using an emotion engine and suggests that the user may be experiencing stress. As a result, alerts that take the user's workload into account are displayed on the dashboard, allowing the project leader to re-evaluate task priorities and allocate resources appropriately.

[0516] Thus, the present invention enables not only direct management of workload but also advanced work management that takes into account the emotional state of users, thereby contributing to the realization of a better work environment.

[0517] The following describes the processing flow.

[0518] Step 1:

[0519] Users input information about their workload and task progress into the terminal. They can also express emotions using voice input as needed. The voice is converted into digital data by the terminal.

[0520] Step 2:

[0521] The terminal converts the user's input data into a format and sends it to the server. This data includes text data, voice data, and indicators of emotion.

[0522] Step 3:

[0523] The server records the data received from the terminal into a database. During this process, it performs checks to ensure data integrity.

[0524] Step 4:

[0525] The server inputs stored data into a generating AI model to analyze workload and task progress. Furthermore, it inputs voice data into an emotion engine to analyze the user's emotional state.

[0526] Step 5:

[0527] The server integrates the analysis results of the generative AI model and the emotion engine to generate visual data. This data visually represents workload, task progress, and emotional state.

[0528] Step 6:

[0529] The server sends the generated visual data to the terminal. On the terminal, the data is displayed on a dashboard and can be interactively manipulated through the user interface.

[0530] Step 7:

[0531] Users can use the dashboard on their device to check their workload, task progress, and sentiment analysis results. They can use the filtering function to narrow down the information as needed.

[0532] Step 8:

[0533] Based on the system's suggestions derived from emotional data, users can adjust their work priorities and optimize task allocation. This allows users to work more efficiently while reducing stress.

[0534] (Example 2)

[0535] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0536] In the workplace, it is essential to accurately understand and efficiently manage workloads, task progress, and employee emotional states. In particular, integrating and visualizing this information in real time is crucial for managers to make appropriate decisions. However, current systems struggle to process and integrate this information quickly and effectively, resulting in insufficient work efficiency and adequate employee stress management.

[0537] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0538] In this invention, the server includes a device for aggregating time-series information from data processing devices, a device for inputting and analyzing the aggregated information into AI technology, and a device for integrating the analyzed output and generating visualization information. This enables accurate, real-time understanding of workload, task progress, and user sentiment, allowing for rapid and effective business management.

[0539] A "data processing device" is a device that allows users to input data, stores information, and transmits it to a server.

[0540] "Time-series information" refers to data collected at regular time intervals, representing the user's work situation and emotional state.

[0541] "Generative AI technology" refers to artificial intelligence technology used to analyze input data and identify specific predictions or patterns.

[0542] "Visualized information" refers to information that represents analyzed data in visual forms such as graphs and charts, and is presented in a way that is easy for users to understand intuitively.

[0543] A "user interface" is a screen that provides an interface for users to manipulate information and obtain necessary data.

[0544] This invention provides a system that visualizes a user's workload and task progress in real time, and optimizes work management by recognizing the user's emotions. This system operates using a combination of a server and terminals. The following describes its embodiments.

[0545] The terminal is a device for users to input business information and emotional data. The terminal is responsible for text input, voice recording, and image data capture, and transmits this data to the server. For voice data, an emotion engine is used to identify the user's emotions from the voice and analyze them as numerical data.

[0546] The server is the central hardware of this system. The server receives data sent from terminals and inputs it into a generative AI model. The generative AI model is built using cloud services such as Microsoft Azure and Google Cloud Platform. This allows for a comprehensive analysis of workload, task progress, and user emotional states. The server integrates and visualizes these analysis results, sending them to terminals as visually easy-to-understand data.

[0547] Users can use a dashboard displayed on their device to monitor their work status in real time. The dashboard allows them to filter information by specific time periods or project members, enabling them to adjust their work accordingly. Furthermore, users can reallocate tasks and improve work efficiency based on suggestions from the system.

[0548] As a concrete example, if a user is behind schedule on a project, they input their daily emotions as voice data. The server analyzes this data using an emotion engine and displays warnings based on their stress level on a dashboard. Based on these results, the project leader can re-evaluate task priorities and allocate resources appropriately.

[0549] An example of a prompt message would be, "Please integrate task management and user sentiment data from the project to provide appropriate suggestions based on my stress level."

[0550] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0551] Step 1:

[0552] The terminal collects work information and emotional states entered by the user. Specifically, the terminal acquires data through text input, voice recording, and image capture. The input voice data is sent to the emotion engine and converted into numerical data representing the emotional state. The output consists of work information and numerical emotional data.

[0553] Step 2:

[0554] The terminal transmits collected business information and sentiment data to the server in real time. It is crucial to maintain data integrity by using a secure communication protocol. The input is the data collected in step 1, and the output is the transmission status confirming that the data successfully reached the server.

[0555] Step 3:

[0556] The server inputs data received from the terminal into the generative AI model. Specifically, this involves converting the data format into a format the model can process. The generative AI model resides in the cloud and performs advanced calculations to integrate and analyze multiple data streams. The input consists of received business information and sentiment data, while the output consists of data patterns and predictive information as a result of the analysis.

[0557] Step 4:

[0558] The server generates integrated visualization data based on the analyzed results. Specifically, it represents the data as graphs and charts, converting it into a format that is easy for users to intuitively understand. The input is the analysis results obtained in step 3, and the output is a visualized information package.

[0559] Step 5:

[0560] The server sends the generated visualization data to the terminal. At this time, it verifies that the data transfer is of high quality and prepares the terminal for display. The input is the visualized information, and the output is the display status that can be viewed on the terminal.

[0561] Step 6:

[0562] Users can monitor their work progress and emotional state through a dashboard displayed on their device, and adjust tasks as needed. Specifically, users can use the interface's filtering function to focus on specific information. The input is the data displayed on the dashboard, and the output is the user's work decisions and action plans for the next steps.

[0563] (Application Example 2)

[0564] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0565] In factory environments, it is crucial to understand workers' workload and mental stress in real time and create an efficient work environment. However, existing systems have not adequately considered workers' emotions or real-time task progress, making it difficult to adjust task priorities appropriately. Therefore, there is a need to improve work efficiency while simultaneously reducing worker stress.

[0566] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0567] In this invention, the server includes means for collecting real-time data from terminals, means for inputting the collected data into a generating AI model and analyzing it, means for integrating the analyzed results and generating interactive visual data, means for evaluating emotional states from acquired audio and image data and calculating workload, and means for making suggestions to adjust the priority of work tasks based on the generated emotional state and workload information. This makes it possible to grasp the emotions and workload of workers in real time and adjust task priorities appropriately.

[0568] "Real-time data" refers to data that instantly reflects ongoing situations and events.

[0569] "Terminal" refers to input and output devices used as part of a system.

[0570] A "generative AI model" is an artificial intelligence algorithm that generates new information or predictions based on input data.

[0571] "Analysis" is the process of processing data to reveal the information and patterns contained within it.

[0572] "Visual data" refers to information presented in a visual format and used to facilitate user understanding.

[0573] "Emotional state" refers to data that indicates an individual's emotional situation or psychological state.

[0574] "Workload" refers to the amount of effort and resources required to perform a task.

[0575] "Task priority" is an indicator that shows the importance and urgency of the tasks that need to be done.

[0576] A "proposal" refers to advice or recommendations made based on specific data or conditions.

[0577] The system for implementing this invention mainly consists of three elements: a server, a terminal, and a user. The server has the function of receiving and collecting real-time data from the terminal within the factory. This data includes audio and images. The server inputs this data into a generating AI model to evaluate emotional states and workload. A common tool as a speech recognition library is used for speech recognition, and an emotion analysis engine is used for emotion analysis. This integrates the analyzed information and generates interactive visual data that can be visually presented to the user.

[0578] The terminal is a device, such as smart glasses worn by the worker, that displays visual data transmitted from the server. This allows the worker to understand their emotional state and workload assessment in real time. To improve work efficiency, the server suggests adjusting the priority of work tasks based on the emotional state and workload information. This suggestion is displayed on the terminal via an interactive user interface, helping the worker to progress through tasks in the most optimal way.

[0579] For example, if a worker is experiencing excessive stress, the system will analyze the data and display a suggestion on the screen indicating that a short break or time to refresh is needed. An example of a prompt message would be, "Based on the progress of your current task and the sentiment analysis, please determine whether a break is necessary."

[0580] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0581] Step 1:

[0582] The user puts on smart glasses and begins working. The device captures the worker's voice and facial expressions in real time. This data is saved as audio and image files for use in the next analysis step.

[0583] Step 2:

[0584] The device converts the acquired audio data into text data using a speech recognition library. This text data is then prepared for input into the sentiment analysis engine. As a result, verbal instructions and situational descriptions are interpreted as textual information.

[0585] Step 3:

[0586] The server receives text and image data sent from the terminal. The received text data is input into the sentiment analysis engine, which outputs an emotion score to evaluate the user's emotional state. For image data, image analysis technology is used to extract additional emotional information from facial expressions.

[0587] Step 4:

[0588] The server integrates sentiment scores and image analysis results to assess the current workload. Based on this assessment, a generative AI model is used to calculate task priorities. This generative AI model refers to historical data and successful examples from similar situations to suggest the optimal plan.

[0589] Step 5:

[0590] The server visualizes integrated emotional states, workload, and task priorities, generating interactive visual data. This visual data includes specific task suggestions and break recommendations for the worker. This data is then sent to the terminal via the user interface in the next step.

[0591] Step 6:

[0592] When the terminal receives visual data transmitted from the server, it displays specific advice and suggestions to the user to improve work efficiency. The user can then view this information through the display on their glasses and adjust their work accordingly.

[0593] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0594] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0595] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0596] [Fourth Embodiment]

[0597] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0598] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0599] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0600] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0601] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0602] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0603] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0604] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0605] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0606] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0607] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0608] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0609] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0610] In order to implement the system of this invention, the server, terminal, and user must cooperate with each other. Their respective roles are described below.

[0611] server

[0612] The server is the central component responsible for data processing and management across the entire system. It utilizes generative AI models to analyze real-time data received from terminals. Specifically, the server centrally collects data from terminals and stores it in a database. The generative AI model then analyzes fluctuations in workload and task progress, generating visual data based on the results. The server sends these results to the user's terminal, displaying them on the interface.

[0613] terminal

[0614] The terminal is a device for users to input data and view system output. Users can input their workload and task progress through the interface on the terminal. Audio and image data formats are also supported as needed. The terminal displays visual data sent from the server using the user interface, enabling user interaction.

[0615] User

[0616] Users are the primary actors who leverage the system's output to efficiently manage tasks. They can instantly check their workload and task progress through their terminals, and use this information to reassign or adjust tasks accordingly. Users can also utilize filtering and search functions to display only information of interest. Furthermore, system improvements can be facilitated through user feedback.

[0617] Specific example

[0618] For example, consider a project team leader who wants to check the workload of team members during a busy period. The leader uses a terminal to input the team's overall work information. The server receives this data and analyzes it in real time using a generated AI model. As a result, the fluctuations in each member's workload and the progress of their tasks are visualized and instantly displayed on the leader's interface from the terminal. Based on this information, the leader can make decisions about reallocating or adjusting tasks. This series of actions contributes to improving the user's work efficiency.

[0619] Thus, the present invention enables the various components of the system to work together to improve the efficiency and productivity of the user's daily work.

[0620] The following describes the processing flow.

[0621] Step 1:

[0622] The terminal acquires information on workload and task progress entered by the user. This information is accepted in text, audio, and image formats, and appropriate preprocessing is performed for each format (for example, converting audio to text).

[0623] Step 2:

[0624] The terminal converts the acquired data into a format that can be sent to the server. For example, it converts it to JSON or XML format and sends it to the server using a secure communication protocol (e.g., HTTPS).

[0625] Step 3:

[0626] The server receives data from terminals in real time and stores it in a database for centralized management. During storage, validation and error checking are performed to maintain data integrity.

[0627] Step 4:

[0628] The server inputs new data from the database into a generative AI model for analysis. The AI ​​model analyzes patterns of workload and task progress, and outputs statistical predictions.

[0629] Step 5:

[0630] The server generates interactive visual data (graphs and charts) based on the analysis results output by the AI ​​model. This visual data is designed in a format that is easy for the user to see and understand.

[0631] Step 6:

[0632] The server sends the generated visual data to the terminal. On the terminal, the received visual data is displayed on the interface, providing an environment where the user can intuitively manipulate the data.

[0633] Step 7:

[0634] Users interact with data using a dashboard displayed on their device. This interaction includes filtering data, viewing detailed information, and specifying time periods.

[0635] Step 8:

[0636] Users can reassign tasks and adjust their workload based on the information obtained from the dashboard. This leads to improved work efficiency and a better balance of workload within the team.

[0637] (Example 1)

[0638] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0639] The challenge is to provide a system that streamlines the process from information input to analysis, visualization, and presentation to the user, effectively manages the progress and workload of tasks in real time, and enables users to make appropriate decisions quickly.

[0640] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0641] In this invention, the server includes means for acquiring real-time information from a device, means for inputting the acquired information into a generating AI model and analyzing it, and means for creating a visual representation based on the analysis results. This allows for the real-time visualization of workload and progress, enabling users to instantly grasp the situation and make appropriate decisions.

[0642] "Real-time information" refers to data that is collected and processed immediately, reflecting changes in current situations and events in real time.

[0643] A "device" is an electronic device used by users to input data or receive results.

[0644] A "generative AI model" is an artificial intelligence algorithm that analyzes data and generates results or patterns that are tailored to a specific purpose.

[0645] "Analysis" is the process of understanding given data and finding meaning and patterns that are appropriate to a specific perspective or purpose.

[0646] "Visual representation" refers to presenting data content and analysis results visually in forms such as graphs and charts, and is used to aid in understanding the information.

[0647] A "human-machine interface" is a means or design for users to interact with machines or systems, enabling effective information input and output.

[0648] This invention is implemented through the collaboration of the server, terminal, and user components. The server, as the heart of the system, acquires, analyzes, and visualizes data. First, the server acquires real-time information from the devices and inputs that data into a generative AI model. The generative AI model used here is designed to analyze workload and task progress.

[0649] The server utilizes a generative AI model to analyze the acquired data. During this process, it is given a prompt message such as, "Analyze and visualize the business data." The analysis results are generated as a visual representation, provided in the form of graphs, charts, and other visual formats. The visualized data is then transferred from the server to the terminal, making it available for user use.

[0650] The terminal serves as an interface for users to input data and receive analysis results. Users can input business data into the terminal in voice, text, or image format. The terminal interface is optimized for users to input information and review visual data from the server through a human-machine interface.

[0651] For example, a user might select "Enter the number of tasks completed this week and save" via their device to manage project progress. The server then uses this information to generate a prompt, "Analyze and visualize the progress," which is passed to an AI model, and the results are provided to the user. This entire process allows the user to quickly grasp a wealth of information, enabling efficient work operations.

[0652] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0653] Step 1:

[0654] The server acquires real-time information from devices. The input is business data provided by the user through their terminal. The server formats this data into the correct format and stores it in a database. This process is necessary for centralized data management and supports subsequent analysis.

[0655] Step 2:

[0656] The terminal transmits data in the form of voice, text, or images input by the user to the server. While the input can vary, the output arrives at the server as structured digital data. The terminal provides preview templates through a user interface to assist with accurate information input, allowing the user to verify the data.

[0657] Step 3:

[0658] The server inputs the acquired data into a generating AI model to analyze workload and task progress. In this process, the input data is passed to the AI ​​model along with the prompt message, "Analyze and visualize the business data." The generating AI model uses various algorithms to process the data and obtain results. As a result of the analysis, specific patterns and trends are extracted.

[0659] Step 4:

[0660] The server creates visual representations based on the analysis results of the generated AI model. The analysis results serve as input, and the output is visualized data. This visual data is created in various formats (e.g., graphs and charts) to make it easy for the user to understand.

[0661] Step 5:

[0662] The server transfers the visual representation to the terminal. The output visual data arrives at the terminal as input from the server. The terminal receives this data and displays it intuitively on the user interface. For example, the user can use zoom in and out functions to examine the data in order to compare data or understand trends.

[0663] Step 6:

[0664] Users adjust their work and make decisions based on the visual data acquired through their devices. Based on the visual data received as input, users determine the prioritization of new tasks and the reallocation of resources. This process enables improved work efficiency and effective project management.

[0665] (Application Example 1)

[0666] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0667] Monitoring and managing the operating status and workload of robots in factories is crucial for maintaining efficient production. However, early detection and rapid response to minor and major anomalies are difficult, potentially delaying work progress. Therefore, there is a need for a system that visualizes robot operating status, quickly detects anomalies, and reallocates tasks.

[0668] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0669] In this invention, the server includes means for collecting real-time information from the device, means for inputting the collected data into a generating AI model and analyzing it, and means for integrating the analyzed results and generating interactive visual information. This makes it possible to visualize the operating status and workload of robots in a factory, quickly detect anomalies, and reallocate tasks.

[0670] "Real-time information" refers to data that allows you to instantly acquire and verify the latest status and changes on the spot.

[0671] A "device" is a machine or mechanism designed to perform a specific function.

[0672] "Collection" is the act of gathering and accumulating necessary information.

[0673] A "generative AI model" is an artificial intelligence algorithm that learns from data and performs tasks such as prediction and classification.

[0674] "Analysis" is the process of breaking down and examining data to reveal its structure and relationships.

[0675] "Integration" is the act of combining multiple pieces of information or data to create a unified whole.

[0676] "Interactive visual information" refers to visual data that is provided in a way that allows users to interact with and use it interactively.

[0677] A "terminal" is a computer or device used by a user for direct operation.

[0678] A "user interface" is the part of a computer or system that allows a user to interact with it.

[0679] "Robot operating status" refers to information that shows what tasks or actions robots in a factory are currently performing.

[0680] "Workload" is a concept that represents the amount of work or processing required for a particular task.

[0681] "Visualization" is a technique that makes data and information visible through graphs, charts, and other visual means.

[0682] "Task reallocation" is the act of readjusting the assignment of tasks or duties.

[0683] "Abnormal" refers to a state or operation that deviates from the normal condition or behavior.

[0684] To implement this invention, it is necessary to build a system in which a server, terminals, and users work together. The server collects real-time information on robots in the factory and analyzes this information using a generative AI model. Here, data is received from sensors mounted on each robot, and data analysis is performed using Python and TensorFlow to understand the robot's operating status and workload. The analysis results are visually integrated as a web application using Django. This visual data is sent to terminals in an interactive format, and operators can access it via smartphones or tablets.

[0685] The device provides users with a user interface that allows them to manipulate information. This interface is developed with React Native and is designed to be intuitive and easy for users to use. Through this interface, users can make quick decisions and reallocate tasks between robots as needed.

[0686] As a concrete example, suppose an abnormal vibration occurs in one of the robots on a manufacturing line. In this case, the server uses a generated AI model to detect the anomaly. The analyzed data is visualized and displayed on the terminal in real time, enabling operators to quickly detect the anomaly and formulate countermeasures.

[0687] An example of a prompt message used is, "Based on the vibration data of robot R123, detect abnormal load patterns and visually report them to the operator." This prompt ensures that the generated AI model operates correctly, enabling a mechanism to immediately detect abnormalities in the robot.

[0688] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0689] Step 1:

[0690] The server collects real-time information from sensors mounted on each robot. The input is data from the robot's sensors, and the raw data is stored as output in temporary storage. This data includes information on the robot's position, operating speed, vibration, and load.

[0691] Step 2:

[0692] The server inputs the collected raw data into a generating AI model for analysis. The raw robot data obtained in Step 1 is used as input. The generating AI model is built using TensorFlow and performs data feature extraction and anomaly pattern detection. The output consists of analysis results regarding the operating status and workload of each robot.

[0693] Step 3:

[0694] The server integrates the analysis results and generates interactive visual information. The input is the analysis results output in step 2, which are then integrated into a web application using Django for visualization. The output consists of graphs and charts that the user can view on their device.

[0695] Step 4:

[0696] The terminal receives visual information transmitted from the server and displays it through the user interface. The input is the visual data generated in step 3, and the output is an interactive dashboard displayed on the terminal's screen. Here, the user can manipulate the data and view details.

[0697] Step 5:

[0698] The user reassigns tasks to the robot based on the information displayed on the terminal. The input is the visual information obtained in step 4 and the user's judgment, and the output is the new instructions for the robot. If necessary, prompts can be used to request the generating AI model to investigate anomalies in detail.

[0699] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0700] This invention provides a system that visualizes the user's workload and task progress in real time, and further recognizes and incorporates the user's emotions into the system, thereby achieving more precise work management. The following describes each element.

[0701] server

[0702] The server plays a central role in the system, receiving and integrating real-time and sentiment data transmitted from terminals. The received data is input into a generative AI model for analysis. This results in the generation of visual data that takes into account workload fluctuations, task progress, and even user emotions. The server then transmits this visual data to terminals and controls its display on the user interface.

[0703] terminal

[0704] The terminal is a device for users to input business information and emotional data. The information entered by the user is acquired as text, audio, or images. In particular, an emotion engine is used to extract the user's emotional state from the audio data. The terminal sends the acquired information to a server and provides a user interface for displaying the visual data received from the server.

[0705] User

[0706] Users input their daily workload and task status into the system, and use the resulting data to improve work efficiency. Users can operate a dashboard displayed on their terminal to filter information for specific periods or team members. Furthermore, they can reduce stress and proceed with their work by referring to the system's task reallocation suggestions based on user sentiment.

[0707] Specific example

[0708] For example, if a user is behind schedule on a project, the system sends daily emotional inputs to the server via voice data. The server analyzes this data using an emotion engine and suggests that the user may be experiencing stress. As a result, alerts that take the user's workload into account are displayed on the dashboard, allowing the project leader to re-evaluate task priorities and allocate resources appropriately.

[0709] Thus, the present invention enables not only direct management of workload but also advanced work management that takes into account the emotional state of users, thereby contributing to the realization of a better work environment.

[0710] The following describes the processing flow.

[0711] Step 1:

[0712] Users input information about their workload and task progress into the terminal. They can also express emotions using voice input as needed. The voice is converted into digital data by the terminal.

[0713] Step 2:

[0714] The terminal converts the user's input data into a format and sends it to the server. This data includes text data, voice data, and indicators of emotion.

[0715] Step 3:

[0716] The server records the data received from the terminal into a database. During this process, it performs checks to ensure data integrity.

[0717] Step 4:

[0718] The server inputs stored data into a generating AI model to analyze workload and task progress. Furthermore, it inputs voice data into an emotion engine to analyze the user's emotional state.

[0719] Step 5:

[0720] The server integrates the analysis results of the generative AI model and the emotion engine to generate visual data. This data visually represents workload, task progress, and emotional state.

[0721] Step 6:

[0722] The server sends the generated visual data to the terminal. On the terminal, the data is displayed on a dashboard and can be interactively manipulated through the user interface.

[0723] Step 7:

[0724] Users can use the dashboard on their device to check their workload, task progress, and sentiment analysis results. They can use the filtering function to narrow down the information as needed.

[0725] Step 8:

[0726] Based on the system's suggestions derived from emotional data, users can adjust their work priorities and optimize task allocation. This allows users to work more efficiently while reducing stress.

[0727] (Example 2)

[0728] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0729] In the workplace, it is essential to accurately understand and efficiently manage workloads, task progress, and employee emotional states. In particular, integrating and visualizing this information in real time is crucial for managers to make appropriate decisions. However, current systems struggle to process and integrate this information quickly and effectively, resulting in insufficient work efficiency and adequate employee stress management.

[0730] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0731] In this invention, the server includes a device for aggregating time-series information from data processing devices, a device for inputting and analyzing the aggregated information into AI technology, and a device for integrating the analyzed output and generating visualization information. This enables accurate, real-time understanding of workload, task progress, and user sentiment, allowing for rapid and effective business management.

[0732] A "data processing device" is a device that allows users to input data, stores information, and transmits it to a server.

[0733] "Time-series information" refers to data collected at regular time intervals, representing the user's work situation and emotional state.

[0734] "Generative AI technology" refers to artificial intelligence technology used to analyze input data and identify specific predictions or patterns.

[0735] "Visualized information" refers to information that represents analyzed data in visual forms such as graphs and charts, and is presented in a way that is easy for users to understand intuitively.

[0736] A "user interface" is a screen that provides an interface for users to manipulate information and obtain necessary data.

[0737] This invention provides a system that visualizes a user's workload and task progress in real time, and optimizes work management by recognizing the user's emotions. This system operates using a combination of a server and terminals. The following describes its embodiments.

[0738] The terminal is a device for users to input business information and emotional data. The terminal is responsible for text input, voice recording, and image data capture, and transmits this data to the server. For voice data, an emotion engine is used to identify the user's emotions from the voice and analyze them as numerical data.

[0739] The server is the central hardware of this system. The server receives data sent from terminals and inputs it into a generative AI model. The generative AI model is built using cloud services such as Microsoft Azure and Google Cloud Platform. This allows for a comprehensive analysis of workload, task progress, and user emotional states. The server integrates and visualizes these analysis results, sending them to terminals as visually easy-to-understand data.

[0740] Users can use a dashboard displayed on their device to monitor their work status in real time. The dashboard allows them to filter information by specific time periods or project members, enabling them to adjust their work accordingly. Furthermore, users can reallocate tasks and improve work efficiency based on suggestions from the system.

[0741] As a concrete example, if a user is behind schedule on a project, they input their daily emotions as voice data. The server analyzes this data using an emotion engine and displays warnings based on their stress level on a dashboard. Based on these results, the project leader can re-evaluate task priorities and allocate resources appropriately.

[0742] An example of a prompt message would be, "Please integrate task management and user sentiment data from the project to provide appropriate suggestions based on my stress level."

[0743] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0744] Step 1:

[0745] The terminal collects work information and emotional states entered by the user. Specifically, the terminal acquires data through text input, voice recording, and image capture. The input voice data is sent to the emotion engine and converted into numerical data representing the emotional state. The output consists of work information and numerical emotional data.

[0746] Step 2:

[0747] The terminal transmits collected business information and sentiment data to the server in real time. It is crucial to maintain data integrity by using a secure communication protocol. The input is the data collected in step 1, and the output is the transmission status confirming that the data successfully reached the server.

[0748] Step 3:

[0749] The server inputs data received from the terminal into the generative AI model. Specifically, this involves converting the data format into a format the model can process. The generative AI model resides in the cloud and performs advanced calculations to integrate and analyze multiple data streams. The input consists of received business information and sentiment data, while the output consists of data patterns and predictive information as a result of the analysis.

[0750] Step 4:

[0751] The server generates integrated visualization data based on the analyzed results. Specifically, it represents the data as graphs and charts, converting it into a format that is easy for users to intuitively understand. The input is the analysis results obtained in step 3, and the output is a visualized information package.

[0752] Step 5:

[0753] The server sends the generated visualization data to the terminal. At this time, it verifies that the data transfer is of high quality and prepares the terminal for display. The input is the visualized information, and the output is the display status that can be viewed on the terminal.

[0754] Step 6:

[0755] Users can monitor their work progress and emotional state through a dashboard displayed on their device, and adjust tasks as needed. Specifically, users can use the interface's filtering function to focus on specific information. The input is the data displayed on the dashboard, and the output is the user's work decisions and action plans for the next steps.

[0756] (Application Example 2)

[0757] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0758] In factory environments, it is crucial to understand workers' workload and mental stress in real time and create an efficient work environment. However, existing systems have not adequately considered workers' emotions or real-time task progress, making it difficult to adjust task priorities appropriately. Therefore, there is a need to improve work efficiency while simultaneously reducing worker stress.

[0759] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0760] In this invention, the server includes means for collecting real-time data from terminals, means for inputting the collected data into a generating AI model and analyzing it, means for integrating the analyzed results and generating interactive visual data, means for evaluating emotional states from acquired audio and image data and calculating workload, and means for making suggestions to adjust the priority of work tasks based on the generated emotional state and workload information. This makes it possible to grasp the emotions and workload of workers in real time and adjust task priorities appropriately.

[0761] "Real-time data" refers to data that instantly reflects ongoing situations and events.

[0762] "Terminal" refers to input and output devices used as part of a system.

[0763] A "generative AI model" is an artificial intelligence algorithm that generates new information or predictions based on input data.

[0764] "Analysis" is the process of processing data to reveal the information and patterns contained within it.

[0765] "Visual data" refers to information presented in a visual format and used to facilitate user understanding.

[0766] "Emotional state" refers to data that indicates an individual's emotional situation or psychological state.

[0767] "Workload" refers to the amount of effort and resources required to perform a task.

[0768] "Task priority" is an indicator that shows the importance and urgency of the tasks that need to be done.

[0769] A "proposal" refers to advice or recommendations made based on specific data or conditions.

[0770] The system for implementing this invention mainly consists of three elements: a server, a terminal, and a user. The server has the function of receiving and collecting real-time data from the terminal within the factory. This data includes audio and images. The server inputs this data into a generating AI model to evaluate emotional states and workload. A common tool as a speech recognition library is used for speech recognition, and an emotion analysis engine is used for emotion analysis. This integrates the analyzed information and generates interactive visual data that can be visually presented to the user.

[0771] The terminal is a device, such as smart glasses worn by the worker, that displays visual data transmitted from the server. This allows the worker to understand their emotional state and workload assessment in real time. To improve work efficiency, the server suggests adjusting the priority of work tasks based on the emotional state and workload information. This suggestion is displayed on the terminal via an interactive user interface, helping the worker to progress through tasks in the most optimal way.

[0772] For example, if a worker is experiencing excessive stress, the system will analyze the data and display a suggestion on the screen indicating that a short break or time to refresh is needed. An example of a prompt message would be, "Based on the progress of your current task and the sentiment analysis, please determine whether a break is necessary."

[0773] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0774] Step 1:

[0775] The user puts on smart glasses and begins working. The device captures the worker's voice and facial expressions in real time. This data is saved as audio and image files for use in the next analysis step.

[0776] Step 2:

[0777] The device converts the acquired audio data into text data using a speech recognition library. This text data is then prepared for input into the sentiment analysis engine. This allows verbal instructions and situational descriptions to be interpreted as textual information.

[0778] Step 3:

[0779] The server receives text and image data sent from the terminal. The received text data is input into the sentiment analysis engine, which outputs an emotion score to evaluate the user's emotional state. For image data, image analysis technology is used to extract additional emotional information from facial expressions.

[0780] Step 4:

[0781] The server integrates sentiment scores and image analysis results to assess the current workload. Based on this assessment, a generative AI model is used to calculate task priorities. This generative AI model refers to historical data and successful examples from similar situations to suggest the optimal plan.

[0782] Step 5:

[0783] The server visualizes integrated emotional states, workload, and task priorities, generating interactive visual data. This visual data includes specific task suggestions and break recommendations for the worker. This data is then sent to the terminal via the user interface in the next step.

[0784] Step 6:

[0785] When the terminal receives visual data transmitted from the server, it displays specific advice and suggestions to the user to improve work efficiency. The user can then view this information through the display on their glasses and adjust their work accordingly.

[0786] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0787] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0788] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0789] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0790] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0791] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0792] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0793] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0794] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0795] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0796] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0797] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0798] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0799] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0800] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0801] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0802] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0803] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0804] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0805] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0806] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted as being incorporated by reference.

[0807] The following is further disclosed regarding the embodiments described above.

[0808] (Claim 1)

[0809] A means of collecting real-time data from a terminal,

[0810] A means of inputting the collected data into a generating AI model and analyzing it,

[0811] A means for integrating the analyzed results and generating interactive visual data,

[0812] A means of transmitting and displaying the generated visual data on a terminal,

[0813] A means of making data manipulable through a user interface,

[0814] A system that includes this.

[0815] (Claim 2)

[0816] The system according to claim 1 for inputting workload information via voice, text, and visual means.

[0817] (Claim 3)

[0818] The system according to claim 1, which predicts the progress of a task based on the analysis results.

[0819] "Example 1"

[0820] (Claim 1)

[0821] A means of obtaining real-time information from a device,

[0822] A means of inputting the acquired information into a generating AI model and analyzing it,

[0823] A means of creating a visual representation based on the analyzed results,

[0824] A means of transferring and displaying the created visual representation on a device,

[0825] Means for making information manageable through a human-machine interface,

[0826] A system that includes this.

[0827] (Claim 2)

[0828] The system according to claim 1, which inputs workload information as audio, text, or visual materials.

[0829] (Claim 3)

[0830] The system according to claim 1, which predicts the progress of work based on the analysis results.

[0831] "Application Example 1"

[0832] (Claim 1)

[0833] A means of collecting real-time information from a device,

[0834] A means of inputting the collected data into a generating AI model and analyzing it,

[0835] A means for integrating the analyzed results and generating interactive visual information,

[0836] A means for transmitting and displaying the generated visual information on a terminal,

[0837] A means of making information manipulable through a user interface,

[0838] A means for analyzing and visualizing the operating status and workload of robots in a factory,

[0839] A means of redistributing tasks among robots based on the results,

[0840] A system that includes this.

[0841] (Claim 2)

[0842] The system according to claim 1, which allows input of workload information via voice, text, and visual means.

[0843] (Claim 3)

[0844] The system according to claim 1, which predicts the progress of a task based on the analysis results and detects abnormalities in the robot.

[0845] "Example 2 of combining an emotion engine"

[0846] (Claim 1)

[0847] A device that aggregates time-series information from data processing equipment,

[0848] A device that inputs and analyzes aggregated information into AI technology,

[0849] A device that integrates the analyzed output and generates visualization information,

[0850] A device that transmits and displays the generated visualization information to a data processing device,

[0851] A device that allows information to be controlled through a user interface,

[0852] A system that includes this.

[0853] (Claim 2)

[0854] The system according to claim 1 for inputting workload information audibly, in written form, and visually.

[0855] (Claim 3)

[0856] The system according to claim 1, which estimates the progress of work based on the analysis results.

[0857] "Application example 2 when combining with an emotional engine"

[0858] (Claim 1)

[0859] A means of collecting real-time data from a terminal,

[0860] A means of inputting the collected data into a generating AI model and analyzing it,

[0861] A means for integrating the analyzed results and generating interactive visual data,

[0862] A means of transmitting and displaying the generated visual data on a terminal,

[0863] A means of making data manipulable through a user interface,

[0864] A means for evaluating emotional state and calculating workload from acquired audio and image data,

[0865] A means for suggesting adjustments to the priority of work tasks based on generated emotional state and workload information,

[0866] A system that includes this.

[0867] (Claim 2)

[0868] The system according to claim 1 for inputting workload information via voice, text, and visual means.

[0869] (Claim 3)

[0870] The system according to claim 1, which predicts the progress of a task based on the analysis results and displays the proposed content on the user interface. [Explanation of Symbols]

[0871] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of collecting real-time data from a terminal, A means of inputting the collected data into a generating AI model and analyzing it, A means for integrating the analyzed results and generating interactive visual data, A means of transmitting and displaying the generated visual data on a terminal, A means of making data manipulable through a user interface, A system that includes this.

2. The system according to claim 1, which inputs workload information via voice, text, and visual means.

3. The system according to claim 1, which predicts the progress of a task based on the analysis results.

Images