system

Abstract

We provide the system. [Solution] A means of collecting operation logs from employees' computers, To analyze the collected operation logs, we use methods employing natural language processing and machine learning algorithms, A means of quantifying employee activities and generating reports based on analysis results, To visualize the generated report, methods using graphs and charts are available. A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of the chatbot's character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] As hybrid work styles including working from home become widespread, problems such as a decline in employee performance and opacity of efficiency have emerged. In particular, there is a lack of means to objectively evaluate individual working hours and the quality of speech during remote work, which consequently leads to problems such as a decline in employee motivation and a negative impact on the productivity of the entire organization.

Means for Solving the Problems

[0005] This invention provides a system for comprehensively collecting and analyzing employees' digital activities and quantifying their results. This system includes means for collecting operation logs from employees' computers and analyzing them using natural language processing and machine learning algorithms. Furthermore, based on the analysis results, it quantifies activities, generates reports, and uses graphs and charts to visualize them. It also collects online meeting content in real time, scores the quality and contribution of contributions, classifies work time and efficiency into specific projects and tasks, and objectively evaluates work efficiency. This enhances transparency and fairness in the workplace and enables appropriate evaluation of employee performance.

[0006] "Employee" refers to individual people who are employed by a company or organization and engage in its work.

[0007] A "computer" refers to an electronic device that automatically processes information and performs calculations.

[0008] An "operation log" refers to data that records the history of operations performed by a user on a computer.

[0009] "Natural language processing" refers to a set of technologies and methods for computers to understand, interpret, and respond to human language.

[0010] A "machine learning algorithm" refers to a method used by computers to learn patterns and rules based on data, and then make predictions and decisions.

[0011] "Analysis" refers to the act of thoroughly examining data and clarifying its structure and relationships.

[0012] "Quantification" refers to expressing things in numerical values ​​or indicators, making them measurable.

[0013] A "report" refers to a document that compiles analysis results and considerations into written form.

[0014] "Visualization" refers to the use of graphs and charts to visually represent information for easier understanding.

[0015] "Graph" refers to a chart for visually displaying data and showing the relationships between information.

[0016] "Chart" refers to a chart for visually showing the relationships between different elements and the structure of data.

[0017] "Online meeting" refers to a voice or video conference conducted over the Internet.

[0018] "Real-time" refers to the processing of events and operations almost simultaneously.

[0019] "Quality of speech" refers to a measure for evaluating the effectiveness, relevance, and constructive nature of speech content.

[0020] "Contribution degree" refers to an indicator for measuring the effective impact of participants on a certain activity or project.

[0021] "Scoring" refers to the act of evaluating or grading numerically based on specific criteria.

[0022] [[ID=三十二]]"Working hours" refers to the time actually spent on a specific task or project. [[ID=3四十一]]

[0023] "Efficiency" refers to the degree of goodness or badness of the relationship between the results towards achieving the goal and the resources required for it.

Brief Explanation of Drawings

[0024] [Figure 1] It is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] It is a conceptual diagram showing an example of the main functions of a data processing device and a smart device according to the first embodiment. [Figure 3]It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

Mode for Carrying Out the Invention

[0025] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0026] First, the language used in the following description will be explained.

[0027] In the following embodiments, the signed processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), and APU (Accelerated Processing Unit).

[0028] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0029] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0030] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0031] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0032] [First Embodiment]

[0033] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0034] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0035] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0036] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0037] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0038] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0039] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0040] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0041] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0042] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0043] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0044] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0045] To implement this invention, it is necessary to install a program on a terminal used by an employee, which will serve to comprehensively collect the employee's digital activities. The terminal will routinely record the application operations performed by the user, the text entered, and the online services connected (email, chat tools, etc.), and transfer this data to a server in accordance with security protocols.

[0046] The server preprocesses the received data, performing data cleaning and format conversion as needed. Through this process, it analyzes the data using natural language processing techniques and machine learning algorithms. Natural language processing allows for the extraction of useful information from text data, such as identifying the emotions and themes of conversations.

[0047] The server also quantifies the data obtained through analysis and numerically evaluates each employee's performance. In online meetings, it analyzes the number of contributions and their constructiveness, and scores their contributions based on this. This makes it possible to clearly demonstrate the specific impact on projects and tasks.

[0048] Users can visualize the data stored on the server through the dashboard. Here, they can visually check individual activity history, efficiency scores, and the overall team performance. Based on this information, administrators can evaluate employees, and employees themselves can receive feedback to improve their work.

[0049] For example, if a user's contributions to a project primarily stem from their participation in remote meetings over a week, their frequency and quality of contributions will be highly valued and displayed as the best contribution on the dashboard. This allows users to receive tangible recognition in their work environment, leading to increased motivation.

[0050] This invention provides a system that comprehensively analyzes various aspects of digital activities and functions as a tool for evaluating employee performance, thus meeting the modern needs of promoting work style reform.

[0051] The following describes the processing flow.

[0052] Step 1:

[0053] The device records the user's access history to applications and web services in real time. During this process, information such as the type of operation, execution time, and details of the action are collected as logs.

[0054] Step 2:

[0055] The terminal sends collected log data to the server at regular intervals. Secure communication protocols are used for data transfer, and privacy protection is taken into consideration.

[0056] Step 3:

[0057] The server stores the received data in the database and simultaneously performs preprocessing. This involves cleaning and formatting the text data, preparing it for analysis.

[0058] Step 4:

[0059] The server applies natural language processing techniques to the pre-processed data, performing sentiment analysis and keyword extraction. This allows for a summary of the intent and content of the user's statements and inputs.

[0060] Step 5:

[0061] The server uses machine learning algorithms to analyze specific patterns and trends, and numerically evaluates contributions to work and the quality of contributions. This involves model-based analysis using historical data and rule-based analysis.

[0062] Step 6:

[0063] The server aggregates the analysis results and quantifies the allocation of work time, efficiency, and contribution for each user. This generates baseline data for evaluating individual performance.

[0064] Step 7:

[0065] Users access a dashboard to view their performance data. The server displays graphs and charts on this dashboard in a visually easy-to-understand format and provides them to the user.

[0066] This series of steps allows the system to analyze employees' digital activities in detail and perform objective evaluations.

[0067] (Example 1)

[0068] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0069] Traditional systems made it difficult to comprehensively and efficiently collect and analyze employees' digital activities. In particular, they could not accurately evaluate the quality and contribution of online meetings, and there was a lack of means to provide objective evaluation criteria. As a result, the evaluation of employee activities and the improvement of work efficiency were delayed, which was a challenge.

[0070] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0071] In this invention, the server includes means for collecting user activity from the operating environment, means for processing and organizing the collected activity data, and means for analyzing the data by combining natural language processing and machine learning algorithms to extract useful information. This makes it possible to comprehensively evaluate employees' digital activities and objectively assess the quality and contribution of their participation in online meetings.

[0072] "Operating environment" refers to the collective term for the computers, related software, and networks that users use when performing their work or tasks.

[0073] "User activity" refers to all digital actions performed by a user on a computer, including actions such as typing, clicking, and application usage history.

[0074] "Processing and organizing operational activity data" refers to a series of steps that involve cleaning up the collected raw data and converting it into a format that is easy to analyze.

[0075] "Natural language processing" refers to the technology of computational algorithms that enable computers to understand and analyze human language.

[0076] A "machine learning algorithm" is a mathematical model that uses large amounts of data to enable computers to improve themselves and perform efficient analysis and prediction.

[0077] "Extracting useful information" is the process of finding important indicators and trends from analyzed data that can be used to improve business operations and make decisions.

[0078] "Activity evaluation" is a method that aims to objectively assess the efficiency and effectiveness of employee performance in carrying out tasks by quantifying their performance.

[0079] "Quality and contribution of contributions in online meetings" refers to a standard for quantitatively measuring the importance and influence of what is said during a meeting.

[0080] To implement this system, a specific program must first be installed on the terminal. This program comprehensively collects operational activity data when the user is using the terminal and records activities related to daily work. The collected data is periodically sent to the server using secure communication. In this process, the terminal ensures the security of the data by using encrypted communication such as HTTPS.

[0081] The server then stores this collected data in a database and performs preprocessing. Specifically, it uses scripting languages ​​such as Python to clean and format the data. The prepared dataset is then further analyzed using natural language processing techniques and machine learning algorithms.

[0082] The server uses generative AI models to extract useful information from text data. Natural language processing enables sentiment analysis and theme identification of the text. For example, it can provide analysis results on the degree of positivity and negativity in conversation content.

[0083] Based on the analysis results, the server quantifies and evaluates user performance. By analyzing the number of contributions and their constructiveness in online meetings, the server can output a score representing the user's contribution. This evaluation result is visually displayed to the user through a dashboard. Users can easily check their own efficiency score and the overall performance of their team.

[0084] For example, if a user actively participates in a remote meeting and their contributions are highly valued for the project, they may be displayed on the dashboard as having the highest level of contribution. This allows users to visually confirm the evaluation of their activities and increase their motivation for their work.

[0085] An example of a prompt sentence to input into the generative AI model is, "Analyze and evaluate the user's statements and contributions." This system functions as a powerful tool for analyzing users' digital activities in detail and comprehensively, aiming to improve performance.

[0086] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0087] Step 1:

[0088] The terminal begins recording user activity. Specific inputs include application data used by the user, entered text, and access logs to online services. This data is collected in real time and output as encrypted logs. This data is temporarily stored in a buffer. The terminal's background processes ensure continuous data collection without interference from user activity.

[0089] Step 2:

[0090] The terminal transfers data to the server at regular intervals. Logs of collected operational activities are input and sent securely using the HTTPS protocol, outputting temporary log data. This transfer is performed periodically by batch processing to ensure data security and integrity.

[0091] Step 3:

[0092] The server stores the received data in a database. The input is log data sent from the terminal, which is converted into a structured data format and output. Specifically, a Python script is used to format the data, and the resulting data record is optimized for subsequent processing.

[0093] Step 4:

[0094] The server performs data cleansing, removing noise and inaccurate data. The input is the structured data stored in step 3, and it outputs a clean dataset through filtering. Specifically, it executes algorithms that perform anomaly detection and duplicate data removal.

[0095] Step 5:

[0096] The server performs analysis using natural language processing and machine learning algorithms. Clean data is used as input, and meaningful insights extracted through text analysis and sentiment analysis are output. The server utilizes generative AI models to perform tasks such as identifying the positivity and themes of conversations.

[0097] Step 6:

[0098] The server quantifies user performance based on the analysis results. The input is already analyzed data, and a corresponding evaluation score is output. A calculation algorithm is applied that visualizes the number of contributions and the depth of content, and calculates a specific contribution score.

[0099] Step 7:

[0100] Users access a dashboard on the server to view visualized data. The input is quantified performance data, which is output as visual graphs and charts. Users can use this information to evaluate their individual performance and improve overall team efficiency.

[0101] (Application Example 1)

[0102] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0103] In modern factories, there is a need to objectively evaluate work efficiency and operational precision and to find appropriate improvement measures tailored to the environment. However, conventional evaluation methods have the challenge of making it difficult to grasp the performance of operating equipment in real time and accurately, thus hindering efficient improvement. In addition, it is difficult to accurately classify work content and work time into specific tasks, requiring considerable effort to conduct detailed analysis. Furthermore, qualitative aspects such as work quality and contribution cannot be quantified, making it impossible to create concrete indicators for increasing added value. Therefore, new technologies are needed to solve these problems.

[0104] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0105] In this invention, the server includes means for collecting operational data from operational devices, means for using information processing technology and automatic learning methods to analyze the collected operational data, and means for quantifying the performance of the operational devices and generating a report based on the analysis results. This makes it possible to evaluate the work efficiency and accuracy of operational devices in a factory in real time and to quantify and visually present the quality and contribution of work. Furthermore, by providing appropriate indicators based on specific tasks, it contributes to the formulation and implementation of efficient improvement measures.

[0106] A "behavioral device" is a mechanical device that automatically performs a specific task and outputs motion data through sensors.

[0107] "Operation data" refers to digital information generated by an operating device during its work, which indicates its operating status and work efficiency.

[0108] "Information processing technology" refers to techniques for analyzing data and extracting meaningful information, and includes natural language processing and data mining.

[0109] "Automated learning methods" are techniques that use machine learning algorithms to autonomously learn patterns from data and perform predictions and classifications.

[0110] "Quantifying performance" refers to expressing the efficiency and accuracy of work numerically based on the operating data of the operating device.

[0111] A "report" is a document that summarizes the analysis results and includes evaluations and suggestions for improvement regarding the performance of the operating device.

[0112] "Visualization" refers to presenting numerical data and analysis results in a format that is easy for humans to understand, and involves using shapes and visual representations.

[0113] The server collects operational data in real time from behavioral devices installed in the factory and stores this data in a central database. This operational data includes the working status, operating time, efficiency, and accuracy of the behavioral devices. The server implements information processing technology and automated learning methods using programming languages ​​such as Python and R. This allows it to analyze the operational data and evaluate the efficiency and accuracy of the behavioral devices. It also generates detailed reports based on the results and provides an interface for visualizing these reports. This uses visual elements such as graphs and charts to make the data easier for users to understand visually.

[0114] Furthermore, based on the analysis results, the performance of the behavioral device can be quantitatively evaluated, and areas for improvement can be identified. This information can be used as an indicator to improve the operational efficiency of the factory.

[0115] As a concrete example, consider a screw-tightening robot placed on a manufacturing line. The server analyzes the motion data obtained from this robot and quantifies the accuracy and efficiency of screw tightening. This allows the user to know in advance when maintenance is needed and when parts need replacing, enabling them to take appropriate improvement measures.

[0116] An example of a prompt would be, "Use sensor data to evaluate the performance of the behavioral device in real time and propose specific improvement measures to enhance efficiency." This allows for advanced data analysis using a generative AI model, enabling the acquisition of practical insights.

[0117] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0118] Step 1:

[0119] The server periodically collects sensor data from the behavioral device. This sensor data includes the device's operating status, operating time, and operational efficiency. It receives raw sensor data as input and stores it in a database. The server preprocesses the received data, removing noise to output clean data.

[0120] Step 2:

[0121] The server uses Python to apply information processing techniques and automated learning methods to analyze clean sensor data. The input is clean sensor data, and the output is an evaluation result regarding the operational efficiency and accuracy of the behavioral device. In this process, statistical analysis of the data and model training are performed to predict the performance of the behavioral device.

[0122] Step 3:

[0123] The server generates a report quantifying the performance of the operating device based on the analysis results. The input is the analysis results, and the output is a detailed report. This report includes numerical data on the work efficiency and accuracy of each operating device, formatted in a user-readable format.

[0124] Step 4:

[0125] Users receive the generated report and review the data using a visualization interface. The input is the report data, and the output is visualized graphs and charts. This interface is designed to provide an intuitive understanding of the data, helping users identify important trends and patterns.

[0126] Step 5:

[0127] Based on the report, the user devises improvement measures for the behavioral device. This includes maintenance schedules to improve operational efficiency and adjustment plans to enhance operational accuracy. To support the user's decision-making, a generative AI model is used to generate prompt statements that produce further suggestions. These prompt statements serve as specific guidelines to support the user's decision-making.

[0128] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0129] To implement this invention, it is necessary to install a system on employee terminals that combines a program for collecting operation logs with an emotion engine. This system comprehensively records digital activities, such as how users operate their computers and what content they communicate using communication tools. The terminals routinely log text entered by users and content spoken in online meetings, and periodically transmit this data to a server.

[0130] The server first preprocesses the received data and analyzes it using natural language processing techniques. During this process, the emotion engine recognizes emotions from the user's text and voice data and integrates the results into the analyzed data. The emotion engine is responsible for identifying the user's emotional state, such as joy, anxiety, or anger, based on factors such as the frequency of specific words and changes in voice waveforms.

[0131] The analysis results are aggregated on the server as numerical data to evaluate the user's work efficiency, the quality of their contributions, and their overall contribution. This quantified data then forms a feedback loop to the terminal. For example, if a user makes positive and proactive statements during a project meeting, the emotion engine may detect this high level of motivation and report that it could positively impact the performance evaluation.

[0132] Users can visually review their activities and emotional tendencies through a dashboard provided by the server. This allows users to understand their own performance trends and devise steps for self-improvement as needed. This entire process functions as a powerful tool to promote employee productivity and self-monitoring. In this way, the present invention builds a system that provides more advanced performance management and insights by incorporating emotion recognition technology.

[0133] The following describes the processing flow.

[0134] Step 1:

[0135] The device records in real time the user's application usage, text input, and online meeting participation on their computer. This includes operation logs and audio recordings, which are stored securely with privacy in mind.

[0136] Step 2:

[0137] The device periodically collects data and sends it to the server via a secure communication protocol. Here, all transmitted data is encrypted to prevent unauthorized access by third parties.

[0138] Step 3:

[0139] The server first cleans the received data, removing noise and converting the data format. This prepares the data for analysis.

[0140] Step 4:

[0141] The server uses natural language processing technology to analyze text and audio data. As a result, it can identify the user's intentions and the content they are communicating.

[0142] Step 5:

[0143] The emotion engine is activated and recognizes the user's emotional state from the analyzed text and audio data. Specifically, it analyzes the vocabulary used and the tone of voice to estimate emotions such as "joy," "sadness," and "anger."

[0144] Step 6:

[0145] The server integrates the analysis results from the emotion engine into the dataset and uses them to quantify user performance, efficiency, and contribution. This data plays a crucial role as an employee evaluation metric.

[0146] Step 7:

[0147] Users access a dashboard to view visualized activity data and emotional trends. Here, the server provides graphs and charts related to emotional state and work efficiency, helping users monitor their own condition.

[0148] This process allows the system to comprehensively analyze employees' daily activities and suggest areas for improvement through emotion recognition.

[0149] (Example 2)

[0150] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0151] Traditional systems for quantifying and visually providing feedback on employee work efficiency and contributions have struggled to adequately analyze detailed emotional states and evaluate the quality of employee comments. Furthermore, there is a need to improve the accuracy of real-time evaluation and visualization when classifying comments and work time during online meetings. As a result, employees were unable to obtain guidance for self-improvement, leading to missed opportunities for increased productivity.

[0152] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0153] In this invention, the server includes means for acquiring work records from employee information processing devices, means for using data processing and computational models to analyze the acquired work records, and means for quantifying work and generating reports based on the analysis results. This makes it possible to accurately evaluate employees' work efficiency and contributions and provide visual feedback.

[0154] An "information processing device" is a device used for inputting, processing, storing, and outputting digital data, and usually refers to a computer.

[0155] "Work records" refer to logs of operations and activities performed by employees on information processing devices, including entered text and applications used.

[0156] "Data processing" refers to a series of operations that analyze collected digital data and transform it into useful information, typically including filtering and format normalization.

[0157] A "computational model" refers to mathematical and statistical methods used for data analysis and pattern recognition, and includes machine learning algorithms.

[0158] "Reporting materials" refer to summary documents and reports generated based on analyzed data, including those that numerically represent work efficiency and contribution.

[0159] "Visual feedback" refers to methods of displaying numerical data and evaluation results in a format that is easy for users to understand, and includes formats such as graphs and charts.

[0160] "Emotional state" refers to a psychological state recognized based on the user's text and voice, and includes various emotions such as joy, anger, sadness, and happiness.

[0161] "Contribution volume" refers to a quantitative indicator that shows how much an individual employee contributed to the work.

[0162] To implement this invention, it is necessary to install a system on the terminals used by employees that combines a program for recording operation logs with an engine for sentiment analysis. The terminals record daily operations and statements made during online meetings, and periodically transmit this data to a server. The purpose of this data is to comprehensively understand digital activities.

[0163] The terminal records user input and speech, collecting the data in an appropriate format. The hardware can be a standard computer or communication device. The software used will include log collection software and a client program incorporating an emotion engine.

[0164] The server analyzes the received data using natural language processing technology and recognizes emotional states using an emotion engine. In particular, it utilizes various data processing and computational models to evaluate the quality and impact of digital activities. The analysis results are quantified into reports, and feedback information is also generated to evaluate work efficiency and contribution.

[0165] Users can use the server-provided dashboard to view reports on their activities and sentiment tendencies. Based on this information, users can review their work performance and perform self-monitoring.

[0166] For example, when a user actively contributes opinions during a project meeting and receives positive feedback, the emotion engine perceives their high level of cooperation and motivation. Evaluation in such situations can contribute to improved performance afterward.

[0167] An example of a prompt message could be input to the generative AI model as, "Please tell me specifically about the system's process for collecting logs of user-performed digital activities and evaluating their emotional tendencies." This prompt allows for the extraction of detailed information about the system's operation and how to utilize it.

[0168] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0169] Step 1:

[0170] The device records user input text and online meeting speech. Input includes keyboard input and audio data via microphone. This data is periodically retrieved and saved as a log file. This logging generates a detailed activity record of how the user is operating the computer.

[0171] Step 2:

[0172] The terminal packages the collected log data and sends it to the server. This process uses a secure communication protocol to ensure data security. The input is the log data obtained in step 1, and the output is the secure transfer of data to the server. This procedure enables centralized, real-time data management.

[0173] Step 3:

[0174] The server first preprocesses the data received from the terminal. The input is unstructured log data, which is then formatted and broken down into necessary components. Data normalization and removal of unnecessary noise are performed, and the data is converted into a format suitable for analysis. This output is ready for the next analysis step.

[0175] Step 4:

[0176] The server uses natural language processing techniques to analyze pre-processed data. The input is formatted data, and an emotion engine recognizes emotional states from text and audio data. Specifically, it analyzes frequent patterns of certain words and changes in voice tone. The output is quantified evaluation data regarding the user's emotional state.

[0177] Step 5:

[0178] The server integrates the analyzed emotional state data and performs scoring to evaluate the user's work efficiency and contribution. The input is the evaluation data obtained in step 4, and the output is a numerical score for work efficiency and contribution. This process visualizes the user's overall performance.

[0179] Step 6:

[0180] Users visually view analysis results using a dashboard provided by the server. Input is scoring data from the server, and output is feedback in the form of graphs and charts that users can understand. This process allows users to analyze their own activities and identify areas for improvement.

[0181] (Application Example 2)

[0182] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0183] Understanding employee work efficiency and emotional states in real time and providing appropriate feedback is a critical challenge for many organizations. However, traditional methods have made it difficult to conduct detailed analyses of employee activities, including emotions, and to provide immediate feedback to improve productivity.

[0184] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0185] In this invention, the server includes means for collecting activity history from employee information processing devices, means for analyzing the activity history using natural language processing and machine learning techniques, and means for analyzing emotions and identifying the emotional state of the workers. This enables detailed analysis of employee activities and immediate feedback.

[0186] "Operation history" refers to data that records the details of operations and activities performed by employees using information processing equipment.

[0187] "Natural language processing technology" is a technology that uses computers to analyze human language and understand its meaning, emotions, and other aspects.

[0188] "Machine learning technology" refers to techniques that use algorithms to learn patterns from data and perform predictions and classifications.

[0189] "Emotional analysis" is a technology that identifies an employee's emotional state from their statements and behavioral data.

[0190] "Visualization" refers to making data understandable intuitively by displaying it visually using diagrams, charts, and other visual aids.

[0191] "Real-time" is a term that indicates that processing and analysis are performed almost simultaneously, and results are obtained without delay.

[0192] "Operational efficiency" is a quantifiable indicator that measures the efficiency with which employees perform their tasks.

[0193] "Emotional state" refers to an individual's temporary psychological state, and includes emotions such as joy, anxiety, and anger.

[0194] This invention is a system that analyzes the digital activity and emotional state of information processing devices used by employees to promote productivity improvement and self-improvement. Specifically, a log collection program is installed on the employee's information processing device to collect a history of their actions during work on a daily basis. A server periodically receives the collected data and analyzes it using natural language processing and machine learning technologies. The analysis also incorporates an emotion engine, which has the function of identifying emotions from speech and actions.

[0195] The server analysis results are aggregated as numerical report data of employee activities and fed back into a visually accessible dashboard. Through this dashboard, users can review their own activities and emotional tendencies and identify areas for improvement.

[0196] As a concrete example, smart glasses can be used to collect visual and auditory information from workers in real time. This makes it easy to manage productivity on-site and monitor employees' psychological health. If the goal to be achieved is set as "improving the quality of communication and boosting overall team morale," the generating AI model can suggest a more effective communication strategy using prompt sentences like the following.

[0197] Example prompt: "Create an analytical report to infer the emotional state of employees and identify potential areas for improvement in their work performance."

[0198] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0199] Step 1:

[0200] The terminal collects the operation history of the information processing device in real time using a log collection program. The input is employee operation data, and the output is saved as a log file. This log file includes a timestamp and action type.

[0201] Step 2:

[0202] The server periodically receives log files sent from terminals. The input is log files transferred from terminals, and the output is storage in a database. During storage in the database, data formatting is standardized and errors are checked.

[0203] Step 3:

[0204] The server utilizes natural language processing and machine learning techniques to analyze the collected data. The input is stored data in a database, and the output is a series of numerical data as a result of the analysis. In this process, an emotion engine is used to identify emotional states from text and audio data, analyzing the frequency of occurrence of specific words and changes in audio waveforms.

[0205] Step 4:

[0206] The server generates report data based on the analysis results and feeds it back into a dashboard for visualization. The input is numerical data from the analysis results, and the output is visual data displayed as graphs and charts. The dashboard displays work efficiency and emotional tendencies in a visually easy-to-understand format.

[0207] Step 5:

[0208] Users can review their activities and emotional tendencies through a dashboard, and develop strategies for business improvement by referring to the generated AI model and prompt messages. The input is visualized data, and the output is the user's own business improvement plan. Through this entire process, the aim is to improve employee productivity.

[0209] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0210] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0211] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0212] [Second Embodiment]

[0213] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0214] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0215] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0216] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0217] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0218] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0219] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0220] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0221] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0222] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0223] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0224] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0225] To implement this invention, it is necessary to install a program on a terminal used by an employee, which will serve to comprehensively collect the employee's digital activities. The terminal will routinely record the application operations performed by the user, the text entered, and the online services connected (email, chat tools, etc.), and transfer this data to a server in accordance with security protocols.

[0226] The server preprocesses the received data, performing data cleaning and format conversion as needed. Through this process, it analyzes the data using natural language processing techniques and machine learning algorithms. Natural language processing allows for the extraction of useful information from text data, such as identifying the emotions and themes of conversations.

[0227] The server also quantifies the data obtained through analysis and numerically evaluates each employee's performance. In online meetings, it analyzes the number of contributions and their constructiveness, and scores their contributions based on this. This makes it possible to clearly demonstrate the specific impact on projects and tasks.

[0228] Users can visualize the data stored on the server through the dashboard. Here, they can visually check individual activity history, efficiency scores, and the overall team performance. Based on this information, administrators can evaluate employees, and employees themselves can receive feedback to improve their work.

[0229] For example, if a user's contributions to a project primarily stem from their participation in remote meetings over a week, their frequency and quality of contributions will be highly valued and displayed as the best contribution on the dashboard. This allows users to receive tangible recognition in their work environment, leading to increased motivation.

[0230] This invention provides a system that comprehensively analyzes various aspects of digital activities and functions as a tool for evaluating employee performance, thus meeting the modern needs of promoting work style reform.

[0231] The following describes the processing flow.

[0232] Step 1:

[0233] The device records the user's access history to applications and web services in real time. During this process, information such as the type of operation, execution time, and details of the action are collected as logs.

[0234] Step 2:

[0235] The terminal sends collected log data to the server at regular intervals. Secure communication protocols are used for data transfer, and privacy protection is taken into consideration.

[0236] Step 3:

[0237] The server stores the received data in the database and simultaneously performs preprocessing. This involves cleaning and formatting the text data, preparing it for analysis.

[0238] Step 4:

[0239] The server applies natural language processing techniques to the pre-processed data, performing sentiment analysis and keyword extraction. This allows for a summary of the intent and content of the user's statements and inputs.

[0240] Step 5:

[0241] The server uses machine learning algorithms to analyze specific patterns and trends, and numerically evaluates contributions to work and the quality of contributions. This involves model-based analysis using historical data and rule-based analysis.

[0242] Step 6:

[0243] The server aggregates the analysis results and quantifies the allocation of work time, efficiency, and contribution for each user. This generates baseline data for individually evaluating performance.

[0244] Step 7:

[0245] Users access a dashboard to view their performance data. The server displays graphs and charts on this dashboard in a visually easy-to-understand format and provides them to the user.

[0246] This series of steps allows the system to analyze employees' digital activities in detail and perform objective evaluations.

[0247] (Example 1)

[0248] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0249] Traditional systems made it difficult to comprehensively and efficiently collect and analyze employees' digital activities. In particular, they could not accurately evaluate the quality and contribution of online meetings, and there was a lack of means to provide objective evaluation criteria. As a result, the evaluation of employee activities and the improvement of work efficiency were delayed, which was a challenge.

[0250] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0251] In this invention, the server includes means for collecting user activity from the operating environment, means for processing and organizing the collected activity data, and means for analyzing the data by combining natural language processing and machine learning algorithms to extract useful information. This makes it possible to comprehensively evaluate employees' digital activities and objectively assess the quality and contribution of their participation in online meetings.

[0252] "Operating environment" refers to the collective term for the computers, related software, and networks that users use when performing their work or tasks.

[0253] "User activity" refers to all digital actions performed by a user on a computer, including actions such as typing, clicking, and application usage history.

[0254] "Processing and organizing operational activity data" refers to a series of steps that involve cleaning up the collected raw data and converting it into a format that is easy to analyze.

[0255] "Natural language processing" refers to the technology of computational algorithms that enable computers to understand and analyze human language.

[0256] A "machine learning algorithm" is a mathematical model that uses large amounts of data to enable computers to improve themselves and perform efficient analysis and prediction.

[0257] "Extracting useful information" is the process of finding important indicators and trends from analyzed data that can be used to improve business operations and make decisions.

[0258] "Activity evaluation" is a method that aims to objectively assess the efficiency and effectiveness of employee performance in carrying out tasks by quantifying their performance.

[0259] "Quality and contribution of contributions in online meetings" refers to a standard for quantitatively measuring the importance and influence of what is said during a meeting.

[0260] To implement this system, a specific program must first be installed on the terminal. This program comprehensively collects operational activity data when the user is using the terminal and records activities related to daily work. The collected data is periodically sent to the server using secure communication. In this process, the terminal ensures the security of the data by using encrypted communication such as HTTPS.

[0261] The server then stores this collected data in a database and performs preprocessing. Specifically, it uses scripting languages ​​such as Python to clean and format the data. The prepared dataset is then further analyzed using natural language processing techniques and machine learning algorithms.

[0262] The server uses generative AI models to extract useful information from text data. Natural language processing enables sentiment analysis and theme identification of the text. For example, it can provide analysis results on the degree of positivity or negativity in conversation content.

[0263] Based on the analysis results, the server quantifies and evaluates user performance. By analyzing the number of contributions and their constructiveness in online meetings, the server can output a score representing the user's contribution. This evaluation result is visually displayed to the user through a dashboard. Users can easily check their own efficiency score and the overall performance of their team.

[0264] For example, if a user actively participates in a remote meeting and their contributions are highly valued for the project, they may be displayed on the dashboard as having the highest level of contribution. This allows users to visually confirm the evaluation of their activities and increase their motivation for their work.

[0265] An example of a prompt sentence to input into the generative AI model is, "Analyze and evaluate the user's statements and contributions." This system functions as a powerful tool for analyzing users' digital activities in detail and comprehensively, aiming to improve performance.

[0266] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0267] Step 1:

[0268] The terminal begins recording user activity. Specific inputs include application data used by the user, entered text, and access logs to online services. This data is collected in real time and output as encrypted logs. This data is temporarily stored in a buffer. The terminal's background processes ensure continuous data collection without interference from user activity.

[0269] Step 2:

[0270] The terminal transfers data to the server at regular intervals. Logs of collected operational activities are input and sent securely using the HTTPS protocol, outputting temporary log data. This transfer is performed periodically by batch processing to ensure data security and integrity.

[0271] Step 3:

[0272] The server stores the received data in a database. The input is log data sent from the terminal, which is converted into a structured data format and output. Specifically, a Python script is used to format the data, and the resulting data record is optimized for subsequent processing.

[0273] Step 4:

[0274] The server performs data cleansing, removing noise and inaccurate data. The input is the structured data stored in step 3, and it outputs a clean dataset through filtering. Specifically, it executes algorithms that perform anomaly detection and duplicate data removal.

[0275] Step 5:

[0276] The server performs analysis using natural language processing and machine learning algorithms. Clean data is used as input, and meaningful insights extracted through text analysis and sentiment analysis are output. The server utilizes generative AI models to perform tasks such as identifying the positivity and themes of conversations.

[0277] Step 6:

[0278] The server quantifies user performance based on the analysis results. The input is already analyzed data, and a corresponding evaluation score is output. A calculation algorithm is applied that visualizes the number of contributions and the depth of content, and calculates a specific contribution score.

[0279] Step 7:

[0280] Users access a dashboard on the server to view visualized data. The input is quantified performance data, which is output as visual graphs and charts. Users can use this information to evaluate their individual performance and improve overall team efficiency.

[0281] (Application Example 1)

[0282] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0283] In modern factories, there is a need to objectively evaluate work efficiency and operational precision and to find appropriate improvement measures tailored to the environment. However, conventional evaluation methods have the challenge of making it difficult to grasp the performance of operating equipment in real time and accurately, thus hindering efficient improvement. In addition, it is difficult to accurately classify work content and work time into specific tasks, requiring considerable effort to conduct detailed analysis. Furthermore, qualitative aspects such as work quality and contribution cannot be quantified, making it impossible to create concrete indicators for increasing added value. Therefore, new technologies are needed to solve these problems.

[0284] The specific processing by the specific processing unit 290 of the data processing apparatus 12 in Application Example 1 is realized by the following means.

[0285] In this invention, the server includes means for collecting operation data from the action device, means for using information processing techniques and automatic learning methods to analyze the collected operation data, and means for quantifying the performance of the action device based on the analysis results and generating a report. As a result, it becomes possible to evaluate the working efficiency and accuracy of the operating devices in the factory in real time, quantify the quality and contribution degree of the work, and present it visually. In addition, by providing appropriate indicators based on specific operations, it contributes to the formulation and implementation of efficient improvement measures.

[0286] The "action device" is a mechanical device that automatically executes a specific operation and outputs operation data through a sensor.

[0287] The "operation data" is digital information generated by the action device during operation, and is data indicating its operation state and working efficiency.

[0288] The "information processing technique" is a technique for analyzing data and extracting meaningful information, and includes natural language processing and data mining.

[0289] The "automatic learning method" is a technology that autonomously learns patterns from data using machine learning algorithms and performs prediction and classification.

[0290] The "quantification of performance" is to express the working efficiency and accuracy as numerical values based on the operation data of the operating device.

[0291] The "report" is a document that summarizes the analysis results and includes evaluations and improvement points regarding the performance of the action device.

[0292] "Visualization" refers to presenting numerical data and analysis results in a format that is easy for humans to understand, and involves using shapes and visual representations.

[0293] The server collects operational data in real time from behavioral devices installed in the factory and stores this data in a central database. This operational data includes the working status, operating time, efficiency, and accuracy of the behavioral devices. The server implements information processing technology and automated learning methods using programming languages ​​such as Python and R. This allows it to analyze the operational data and evaluate the efficiency and accuracy of the behavioral devices. It also generates detailed reports based on the results and provides an interface for visualizing these reports. This uses visual elements such as graphs and charts to make the data easier for users to understand visually.

[0294] Furthermore, based on the analysis results, the performance of the behavioral device can be quantitatively evaluated, and areas for improvement can be identified. This information can be used as an indicator to improve the operational efficiency of the factory.

[0295] As a concrete example, consider a screw-tightening robot placed on a manufacturing line. The server analyzes the motion data obtained from this robot and quantifies the accuracy and efficiency of screw tightening. This allows the user to know in advance when maintenance is needed and when parts need replacing, enabling them to take appropriate improvement measures.

[0296] An example of a prompt would be, "Use sensor data to evaluate the performance of the behavioral device in real time and propose specific improvement measures to enhance efficiency." This allows for advanced data analysis using a generative AI model, enabling the acquisition of practical insights.

[0297] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0298] Step 1:

[0299] The server periodically collects sensor data from the behavioral device. This sensor data includes the device's operating status, operating time, and operational efficiency. It receives raw sensor data as input and stores it in a database. The server preprocesses the received data, removing noise to output clean data.

[0300] Step 2:

[0301] The server uses Python to apply information processing techniques and automated learning methods to analyze clean sensor data. The input is clean sensor data, and the output is an evaluation result regarding the operational efficiency and accuracy of the behavioral device. In this process, statistical analysis of the data and model training are performed to predict the performance of the behavioral device.

[0302] Step 3:

[0303] The server generates a report quantifying the performance of the operating device based on the analysis results. The input is the analysis results, and the output is a detailed report. This report includes numerical data on the work efficiency and accuracy of each operating device, formatted in a user-readable format.

[0304] Step 4:

[0305] Users receive the generated report and review the data using a visualization interface. The input is the report data, and the output is visualized graphs and charts. This interface is designed to provide an intuitive understanding of the data, helping users identify important trends and patterns.

[0306] Step 5:

[0307] Based on the report, the user devises improvement measures for the mobile device. This includes a maintenance schedule to improve the reduction in operation efficiency and an adjustment plan to enhance operation accuracy. To support the user's judgment, a prompt sentence is created using the generative AI model to generate further proposals. This prompt sentence becomes the specific guideline to support the user's decision-making.

[0308] Furthermore, an emotion engine for estimating the user's emotions may be combined. That is, the specific processing unit 290 may estimate the user's emotions using the emotion recognition model 59 and perform specific processing using the user's emotions.

[0309] To implement this invention, it is necessary to install a system that combines a program for collecting operation logs and an emotion engine on the terminals used by employees. This system comprehensively records digital activities such as how the user operates the computer and what content is transmitted through communication tools. The terminal routinely logs the text input by the user and the speech content in online meetings, and periodically transmits this data to the server.

[0310] The server first preprocesses the received data and analyzes the data by making full use of natural language processing technology. At this time, the emotion engine recognizes the emotion from the user's text and voice data, and integrates the result into the analysis data. The emotion engine plays a role in identifying the user's emotional state such as joy, anxiety, anger, etc. based on the frequency of occurrence of specific words and changes in voice waveforms.

[0311] The analysis results are aggregated on the server as numerical data for evaluating the user's work efficiency, speech quality, and contribution. The quantified data obtained here forms a feedback loop to the terminal. As a specific example, when the user makes a positive and enthusiastic speech during a project meeting, the emotion engine senses the high motivation and reports that it may have a positive impact on the performance evaluation.

[0312] Users can visually review their activities and emotional tendencies through a dashboard provided by the server. This allows users to understand their own performance trends and devise steps for self-improvement as needed. This entire process functions as a powerful tool to promote employee productivity and self-monitoring. In this way, the present invention builds a system that provides more advanced performance management and insights by incorporating emotion recognition technology.

[0313] The following describes the processing flow.

[0314] Step 1:

[0315] The device records in real time the user's application usage, text input, and online meeting participation on their computer. This includes operation logs and audio recordings, which are stored securely with privacy in mind.

[0316] Step 2:

[0317] The device periodically collects data and sends it to the server via a secure communication protocol. Here, all transmitted data is encrypted to prevent unauthorized access by third parties.

[0318] Step 3:

[0319] The server first cleans the received data, removing noise and converting the data format. This prepares the data for analysis.

[0320] Step 4:

[0321] The server uses natural language processing technology to analyze text and audio data. As a result, it can identify the user's intentions and the content they are communicating.

[0322] Step 5:

[0323] The emotion engine is activated and recognizes the user's emotional state from the analyzed text and audio data. Specifically, it analyzes the vocabulary used and the tone of voice to estimate emotions such as "joy," "sadness," and "anger."

[0324] Step 6:

[0325] The server integrates the analysis results from the emotion engine into the dataset and uses them to quantify user performance, efficiency, and contribution. This data plays a crucial role as an employee evaluation metric.

[0326] Step 7:

[0327] Users access a dashboard to view visualized activity data and emotional trends. Here, the server provides graphs and charts related to emotional state and work efficiency, helping users monitor their own condition.

[0328] This process allows the system to comprehensively analyze employees' daily activities and suggest areas for improvement through emotion recognition.

[0329] (Example 2)

[0330] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0331] Traditional systems for quantifying and visually providing feedback on employee work efficiency and contributions have struggled to adequately analyze detailed emotional states and evaluate the quality of employee comments. Furthermore, there is a need to improve the accuracy of real-time evaluation and visualization when classifying comments and work time during online meetings. As a result, employees were unable to obtain guidance for self-improvement, leading to missed opportunities for increased productivity.

[0332] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0333] In this invention, the server includes means for acquiring work records from employee information processing devices, means for using data processing and computational models to analyze the acquired work records, and means for quantifying work and generating reports based on the analysis results. This makes it possible to accurately evaluate employees' work efficiency and contributions and provide visual feedback.

[0334] An "information processing device" is a device used for inputting, processing, storing, and outputting digital data, and usually refers to a computer.

[0335] "Work records" refer to logs of operations and activities performed by employees on information processing devices, including entered text and applications used.

[0336] "Data processing" refers to a series of operations that analyze collected digital data and transform it into useful information, typically including filtering and format normalization.

[0337] A "computational model" refers to mathematical and statistical methods used for data analysis and pattern recognition, and includes machine learning algorithms.

[0338] "Reporting materials" refer to summary documents and reports generated based on analyzed data, including those that numerically represent work efficiency and contribution.

[0339] "Visual feedback" refers to methods of displaying numerical data and evaluation results in a format that is easy for users to understand, and includes formats such as graphs and charts.

[0340] "Emotional state" refers to a psychological state recognized based on the user's text and voice, and includes various emotions such as joy, anger, sadness, and happiness.

[0341] "Contribution volume" refers to a quantitative indicator that shows how much an individual employee contributed to the work.

[0342] To implement this invention, it is necessary to install a system on the terminals used by employees that combines a program for recording operation logs with an engine for sentiment analysis. The terminals record daily operations and statements made during online meetings, and periodically transmit this data to a server. The purpose of this data is to comprehensively understand digital activities.

[0343] The terminal records user input and speech, collecting the data in an appropriate format. The hardware can be a standard computer or communication device. The software used will include log collection software and a client program incorporating an emotion engine.

[0344] The server analyzes the received data using natural language processing technology and recognizes emotional states using an emotion engine. In particular, it utilizes various data processing and computational models to evaluate the quality and impact of digital activities. The analysis results are quantified into reports, and feedback information is also generated to evaluate work efficiency and contribution.

[0345] Users can use the server-provided dashboard to view reports on their activities and sentiment tendencies. Based on this information, users can review their work performance and perform self-monitoring.

[0346] For example, when a user actively contributes opinions during a project meeting and receives positive feedback, the emotion engine perceives their high level of cooperation and motivation. Evaluation in such situations can contribute to improved performance afterward.

[0347] An example of a prompt message could be input to the generative AI model as, "Please tell me specifically about the system's process for collecting logs of user-performed digital activities and evaluating their emotional tendencies." This prompt allows for the extraction of detailed information about the system's operation and how to utilize it.

[0348] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0349] Step 1:

[0350] The device records user input text and online meeting speech. Input includes keyboard input and audio data via microphone. This data is periodically retrieved and saved as a log file. This logging generates a detailed activity record of how the user is operating the computer.

[0351] Step 2:

[0352] The terminal packages the collected log data and sends it to the server. This process uses a secure communication protocol to ensure data security. The input is the log data obtained in step 1, and the output is the secure transfer of data to the server. This procedure enables centralized, real-time data management.

[0353] Step 3:

[0354] The server first preprocesses the data received from the terminal. The input is unstructured log data, which is then formatted and broken down into necessary components. Data normalization and removal of unnecessary noise are performed, and the data is converted into a format suitable for analysis. This output is ready for the next analysis step.

[0355] Step 4:

[0356] The server uses natural language processing techniques to analyze pre-processed data. The input is formatted data, and an emotion engine recognizes emotional states from text and audio data. Specifically, it analyzes frequent patterns of certain words and changes in voice tone. The output is quantified evaluation data regarding the user's emotional state.

[0357] Step 5:

[0358] The server integrates the analyzed emotional state data and performs scoring to evaluate the user's work efficiency and contribution. The input is the evaluation data obtained in step 4, and the output is a numerical score for work efficiency and contribution. This process visualizes the user's overall performance.

[0359] Step 6:

[0360] Users visually view analysis results using a dashboard provided by the server. Input is scoring data from the server, and output is feedback in the form of graphs and charts that users can understand. This process allows users to analyze their own activities and identify areas for improvement.

[0361] (Application Example 2)

[0362] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0363] Understanding employee work efficiency and emotional states in real time and providing appropriate feedback is a critical challenge for many organizations. However, traditional methods have made it difficult to conduct detailed analyses of employee activities, including emotions, and to provide immediate feedback to improve productivity.

[0364] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0365] In this invention, the server includes means for collecting activity history from employee information processing devices, means for analyzing the activity history using natural language processing and machine learning technologies, and means for analyzing emotions and identifying the emotional state of the workers. This enables detailed analysis of employee activities and immediate feedback.

[0366] "Operation history" refers to data that records the details of operations and activities performed by employees using information processing equipment.

[0367] "Natural language processing technology" is a technology that uses computers to analyze human language and understand its meaning, emotions, and other aspects.

[0368] "Machine learning technology" refers to techniques that use algorithms to learn patterns from data and perform predictions and classifications.

[0369] "Emotional analysis" is a technology that identifies an employee's emotional state from their statements and behavioral data.

[0370] "Visualization" refers to making data understandable intuitively by displaying it visually using diagrams, charts, and other visual aids.

[0371] "Real-time" is a term that indicates that processing and analysis are performed almost simultaneously, and results are obtained without delay.

[0372] "Operational efficiency" is a quantifiable indicator that measures the efficiency with which employees perform their tasks.

[0373] "Emotional state" refers to an individual's temporary psychological state, and includes emotions such as joy, anxiety, and anger.

[0374] This invention is a system that analyzes the digital activity and emotional state of information processing devices used by employees to promote productivity improvement and self-improvement. Specifically, a log collection program is installed on the employee's information processing device to collect a history of their actions during work on a daily basis. A server periodically receives the collected data and analyzes it using natural language processing and machine learning technologies. The analysis also incorporates an emotion engine, which has the function of identifying emotions from speech and actions.

[0375] The server analysis results are aggregated as numerical report data of employee activities and fed back into a visually accessible dashboard. Through this dashboard, users can review their own activities and emotional tendencies and identify areas for improvement.

[0376] As a concrete example, smart glasses can be used to collect visual and auditory information from workers in real time. This makes it easy to manage productivity on-site and monitor employees' psychological health. If the goal to be achieved is set as "improving the quality of communication and boosting overall team morale," the generating AI model can suggest a more effective communication strategy using prompt sentences like the following.

[0377] Example prompt: "Create an analytical report to infer the emotional state of employees and identify potential areas for improvement in their work performance."

[0378] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0379] Step 1:

[0380] The terminal collects the operation history of the information processing device in real time using a log collection program. The input is employee operation data, and the output is saved as a log file. This log file includes a timestamp and action type.

[0381] Step 2:

[0382] The server periodically receives log files sent from terminals. The input is log files transferred from terminals, and the output is storage in a database. During storage in the database, data formatting is standardized and errors are checked.

[0383] Step 3:

[0384] The server utilizes natural language processing and machine learning techniques to analyze the collected data. The input is stored data in a database, and the output is a series of numerical data as a result of the analysis. In this process, an emotion engine is used to identify emotional states from text and audio data, analyzing the frequency of occurrence of specific words and changes in audio waveforms.

[0385] Step 4:

[0386] The server generates report data based on the analysis results and feeds it back into a dashboard for visualization. The input is numerical data from the analysis results, and the output is visual data displayed as graphs and charts. The dashboard displays work efficiency and emotional tendencies in a visually easy-to-understand format.

[0387] Step 5:

[0388] Users can review their activities and emotional tendencies through a dashboard, and develop strategies for business improvement by referring to the generated AI model and prompt messages. The input is visualized data, and the output is the user's own business improvement plan. Through this entire process, the aim is to improve employee productivity.

[0389] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0390] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0391] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0392] [Third Embodiment]

[0393] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0394] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0395] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0396] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0397] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0398] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0399] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0400] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0401] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0402] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0403] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0404] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0405] To implement this invention, it is necessary to install a program on a terminal used by an employee, which will serve to comprehensively collect the employee's digital activities. The terminal will routinely record the application operations performed by the user, the text entered, and the online services connected (email, chat tools, etc.), and transfer this data to a server in accordance with security protocols.

[0406] The server preprocesses the received data, performing data cleaning and format conversion as needed. Through this process, it analyzes the data using natural language processing techniques and machine learning algorithms. Natural language processing allows for the extraction of useful information from text data, such as identifying the emotions and themes of conversations.

[0407] The server also quantifies the data obtained through analysis and numerically evaluates each employee's performance. In online meetings, it analyzes the number of contributions and their constructiveness, and scores their contributions based on this. This makes it possible to clearly demonstrate the specific impact on projects and tasks.

[0408] Users can visualize the data stored on the server through the dashboard. Here, they can visually check individual activity history, efficiency scores, and the overall team performance. Based on this information, administrators can evaluate employees, and employees themselves can receive feedback to improve their work.

[0409] For example, if a user's contributions to a project primarily stem from their participation in remote meetings over a week, their frequency and quality of contributions will be highly valued and displayed as the best contribution on the dashboard. This allows users to receive tangible recognition in their work environment, leading to increased motivation.

[0410] This invention provides a system that comprehensively analyzes various aspects of digital activities and functions as a tool for evaluating employee performance, thus meeting the modern needs of promoting work style reform.

[0411] The following describes the processing flow.

[0412] Step 1:

[0413] The device records the user's access history to applications and web services in real time. During this process, information such as the type of operation, execution time, and details of the action are collected as logs.

[0414] Step 2:

[0415] The terminal sends collected log data to the server at regular intervals. Secure communication protocols are used for data transfer, and privacy protection is taken into consideration.

[0416] Step 3:

[0417] The server stores the received data in the database and simultaneously performs preprocessing. This involves cleaning and formatting the text data, preparing it for analysis.

[0418] Step 4:

[0419] The server applies natural language processing techniques to the pre-processed data, performing sentiment analysis and keyword extraction. This allows for a summary of the intent and content of the user's statements and inputs.

[0420] Step 5:

[0421] The server uses machine learning algorithms to analyze specific patterns and trends, and numerically evaluates contributions to work and the quality of contributions. This involves model-based analysis using historical data and rule-based analysis.

[0422] Step 6:

[0423] The server aggregates the analysis results and quantifies the allocation of work time, efficiency, and contribution for each user. This generates baseline data for individually evaluating performance.

[0424] Step 7:

[0425] Users access a dashboard to view their performance data. The server displays graphs and charts on this dashboard in a visually easy-to-understand format and provides them to the user.

[0426] This series of steps allows the system to analyze employees' digital activities in detail and perform objective evaluations.

[0427] (Example 1)

[0428] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0429] Traditional systems made it difficult to comprehensively and efficiently collect and analyze employees' digital activities. In particular, they could not accurately evaluate the quality and contribution of online meetings, and there was a lack of means to provide objective evaluation criteria. As a result, the evaluation of employee activities and the improvement of work efficiency were delayed, which was a challenge.

[0430] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0431] In this invention, the server includes means for collecting user activity from the operating environment, means for processing and organizing the collected activity data, and means for analyzing the data by combining natural language processing and machine learning algorithms to extract useful information. This makes it possible to comprehensively evaluate employees' digital activities and objectively assess the quality and contribution of their participation in online meetings.

[0432] "Operating environment" refers to the collective term for the computers, related software, and networks that users use when performing their work or tasks.

[0433] "User activity" refers to all digital actions performed by a user on a computer, including actions such as typing, clicking, and application usage history.

[0434] "Processing and organizing operational activity data" refers to a series of steps that involve cleaning up the collected raw data and converting it into a format that is easy to analyze.

[0435] "Natural language processing" refers to the technology of computational algorithms that enable computers to understand and analyze human language.

[0436] A "machine learning algorithm" is a mathematical model that uses large amounts of data to enable computers to improve themselves and perform efficient analysis and prediction.

[0437] "Extracting useful information" is the process of finding important indicators and trends from analyzed data that can be used to improve business operations and make decisions.

[0438] "Activity evaluation" is a method that aims to objectively assess the efficiency and effectiveness of employee performance in carrying out tasks by quantifying their performance.

[0439] "Quality and contribution of contributions in online meetings" refers to a standard for quantitatively measuring the importance and influence of what is said during a meeting.

[0440] To implement this system, a specific program must first be installed on the terminal. This program comprehensively collects operational activity data when the user is using the terminal and records activities related to daily work. The collected data is periodically sent to the server using secure communication. In this process, the terminal ensures the security of the data by using encrypted communication such as HTTPS.

[0441] The server then stores this collected data in a database and performs preprocessing. Specifically, it uses scripting languages ​​such as Python to clean and format the data. The prepared dataset is then further analyzed using natural language processing techniques and machine learning algorithms.

[0442] The server uses generative AI models to extract useful information from text data. Natural language processing enables sentiment analysis and theme identification of the text. For example, it can provide analysis results on the degree of positivity or negativity in conversation content.

[0443] Based on the analysis results, the server quantifies and evaluates user performance. By analyzing the number of contributions and their constructiveness in online meetings, the server can output a score representing the user's contribution. This evaluation result is visually displayed to the user through a dashboard. Users can easily check their own efficiency score and the overall performance of their team.

[0444] For example, if a user actively participates in a remote meeting and their contributions are highly valued for the project, they may be displayed on the dashboard as having the highest level of contribution. This allows users to visually confirm the evaluation of their activities and increase their motivation for their work.

[0445] An example of a prompt sentence to input into the generative AI model is, "Analyze and evaluate the user's statements and contributions." This system functions as a powerful tool for analyzing users' digital activities in detail and comprehensively, aiming to improve performance.

[0446] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0447] Step 1:

[0448] The terminal begins recording user activity. Specific inputs include application data used by the user, entered text, and access logs to online services. This data is collected in real time and output as encrypted logs. This data is temporarily stored in a buffer. The terminal's background processes ensure continuous data collection without interference from user activity.

[0449] Step 2:

[0450] The terminal transfers data to the server at regular intervals. Logs of collected operational activities are input and sent securely using the HTTPS protocol, outputting temporary log data. This transfer is performed periodically by batch processing to ensure data security and integrity.

[0451] Step 3:

[0452] The server stores the received data in a database. The input is log data sent from the terminal, which is converted into a structured data format and output. Specifically, a Python script is used to format the data, and the resulting data record is optimized for subsequent processing.

[0453] Step 4:

[0454] The server performs data cleansing, removing noise and inaccurate data. The input is the structured data stored in step 3, and it outputs a clean dataset through filtering. Specifically, it executes algorithms that perform anomaly detection and duplicate data removal.

[0455] Step 5:

[0456] The server performs analysis using natural language processing and machine learning algorithms. Clean data is used as input, and meaningful insights extracted through text analysis and sentiment analysis are output. The server utilizes generative AI models to perform tasks such as identifying the positivity and themes of conversations.

[0457] Step 6:

[0458] The server quantifies user performance based on the analysis results. The input is already analyzed data, and a corresponding evaluation score is output. A calculation algorithm is applied that visualizes the number of contributions and the depth of content, and calculates a specific contribution score.

[0459] Step 7:

[0460] Users access a dashboard on the server to view visualized data. The input is quantified performance data, which is output as visual graphs and charts. Users can use this information to evaluate their individual performance and improve overall team efficiency.

[0461] (Application Example 1)

[0462] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0463] In modern factories, there is a need to objectively evaluate work efficiency and operational precision and to find appropriate improvement measures tailored to the environment. However, conventional evaluation methods have the challenge of making it difficult to grasp the performance of operating equipment in real time and accurately, thus hindering efficient improvement. In addition, it is difficult to accurately classify work content and work time into specific tasks, requiring considerable effort to conduct detailed analysis. Furthermore, qualitative aspects such as work quality and contribution cannot be quantified, making it impossible to create concrete indicators for increasing added value. Therefore, new technologies are needed to solve these problems.

[0464] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0465] In this invention, the server includes means for collecting operational data from operational devices, means for using information processing technology and automatic learning methods to analyze the collected operational data, and means for quantifying the performance of the operational devices and generating a report based on the analysis results. This makes it possible to evaluate the work efficiency and accuracy of operational devices in a factory in real time and to quantify and visually present the quality and contribution of work. Furthermore, by providing appropriate indicators based on specific tasks, it contributes to the formulation and implementation of efficient improvement measures.

[0466] A "behavioral device" is a mechanical device that automatically performs a specific task and outputs operational data through sensors.

[0467] "Operation data" refers to digital information generated by an operating device during its work, which indicates its operating status and work efficiency.

[0468] "Information processing technology" refers to techniques for analyzing data and extracting meaningful information, and includes natural language processing and data mining.

[0469] "Automated learning methods" are techniques that use machine learning algorithms to autonomously learn patterns from data and perform predictions and classifications.

[0470] "Quantifying performance" refers to expressing the efficiency and accuracy of work numerically based on the operating data of the operating device.

[0471] A "report" is a document that summarizes the analysis results and includes evaluations and suggestions for improvement regarding the performance of the operating device.

[0472] "Visualization" refers to presenting numerical data and analysis results in a format that is easy for humans to understand, and involves using shapes and visual representations.

[0473] The server collects operational data in real time from behavioral devices installed in the factory and stores this data in a central database. This operational data includes the working status, operating time, efficiency, and accuracy of the behavioral devices. The server implements information processing technology and automated learning methods using programming languages ​​such as Python and R. This allows it to analyze the operational data and evaluate the efficiency and accuracy of the behavioral devices. It also generates detailed reports based on the results and provides an interface for visualizing these reports. This uses visual elements such as graphs and charts to make the data easier for users to understand visually.

[0474] Furthermore, based on the analysis results, the performance of the behavioral device can be quantitatively evaluated, and areas for improvement can be identified. This information can be used as an indicator to improve the operational efficiency of the factory.

[0475] As a concrete example, consider a screw-tightening robot placed on a manufacturing line. The server analyzes the motion data obtained from this robot and quantifies the accuracy and efficiency of screw tightening. This allows the user to know in advance when maintenance is needed and when parts need replacing, enabling them to take appropriate improvement measures.

[0476] An example of a prompt would be, "Use sensor data to evaluate the performance of the behavioral device in real time and propose specific improvement measures to enhance efficiency." This allows for advanced data analysis using a generative AI model, enabling the acquisition of practical insights.

[0477] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0478] Step 1:

[0479] The server periodically collects sensor data from the behavioral device. This sensor data includes the device's operating status, operating time, and operational efficiency. It receives raw sensor data as input and stores it in a database. The server preprocesses the received data, removing noise to output clean data.

[0480] Step 2:

[0481] The server uses Python to apply information processing techniques and automated learning methods to analyze clean sensor data. The input is clean sensor data, and the output is an evaluation result regarding the operational efficiency and accuracy of the behavioral device. In this process, statistical analysis of the data and model training are performed to predict the performance of the behavioral device.

[0482] Step 3:

[0483] The server generates a report quantifying the performance of the operating device based on the analysis results. The input is the analysis results, and the output is a detailed report. This report includes numerical data on the work efficiency and accuracy of each operating device, formatted in a user-readable format.

[0484] Step 4:

[0485] Users receive the generated report and review the data using a visualization interface. The input is the report data, and the output is visualized graphs and charts. This interface is designed to provide an intuitive understanding of the data, helping users identify important trends and patterns.

[0486] Step 5:

[0487] Based on the report, the user devises improvement measures for the behavioral device. This includes maintenance schedules to improve operational efficiency and adjustment plans to enhance operational accuracy. To support the user's decision-making, a generative AI model is used to generate prompt statements that produce further suggestions. These prompt statements serve as specific guidelines to support the user's decision-making.

[0488] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0489] To implement this invention, it is necessary to install a system on employee terminals that combines a program for collecting operation logs with an emotion engine. This system comprehensively records digital activities, such as how users operate their computers and what content they communicate using communication tools. The terminals routinely log text entered by users and content spoken in online meetings, and periodically transmit this data to a server.

[0490] The server first preprocesses the received data and analyzes it using natural language processing techniques. During this process, the emotion engine recognizes emotions from the user's text and voice data and integrates the results into the analyzed data. The emotion engine is responsible for identifying the user's emotional state, such as joy, anxiety, or anger, based on factors such as the frequency of specific words and changes in voice waveforms.

[0491] The analysis results are aggregated on the server as numerical data to evaluate the user's work efficiency, the quality of their contributions, and their overall contribution. This quantified data then forms a feedback loop to the terminal. For example, if a user makes positive and proactive statements during a project meeting, the emotion engine may detect this high level of motivation and report that it could positively impact the performance evaluation.

[0492] Users can visually review their activities and emotional tendencies through a dashboard provided by the server. This allows users to understand their own performance trends and devise steps for self-improvement as needed. This entire process functions as a powerful tool to promote employee productivity and self-monitoring. In this way, the present invention builds a system that provides more advanced performance management and insights by incorporating emotion recognition technology.

[0493] The following describes the processing flow.

[0494] Step 1:

[0495] The device records in real time the user's application usage, text input, and online meeting participation on their computer. This includes operation logs and audio recordings, which are stored securely with privacy in mind.

[0496] Step 2:

[0497] The device periodically collects data and sends it to the server via a secure communication protocol. Here, all transmitted data is encrypted to prevent unauthorized access by third parties.

[0498] Step 3:

[0499] The server first cleans the received data, removing noise and converting the data format. This prepares the data for analysis.

[0500] Step 4:

[0501] The server uses natural language processing technology to analyze text and audio data. As a result, it can identify the user's intentions and the content they are communicating.

[0502] Step 5:

[0503] The emotion engine is activated and recognizes the user's emotional state from the analyzed text and audio data. Specifically, it analyzes the vocabulary used and the tone of voice to estimate emotions such as "joy," "sadness," and "anger."

[0504] Step 6:

[0505] The server integrates the analysis results from the emotion engine into the dataset and uses them to quantify user performance, efficiency, and contribution. This data plays a crucial role as an employee evaluation metric.

[0506] Step 7:

[0507] Users access a dashboard to view visualized activity data and emotional trends. Here, the server provides graphs and charts related to emotional state and work efficiency, helping users monitor their own condition.

[0508] This process allows the system to comprehensively analyze employees' daily activities and suggest areas for improvement through emotion recognition.

[0509] (Example 2)

[0510] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0511] Traditional systems for quantifying and visually providing feedback on employee work efficiency and contributions have struggled to adequately analyze detailed emotional states and evaluate the quality of employee comments. Furthermore, there is a need to improve the accuracy of real-time evaluation and visualization when classifying comments and work time during online meetings. As a result, employees were unable to obtain guidance for self-improvement, leading to missed opportunities for increased productivity.

[0512] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0513] In this invention, the server includes means for acquiring work records from employee information processing devices, means for using data processing and computational models to analyze the acquired work records, and means for quantifying work and generating reports based on the analysis results. This makes it possible to accurately evaluate employees' work efficiency and contributions and provide visual feedback.

[0514] An "information processing device" is a device used for inputting, processing, storing, and outputting digital data, and usually refers to a computer.

[0515] "Work records" refer to logs of operations and activities performed by employees on information processing devices, including entered text and applications used.

[0516] "Data processing" refers to a series of operations that analyze collected digital data and transform it into useful information, typically including filtering and format normalization.

[0517] A "computational model" refers to mathematical and statistical methods used for data analysis and pattern recognition, and includes machine learning algorithms.

[0518] "Reporting materials" refer to summary documents and reports generated based on analyzed data, including those that numerically represent work efficiency and contribution.

[0519] "Visual feedback" refers to methods of displaying numerical data and evaluation results in a format that is easy for users to understand, and includes formats such as graphs and charts.

[0520] "Emotional state" refers to a psychological state recognized based on the user's text and voice, and includes various emotions such as joy, anger, sadness, and happiness.

[0521] "Contribution volume" refers to a quantitative indicator that shows how much an individual employee contributed to the work.

[0522] To implement this invention, it is necessary to install a system on the terminals used by employees that combines a program for recording operation logs with an engine for sentiment analysis. The terminals record daily operations and statements made during online meetings, and periodically transmit this data to a server. The purpose of this data is to comprehensively understand digital activities.

[0523] The terminal records user input and speech, collecting the data in an appropriate format. The hardware can be a standard computer or communication device. The software used will include log collection software and a client program incorporating an emotion engine.

[0524] The server analyzes the received data using natural language processing technology and recognizes emotional states using an emotion engine. In particular, it utilizes various data processing and computational models to evaluate the quality and impact of digital activities. The analysis results are quantified into reports, and feedback information is also generated to evaluate work efficiency and contribution.

[0525] Users can use the server-provided dashboard to view reports on their activities and sentiment tendencies. Based on this information, users can review their work performance and perform self-monitoring.

[0526] For example, when a user actively contributes opinions during a project meeting and receives positive feedback, the emotion engine perceives their high level of cooperation and motivation. Evaluation in such situations can contribute to improved performance afterward.

[0527] An example of a prompt message could be input to the generative AI model as, "Please tell me specifically about the system's process for collecting logs of user-performed digital activities and evaluating their emotional tendencies." This prompt allows for the extraction of detailed information about the system's operation and how to utilize it.

[0528] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0529] Step 1:

[0530] The device records user input text and online meeting speech. Input includes keyboard input and audio data via microphone. This data is periodically retrieved and saved as a log file. This logging generates a detailed activity record of how the user is operating the computer.

[0531] Step 2:

[0532] The terminal packages the collected log data and sends it to the server. This process uses a secure communication protocol to ensure data security. The input is the log data obtained in step 1, and the output is the secure transfer of data to the server. This procedure enables centralized, real-time data management.

[0533] Step 3:

[0534] The server first preprocesses the data received from the terminal. The input is unstructured log data, which is then formatted and broken down into necessary components. Data normalization and removal of unnecessary noise are performed, and the data is converted into a format suitable for analysis. This output is ready for the next analysis step.

[0535] Step 4:

[0536] The server uses natural language processing techniques to analyze pre-processed data. The input is formatted data, and an emotion engine recognizes emotional states from text and audio data. Specifically, it analyzes frequent patterns of certain words and changes in voice tone. The output is quantified evaluation data regarding the user's emotional state.

[0537] Step 5:

[0538] The server integrates the analyzed emotional state data and performs scoring to evaluate the user's work efficiency and contribution. The input is the evaluation data obtained in step 4, and the output is a numerical score for work efficiency and contribution. This process visualizes the user's overall performance.

[0539] Step 6:

[0540] Users visually view analysis results using a dashboard provided by the server. Input is scoring data from the server, and output is feedback in the form of graphs and charts that users can understand. This process allows users to analyze their own activities and identify areas for improvement.

[0541] (Application Example 2)

[0542] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0543] Understanding employee work efficiency and emotional states in real time and providing appropriate feedback is a critical challenge for many organizations. However, traditional methods have made it difficult to conduct detailed analyses of employee activities, including emotions, and to provide immediate feedback to improve productivity.

[0544] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0545] In this invention, the server includes means for collecting activity history from employee information processing devices, means for analyzing the activity history using natural language processing and machine learning technologies, and means for analyzing emotions and identifying the emotional state of the workers. This enables detailed analysis of employee activities and immediate feedback.

[0546] "Operation history" refers to data that records the details of operations and activities performed by employees using information processing equipment.

[0547] "Natural language processing technology" is a technology that uses computers to analyze human language and understand its meaning, emotions, and other aspects.

[0548] "Machine learning technology" refers to techniques that use algorithms to learn patterns from data and perform predictions and classifications.

[0549] "Emotional analysis" is a technology that identifies an employee's emotional state from their statements and behavioral data.

[0550] "Visualization" refers to making data understandable intuitively by displaying it visually using diagrams, charts, and other visual aids.

[0551] "Real-time" is a term that indicates that processing and analysis are performed almost simultaneously, and results are obtained without delay.

[0552] "Operational efficiency" is a quantifiable indicator that measures the efficiency with which employees perform their tasks.

[0553] "Emotional state" refers to an individual's temporary psychological state, and includes emotions such as joy, anxiety, and anger.

[0554] This invention is a system that analyzes the digital activity and emotional state of information processing devices used by employees to promote productivity improvement and self-improvement. Specifically, a log collection program is installed on the employee's information processing device to collect a history of their actions during work on a daily basis. A server periodically receives the collected data and analyzes it using natural language processing and machine learning technologies. The analysis also incorporates an emotion engine, which has the function of identifying emotions from speech and actions.

[0555] The server analysis results are aggregated as numerical report data of employee activities and fed back into a visually accessible dashboard. Through this dashboard, users can review their own activities and emotional tendencies and identify areas for improvement.

[0556] As a concrete example, smart glasses can be used to collect visual and auditory information from workers in real time. This makes it easy to manage productivity on-site and monitor employees' psychological health. If the goal to be achieved is set as "improving the quality of communication and boosting overall team morale," the generating AI model can suggest a more effective communication strategy using prompt sentences like the following.

[0557] Example prompt: "Create an analytical report to infer the emotional state of employees and identify potential areas for improvement in their work performance."

[0558] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0559] Step 1:

[0560] The terminal collects the operation history of the information processing device in real time using a log collection program. The input is employee operation data, and the output is saved as a log file. This log file includes a timestamp and action type.

[0561] Step 2:

[0562] The server periodically receives log files sent from terminals. The input is log files transferred from terminals, and the output is storage in a database. During storage in the database, data formatting is standardized and errors are checked.

[0563] Step 3:

[0564] The server utilizes natural language processing and machine learning techniques to analyze the collected data. The input is stored data in a database, and the output is a series of numerical data as a result of the analysis. In this process, an emotion engine is used to identify emotional states from text and audio data, analyzing the frequency of occurrence of specific words and changes in audio waveforms.

[0565] Step 4:

[0566] The server generates report data based on the analysis results and feeds it back into a dashboard for visualization. The input is numerical data from the analysis results, and the output is visual data displayed as graphs and charts. The dashboard displays work efficiency and emotional tendencies in a visually easy-to-understand format.

[0567] Step 5:

[0568] Users can review their activities and emotional tendencies through a dashboard, and develop strategies for business improvement by referring to the generated AI model and prompt messages. The input is visualized data, and the output is the user's own business improvement plan. Through this entire process, the aim is to improve employee productivity.

[0569] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0570] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0571] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0572] [Fourth Embodiment]

[0573] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0574] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0575] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0576] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0577] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0578] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0579] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0580] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0581] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0582] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0583] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0584] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0585] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0586] To implement this invention, it is necessary to install a program on a terminal used by an employee, which will serve to comprehensively collect the employee's digital activities. The terminal will routinely record the application operations performed by the user, the text entered, and the online services connected (email, chat tools, etc.), and transfer this data to a server in accordance with security protocols.

[0587] The server preprocesses the received data, performing data cleaning and format conversion as needed. Through this process, it analyzes the data using natural language processing techniques and machine learning algorithms. Natural language processing allows for the extraction of useful information from text data, such as identifying the emotions and themes of conversations.

[0588] The server also quantifies the data obtained through analysis and numerically evaluates each employee's performance. In online meetings, it analyzes the number of contributions and their constructiveness, and scores their contributions based on this. This makes it possible to clearly demonstrate the specific impact on projects and tasks.

[0589] Users can visualize the data stored on the server through the dashboard. Here, they can visually check individual activity history, efficiency scores, and the overall team performance. Based on this information, administrators can evaluate employees, and employees themselves can receive feedback to improve their work.

[0590] For example, if a user's contributions to a project primarily stem from their participation in remote meetings over a week, their frequency and quality of contributions will be highly valued and displayed as the best contribution on the dashboard. This allows users to receive tangible recognition in their work environment, leading to increased motivation.

[0591] This invention provides a system that comprehensively analyzes various aspects of digital activities and functions as a tool for evaluating employee performance, thus meeting the modern needs of promoting work style reform.

[0592] The following describes the processing flow.

[0593] Step 1:

[0594] The device records the user's access history to applications and web services in real time. During this process, information such as the type of operation, execution time, and details of the action are collected as logs.

[0595] Step 2:

[0596] The terminal sends collected log data to the server at regular intervals. Secure communication protocols are used for data transfer, and privacy protection is taken into consideration.

[0597] Step 3:

[0598] The server stores the received data in the database and simultaneously performs preprocessing. This involves cleaning and formatting the text data, preparing it for analysis.

[0599] Step 4:

[0600] The server applies natural language processing techniques to the pre-processed data, performing sentiment analysis and keyword extraction. This allows for a summary of the intent and content of the user's statements and inputs.

[0601] Step 5:

[0602] The server uses machine learning algorithms to analyze specific patterns and trends, and numerically evaluates contributions to work and the quality of contributions. This involves model-based analysis using historical data and rule-based analysis.

[0603] Step 6:

[0604] The server aggregates the analysis results and quantifies the allocation of work time, efficiency, and contribution for each user. This generates baseline data for individually evaluating performance.

[0605] Step 7:

[0606] Users access a dashboard to view their performance data. The server displays graphs and charts on this dashboard in a visually easy-to-understand format and provides them to the user.

[0607] This series of steps allows the system to analyze employees' digital activities in detail and perform objective evaluations.

[0608] (Example 1)

[0609] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0610] Traditional systems made it difficult to comprehensively and efficiently collect and analyze employees' digital activities. In particular, they could not accurately evaluate the quality and contribution of online meetings, and there was a lack of means to provide objective evaluation criteria. As a result, the evaluation of employee activities and the improvement of work efficiency were delayed, which was a challenge.

[0611] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0612] In this invention, the server includes means for collecting user activity from the operating environment, means for processing and organizing the collected activity data, and means for analyzing the data by combining natural language processing and machine learning algorithms to extract useful information. This makes it possible to comprehensively evaluate employees' digital activities and objectively assess the quality and contribution of their participation in online meetings.

[0613] "Operating environment" refers to the collective term for the computers, related software, and networks that users use when performing their work or tasks.

[0614] "User activity" refers to all digital actions performed by a user on a computer, including actions such as typing, clicking, and application usage history.

[0615] "Processing and organizing operational activity data" refers to a series of steps that involve cleaning up the collected raw data and converting it into a format that is easy to analyze.

[0616] "Natural language processing" refers to the technology of computational algorithms that enable computers to understand and analyze human language.

[0617] A "machine learning algorithm" is a mathematical model that uses large amounts of data to enable computers to improve themselves and perform efficient analysis and prediction.

[0618] "Extracting useful information" is the process of finding important indicators and trends from analyzed data that can be used to improve business operations and make decisions.

[0619] "Activity evaluation" is a method that aims to objectively assess the efficiency and effectiveness of employee performance in carrying out tasks by quantifying their performance.

[0620] "Quality and contribution of contributions in online meetings" refers to a standard for quantitatively measuring the importance and influence of what is said during a meeting.

[0621] To implement this system, a specific program must first be installed on the terminal. This program comprehensively collects operational activity data when the user is using the terminal and records activities related to daily work. The collected data is periodically sent to the server using secure communication. In this process, the terminal ensures the security of the data by using encrypted communication such as HTTPS.

[0622] The server then stores this collected data in a database and performs preprocessing. Specifically, it uses scripting languages ​​such as Python to clean and format the data. The prepared dataset is then further analyzed using natural language processing techniques and machine learning algorithms.

[0623] The server uses generative AI models to extract useful information from text data. Natural language processing enables sentiment analysis and theme identification of the text. For example, it can provide analysis results on the degree of positivity or negativity in conversation content.

[0624] Based on the analysis results, the server quantifies and evaluates user performance. By analyzing the number of contributions and their constructiveness in online meetings, the server can output a score representing the user's contribution. This evaluation result is visually displayed to the user through a dashboard. Users can easily check their own efficiency score and the overall performance of their team.

[0625] For example, if a user actively participates in a remote meeting and their contributions are highly valued for the project, they may be displayed on the dashboard as having the highest level of contribution. This allows users to visually confirm the evaluation of their activities and increase their motivation for their work.

[0626] An example of a prompt sentence to input into the generative AI model is, "Analyze and evaluate the user's statements and contributions." This system functions as a powerful tool for analyzing users' digital activities in detail and comprehensively, aiming to improve performance.

[0627] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0628] Step 1:

[0629] The terminal begins recording user activity. Specific inputs include application data used by the user, entered text, and access logs to online services. This data is collected in real time and output as encrypted logs. This data is temporarily stored in a buffer. The terminal's background processes ensure continuous data collection without interference from user activity.

[0630] Step 2:

[0631] The terminal transfers data to the server at regular intervals. Logs of collected operational activities are input and sent securely using the HTTPS protocol, outputting temporary log data. This transfer is performed periodically by batch processing to ensure data security and integrity.

[0632] Step 3:

[0633] The server stores the received data in a database. The input is log data sent from the terminal, which is converted into a structured data format and output. Specifically, a Python script is used to format the data, and the resulting data record is optimized for subsequent processing.

[0634] Step 4:

[0635] The server performs data cleansing, removing noise and inaccurate data. The input is the structured data stored in step 3, and it outputs a clean dataset through filtering. Specifically, it executes algorithms that perform anomaly detection and duplicate data removal.

[0636] Step 5:

[0637] The server performs analysis using natural language processing and machine learning algorithms. Clean data is used as input, and meaningful insights extracted through text analysis and sentiment analysis are output. The server utilizes generative AI models to perform tasks such as identifying the positivity and themes of conversations.

[0638] Step 6:

[0639] The server quantifies user performance based on the analysis results. The input is already analyzed data, and a corresponding evaluation score is output. A calculation algorithm is applied that visualizes the number of contributions and the depth of content, and calculates a specific contribution score.

[0640] Step 7:

[0641] Users access a dashboard on the server to view visualized data. The input is quantified performance data, which is output as visual graphs and charts. Users can use this information to evaluate their individual performance and improve overall team efficiency.

[0642] (Application Example 1)

[0643] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0644] In modern factories, there is a need to objectively evaluate work efficiency and operational precision and to find appropriate improvement measures tailored to the environment. However, conventional evaluation methods have the challenge of making it difficult to grasp the performance of operating equipment in real time and accurately, thus hindering efficient improvement. In addition, it is difficult to accurately classify work content and work time into specific tasks, requiring considerable effort to conduct detailed analysis. Furthermore, qualitative aspects such as work quality and contribution cannot be quantified, making it impossible to create concrete indicators for increasing added value. Therefore, new technologies are needed to solve these problems.

[0645] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0646] In this invention, the server includes means for collecting operational data from operational devices, means for using information processing technology and automatic learning methods to analyze the collected operational data, and means for quantifying the performance of the operational devices and generating a report based on the analysis results. This makes it possible to evaluate the work efficiency and accuracy of operational devices in a factory in real time and to quantify and visually present the quality and contribution of work. Furthermore, by providing appropriate indicators based on specific tasks, it contributes to the formulation and implementation of efficient improvement measures.

[0647] A "behavioral device" is a mechanical device that automatically performs a specific task and outputs operational data through sensors.

[0648] "Operation data" refers to digital information generated by an operating device during its work, which indicates its operating status and work efficiency.

[0649] "Information processing technology" refers to techniques for analyzing data and extracting meaningful information, and includes natural language processing and data mining.

[0650] "Automated learning methods" are techniques that use machine learning algorithms to autonomously learn patterns from data and perform predictions and classifications.

[0651] "Quantifying performance" refers to expressing the efficiency and accuracy of work numerically based on the operating data of the operating device.

[0652] A "report" is a document that summarizes the analysis results and includes evaluations and suggestions for improvement regarding the performance of the operating device.

[0653] "Visualization" refers to presenting numerical data and analysis results in a format that is easy for humans to understand, and involves using shapes and visual representations.

[0654] The server collects operational data in real time from behavioral devices installed in the factory and stores this data in a central database. This operational data includes the working status, operating time, efficiency, and accuracy of the behavioral devices. The server implements information processing technology and automated learning methods using programming languages ​​such as Python and R. This allows it to analyze the operational data and evaluate the efficiency and accuracy of the behavioral devices. It also generates detailed reports based on the results and provides an interface for visualizing these reports. This uses visual elements such as graphs and charts to make the data easier for users to understand visually.

[0655] Furthermore, based on the analysis results, the performance of the behavioral device can be quantitatively evaluated, and areas for improvement can be identified. This information can be used as an indicator to improve the operational efficiency of the factory.

[0656] As a concrete example, consider a screw-tightening robot placed on a manufacturing line. The server analyzes the motion data obtained from this robot and quantifies the accuracy and efficiency of screw tightening. This allows the user to know in advance when maintenance is needed and when parts need replacing, enabling them to take appropriate improvement measures.

[0657] An example of a prompt would be, "Use sensor data to evaluate the performance of the behavioral device in real time and propose specific improvement measures to enhance efficiency." This allows for advanced data analysis using a generative AI model, enabling the acquisition of practical insights.

[0658] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0659] Step 1:

[0660] The server periodically collects sensor data from the behavioral device. This sensor data includes the device's operating status, operating time, and operational efficiency. It receives raw sensor data as input and stores it in a database. The server preprocesses the received data, removing noise to output clean data.

[0661] Step 2:

[0662] The server uses Python to apply information processing techniques and automated learning methods to analyze clean sensor data. The input is clean sensor data, and the output is an evaluation result regarding the operational efficiency and accuracy of the behavioral device. In this process, statistical analysis of the data and model training are performed to predict the performance of the behavioral device.

[0663] Step 3:

[0664] The server generates a report quantifying the performance of the operating device based on the analysis results. The input is the analysis results, and the output is a detailed report. This report includes numerical data on the work efficiency and accuracy of each operating device, formatted in a user-readable format.

[0665] Step 4:

[0666] Users receive the generated report and review the data using a visualization interface. The input is the report data, and the output is visualized graphs and charts. This interface is designed to provide an intuitive understanding of the data, helping users identify important trends and patterns.

[0667] Step 5:

[0668] Based on the report, the user devises improvement measures for the behavioral device. This includes maintenance schedules to improve operational efficiency and adjustment plans to enhance operational accuracy. To support the user's decision-making, a generative AI model is used to generate prompt statements that produce further suggestions. These prompt statements serve as specific guidelines to support the user's decision-making.

[0669] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0670] To implement this invention, it is necessary to install a system on employee terminals that combines a program for collecting operation logs with an emotion engine. This system comprehensively records digital activities, such as how users operate their computers and what content they communicate using communication tools. The terminals routinely log text entered by users and content spoken in online meetings, and periodically transmit this data to a server.

[0671] The server first preprocesses the received data and analyzes it using natural language processing techniques. During this process, the emotion engine recognizes emotions from the user's text and voice data and integrates the results into the analyzed data. The emotion engine is responsible for identifying the user's emotional state, such as joy, anxiety, or anger, based on factors such as the frequency of specific words and changes in voice waveforms.

[0672] The analysis results are aggregated on the server as numerical data to evaluate the user's work efficiency, the quality of their contributions, and their overall contribution. This quantified data then forms a feedback loop to the terminal. For example, if a user makes positive and proactive statements during a project meeting, the emotion engine may detect this high level of motivation and report that it could positively impact the performance evaluation.

[0673] Users can visually review their activities and emotional tendencies through a dashboard provided by the server. This allows users to understand their own performance trends and devise steps for self-improvement as needed. This entire process functions as a powerful tool to promote employee productivity and self-monitoring. In this way, the present invention builds a system that provides more advanced performance management and insights by incorporating emotion recognition technology.

[0674] The following describes the processing flow.

[0675] Step 1:

[0676] The device records in real time the user's application usage, text input, and online meeting participation on their computer. This includes operation logs and audio recordings, which are stored securely with privacy in mind.

[0677] Step 2:

[0678] The device periodically collects data and sends it to the server via a secure communication protocol. Here, all transmitted data is encrypted to prevent unauthorized access by third parties.

[0679] Step 3:

[0680] The server first cleans the received data, removing noise and converting the data format. This prepares the data for analysis.

[0681] Step 4:

[0682] The server uses natural language processing technology to analyze text and audio data. As a result, it can identify the user's intentions and the content they are communicating.

[0683] Step 5:

[0684] The emotion engine is activated and recognizes the user's emotional state from the analyzed text and audio data. Specifically, it analyzes the vocabulary used and the tone of voice to estimate emotions such as "joy," "sadness," and "anger."

[0685] Step 6:

[0686] The server integrates the analysis results from the emotion engine into the dataset and uses them to quantify user performance, efficiency, and contribution. This data plays a crucial role as an employee evaluation metric.

[0687] Step 7:

[0688] Users access a dashboard to view visualized activity data and emotional trends. Here, the server provides graphs and charts related to emotional state and work efficiency, helping users monitor their own condition.

[0689] This process allows the system to comprehensively analyze employees' daily activities and suggest areas for improvement through emotion recognition.

[0690] (Example 2)

[0691] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0692] Traditional systems for quantifying and visually providing feedback on employee work efficiency and contributions have struggled to adequately analyze detailed emotional states and evaluate the quality of employee comments. Furthermore, there is a need to improve the accuracy of real-time evaluation and visualization when classifying comments and work time during online meetings. As a result, employees were unable to obtain guidance for self-improvement, leading to missed opportunities for increased productivity.

[0693] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0694] In this invention, the server includes means for acquiring work records from employee information processing devices, means for using data processing and computational models to analyze the acquired work records, and means for quantifying work and generating reports based on the analysis results. This makes it possible to accurately evaluate employees' work efficiency and contributions and provide visual feedback.

[0695] An "information processing device" is a device used for inputting, processing, storing, and outputting digital data, and usually refers to a computer.

[0696] "Work records" refer to logs of operations and activities performed by employees on information processing devices, including entered text and applications used.

[0697] "Data processing" refers to a series of operations that analyze collected digital data and transform it into useful information, typically including filtering and format normalization.

[0698] A "computational model" refers to mathematical and statistical methods used for data analysis and pattern recognition, and includes machine learning algorithms.

[0699] "Reporting materials" refer to summary documents and reports generated based on analyzed data, including those that numerically represent work efficiency and contribution.

[0700] "Visual feedback" refers to methods of displaying numerical data and evaluation results in a format that is easy for users to understand, and includes formats such as graphs and charts.

[0701] "Emotional state" refers to a psychological state recognized based on the user's text and voice, and includes various emotions such as joy, anger, sadness, and happiness.

[0702] "Contribution volume" refers to a quantitative indicator that shows how much an individual employee contributed to the work.

[0703] To implement this invention, it is necessary to install a system on the terminals used by employees that combines a program for recording operation logs with an engine for sentiment analysis. The terminals record daily operations and statements made during online meetings, and periodically transmit this data to a server. The purpose of this data is to comprehensively understand digital activities.

[0704] The terminal records user input and speech, collecting the data in an appropriate format. The hardware can be a standard computer or communication device. The software used will include log collection software and a client program incorporating an emotion engine.

[0705] The server analyzes the received data using natural language processing technology and recognizes emotional states using an emotion engine. In particular, it utilizes various data processing and computational models to evaluate the quality and impact of digital activities. The analysis results are quantified into reports, and feedback information is also generated to evaluate work efficiency and contribution.

[0706] Users can use the server-provided dashboard to view reports on their activities and sentiment tendencies. Based on this information, users can review their work performance and perform self-monitoring.

[0707] For example, when a user actively contributes opinions during a project meeting and receives positive feedback, the emotion engine perceives their high level of cooperation and motivation. Evaluation in such situations can contribute to improved performance afterward.

[0708] An example of a prompt message could be input to the generative AI model as, "Please tell me specifically about the system's process for collecting logs of user-performed digital activities and evaluating their emotional tendencies." This prompt allows for the extraction of detailed information about the system's operation and how to utilize it.

[0709] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0710] Step 1:

[0711] The device records user input text and online meeting speech. Input includes keyboard input and audio data via microphone. This data is periodically retrieved and saved as a log file. This logging generates a detailed activity record of how the user is operating the computer.

[0712] Step 2:

[0713] The terminal packages the collected log data and sends it to the server. This process uses a secure communication protocol to ensure data security. The input is the log data obtained in step 1, and the output is the secure transfer of data to the server. This procedure enables centralized, real-time data management.

[0714] Step 3:

[0715] The server first preprocesses the data received from the terminal. The input is unstructured log data, which is then formatted and broken down into necessary components. Data normalization and removal of unnecessary noise are performed, and the data is converted into a format suitable for analysis. This output is ready for the next analysis step.

[0716] Step 4:

[0717] The server uses natural language processing techniques to analyze pre-processed data. The input is formatted data, and an emotion engine recognizes emotional states from text and audio data. Specifically, it analyzes frequent patterns of certain words and changes in voice tone. The output is quantified evaluation data regarding the user's emotional state.

[0718] Step 5:

[0719] The server integrates the analyzed emotional state data and performs scoring to evaluate the user's work efficiency and contribution. The input is the evaluation data obtained in step 4, and the output is a numerical score for work efficiency and contribution. This process visualizes the user's overall performance.

[0720] Step 6:

[0721] Users visually view analysis results using a dashboard provided by the server. Input is scoring data from the server, and output is feedback in the form of graphs and charts that users can understand. This process allows users to analyze their own activities and identify areas for improvement.

[0722] (Application Example 2)

[0723] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0724] Understanding employee work efficiency and emotional states in real time and providing appropriate feedback is a critical challenge for many organizations. However, traditional methods have made it difficult to conduct detailed analyses of employee activities, including emotions, and to provide immediate feedback to improve productivity.

[0725] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0726] In this invention, the server includes means for collecting activity history from employee information processing devices, means for analyzing the activity history using natural language processing and machine learning technologies, and means for analyzing emotions and identifying the emotional state of the workers. This enables detailed analysis of employee activities and immediate feedback.

[0727] "Operation history" refers to data that records the details of operations and activities performed by employees using information processing equipment.

[0728] "Natural language processing technology" is a technology that uses computers to analyze human language and understand its meaning, emotions, and other aspects.

[0729] "Machine learning technology" refers to techniques that use algorithms to learn patterns from data and perform predictions and classifications.

[0730] "Emotional analysis" is a technology that identifies an employee's emotional state from their statements and behavioral data.

[0731] "Visualization" refers to making data understandable intuitively by displaying it visually using diagrams, charts, and other visual aids.

[0732] "Real-time" is a term that indicates that processing and analysis are performed almost simultaneously, and results are obtained without delay.

[0733] "Operational efficiency" is a quantifiable indicator that measures the efficiency with which employees perform their tasks.

[0734] "Emotional state" refers to an individual's temporary psychological state, and includes emotions such as joy, anxiety, and anger.

[0735] This invention is a system that analyzes the digital activity and emotional state of information processing devices used by employees to promote productivity improvement and self-improvement. Specifically, a log collection program is installed on the employee's information processing device to collect a history of their actions during work on a daily basis. A server periodically receives the collected data and analyzes it using natural language processing and machine learning technologies. The analysis also incorporates an emotion engine, which has the function of identifying emotions from speech and actions.

[0736] The server analysis results are aggregated as numerical report data of employee activities and fed back into a visually accessible dashboard. Through this dashboard, users can review their own activities and emotional tendencies and identify areas for improvement.

[0737] As a concrete example, smart glasses can be used to collect visual and auditory information from workers in real time. This makes it easy to manage productivity on-site and monitor employees' psychological health. If the goal to be achieved is set as "improving the quality of communication and boosting overall team morale," the generating AI model can suggest a more effective communication strategy using prompt sentences like the following.

[0738] Example prompt: "Create an analytical report to infer the emotional state of employees and identify potential areas for improvement in their work performance."

[0739] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0740] Step 1:

[0741] The terminal collects the operation history of the information processing device in real time using a log collection program. The input is employee operation data, and the output is saved as a log file. This log file includes a timestamp and action type.

[0742] Step 2:

[0743] The server periodically receives log files sent from terminals. The input is log files transferred from terminals, and the output is storage in a database. During storage in the database, data formatting is standardized and errors are checked.

[0744] Step 3:

[0745] The server utilizes natural language processing and machine learning techniques to analyze the collected data. The input is stored data in a database, and the output is a series of numerical data as a result of the analysis. In this process, an emotion engine is used to identify emotional states from text and audio data, analyzing the frequency of occurrence of specific words and changes in audio waveforms.

[0746] Step 4:

[0747] The server generates report data based on the analysis results and feeds it back into a dashboard for visualization. The input is numerical data from the analysis results, and the output is visual data displayed as graphs and charts. The dashboard displays work efficiency and emotional tendencies in a visually easy-to-understand format.

[0748] Step 5:

[0749] Users can review their activities and emotional tendencies through a dashboard, and develop strategies for business improvement by referring to the generated AI model and prompt messages. The input is visualized data, and the output is the user's own business improvement plan. Through this entire process, the aim is to improve employee productivity.

[0750] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0751] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0752] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0753] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0754] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0755] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0756] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0757] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0758] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0759] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0760] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0761] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0762] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0763] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0764] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0765] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0766] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0767] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0768] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0769] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0770] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0771] The following is further disclosed regarding the embodiments described above.

[0772] (Claim 1)

[0773] A means of collecting operation logs from employees' computers,

[0774] To analyze the collected operation logs, we use methods employing natural language processing and machine learning algorithms,

[0775] A means of quantifying employee activities and generating reports based on analysis results,

[0776] To visualize the generated report, methods using graphs and charts are available.

[0777] A system that includes this.

[0778] (Claim 2)

[0779] A means of collecting and recording the content of online meetings in real time,

[0780] A method for analyzing recorded statements and scoring their quality and contribution,

[0781] The system according to claim 1, further comprising:

[0782] (Claim 3)

[0783] A means of classifying employee work time into specific projects or tasks,

[0784] A means for calculating an index to evaluate work efficiency using classified data,

[0785] The system according to claim 1, further comprising:

[0786] "Example 1"

[0787] (Claim 1)

[0788] A means of collecting user activity from the operating environment,

[0789] A means of processing and organizing the collected operational activity data,

[0790] A means of analyzing data and extracting useful information by combining natural language processing and machine learning algorithms,

[0791] A means of quantitatively evaluating user activity based on information obtained through analysis,

[0792] To visualize the generated evaluation results, a means of using graphic display means is provided,

[0793] A system that includes this.

[0794] (Claim 2)

[0795] A means of collecting information about what is said during a meeting in real time using communication technology,

[0796] A means of quantifying the quality and contribution of statements based on collected statement information,

[0797] The system according to claim 1, including the following:

[0798] (Claim 3)

[0799] A means of classifying activity time into specific work items or tasks,

[0800] A means for generating numerical values ​​to evaluate activity efficiency based on classified time data,

[0801] The system according to claim 1, including the following:

[0802] "Application Example 1"

[0803] (Claim 1)

[0804] A means for collecting motion data from a behavioral device,

[0805] To analyze the collected motion data, means using information processing technology and automated learning methods,

[0806] A means for quantifying the performance of the behavioral device based on the analysis results and generating a report,

[0807] To visualize the generated report, methods using shapes and visual representations,

[0808] A system that includes this.

[0809] (Claim 2)

[0810] A means for immediately collecting and recording the operation details of the operating device,

[0811] A means of analyzing recorded work to quantify the quality and contribution of the work,

[0812] The system according to claim 1, further comprising:

[0813] (Claim 3)

[0814] A means for classifying the working time of an action device into specific processes or tasks,

[0815] A means for calculating criteria for evaluating work efficiency using classified data,

[0816] The system according to claim 1, further comprising:

[0817] "Example 2 of combining an emotion engine"

[0818] (Claim 1)

[0819] A means of obtaining work records from employee information processing devices,

[0820] To analyze the acquired work records, a means of using data processing and computational models is employed.

[0821] A means of quantifying employee work based on analysis results and generating reports,

[0822] To visualize the generated report data, methods using image displays and charts are employed.

[0823] A means for using an analysis mechanism to recognize emotional states from the user's language data and voice data,

[0824] A means for integrating analyzed emotional states to evaluate work efficiency and contribution levels,

[0825] A system that includes this.

[0826] (Claim 2)

[0827] A means of instantly acquiring and recording the content of statements made in online group communications,

[0828] A means of analyzing recorded statements to evaluate the quality of the statements and the degree of tribute,

[0829] The system according to claim 1, further comprising:

[0830] (Claim 3)

[0831] A means of classifying employee work time into specific tasks or activities,

[0832] A means for calculating an index to evaluate operational efficiency using classified information,

[0833] The system according to claim 1, further comprising:

[0834] "Application example 2 when combining with an emotional engine"

[0835] (Claim 1)

[0836] A means of collecting operation history from employee information processing devices,

[0837] To analyze the collected behavioral history, a means of using natural language processing and machine learning techniques is employed.

[0838] A means of quantifying employee activities and generating report data based on the analysis results,

[0839] To visualize the generated report data, methods using figures and charts are employed,

[0840] A means of analyzing emotions and identifying the emotional state of workers,

[0841] A means for calculating an index that evaluates work efficiency and emotional tendencies based on emotional state,

[0842] A system that includes this.

[0843] (Claim 2)

[0844] A means of collecting and recording the content of online communications in real time,

[0845] A method for analyzing recorded statements and quantifying their quality and contribution,

[0846] A means of using an emotion engine to identify emotions from statements and reflect them in evaluations,

[0847] The system according to claim 1, further comprising:

[0848] (Claim 3)

[0849] A means of classifying employee work time into specific tasks or activities,

[0850] A means for calculating an index to evaluate work efficiency using classified data,

[0851] A means using a device for collecting visual and auditory information of the work,

[0852] The system according to claim 1, further comprising: [Explanation of symbols]

[0853] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of collecting operation logs from employees' computers, To analyze the collected operation logs, we use methods employing natural language processing and machine learning algorithms, A means of quantifying employee activities and generating reports based on analysis results, To visualize the generated report, methods using graphs and charts are available. A system that includes this.

2. A means of collecting and recording the content of online meetings in real time, A method for analyzing recorded statements and scoring their quality and contribution, The system according to claim 1, further comprising:

3. A means of classifying employee work time into specific projects or tasks, A means for calculating an index to evaluate work efficiency using classified data, The system according to claim 1, further comprising:

Images