system

Abstract

We provide the system. [Solution] Information processing means for generating individual profile data, A generation means for automatically generating personalized content based on the generated individual profile, A response generation means that generates and provides responses in real time to user questions, A machine learning method that improves the system's response accuracy by receiving feedback, A system that includes this.

Description

Technical Field

[0003] ,

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, the method including receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Conventional training programs and skill improvement methods are uniform, and it is difficult to provide personalized support according to the growth needs of each individual. As a result, there is a problem that specific questions held by employees cannot be quickly answered, and an efficient skill improvement process is hindered. Furthermore, there are also limitations in grasping the growth status of employees and effective learning methods, and as a result, the improvement of the overall skill level of the entire company is delayed. There is a problem that the maximum ability of human resources in a company is hindered by this.

Means for Solving the Problems

[0005] This invention provides a means for generating individual profile data and automatically generating personalized content suitable for each employee based on this data. Furthermore, it includes means for generating real-time responses to employee questions to support effective learning. It also incorporates machine learning means to improve the accuracy of the system itself through feedback, monitors employee growth, and generates reports according to their growth, thereby promoting company-wide skill improvement. In this way, it provides a learning environment tailored to individual needs and enables employees to maximize their potential.

[0006] "Individual profile data" refers to a personalized set of information that includes user performance, skills, goals, and other details.

[0007] "Information processing means" refers to devices or programs that collect, analyze, and store data, and generate individual profile data.

[0008] "Generation means" refers to devices or programs that use AI technology and algorithms to create customized content based on user profile data.

[0009] A "response generation means" is a device or program that immediately generates and provides an appropriate answer to a user's inquiry or question.

[0010] "Machine learning methods" are technical techniques that continuously improve the accuracy and functionality of a system's response through the analysis of data and feedback.

[0011] "Multimedia delivery means" refers to devices and programs that provide generated content to users in various formats such as text, audio, and video.

[0012] "Monitoring" is the process of continuously tracking employees' growth and learning progress and recording information as needed.

[0013] A "report generation means" is a device or program that creates reports on user growth and performance based on collected data. [Brief explanation of the drawing]

[0014] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14]It is a sequence diagram showing the processing flow of a data processing system in Application Example 2 when a sentiment engine is combined.

Embodiments for Carrying Out the Invention

[0015] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0018] In the following embodiments, a numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0019] In the following embodiments, a numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0020] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0022] [First Embodiment]

[0023] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0024] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0027] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0030] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0034] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0035] This invention is a system for realizing a personalized mentorship program that efficiently supports the skill development of each employee within a company. A specific embodiment of this system is described below.

[0036] The server first collects profile data for each employee. This includes past work performance, skill assessments, and growth goals based on self-analysis. After collecting the information, the server analyzes the data and evaluates the gap between the employee's current abilities and their goals.

[0037] Subsequently, the server uses a generation AI to create personalized content best suited to each employee. This content includes specific skill improvement methods, industry success stories, and strategies for achieving goals, and is delivered in various media formats. The terminals present this content to employees, creating an environment where they can learn intuitively and interactively.

[0038] When employees have questions or problems, they send their inquiries to the server via their devices. The server then uses AI to generate immediate responses in real time, providing specific solutions and additional resources. These responses are optimized based on employee profiles and up-to-date data analysis.

[0039] Furthermore, as users provide feedback on each piece of content, the server collects and learns from this data, constantly improving the system's accuracy and usefulness. This allows the quality of the content and responses provided to continuously evolve, making employee learning more efficient and effective.

[0040] For example, if a sales employee is exploring sales methods for a new product, the server will present a customized strategy based on the employee's sales style and market trends. The user can then review this strategy and apply it to their actual sales activities via their terminal. Furthermore, by providing feedback on the results of these activities, the system can achieve greater accuracy and effectiveness in future strategy proposals.

[0041] Thus, the invention provides a means to promote individual growth support for each employee, improve the overall skill level of the organization, and maximize the potential of human resources.

[0042] The following describes the processing flow.

[0043] Step 1:

[0044] The server retrieves each employee's performance data, skill assessments, and learning history information from internal databases and external data sources to create employee profiles.

[0045] Step 2:

[0046] The device allows users to input their personal growth goals and areas of interest, and collects data based on this input. This user input is used to understand individual needs.

[0047] Step 3:

[0048] The server analyzes user input and existing profile data to identify gaps in the skill set needed for growth. Based on this analysis, it prepares to generate personalized content.

[0049] Step 4:

[0050] The server uses generative AI to automatically generate appropriate learning content based on the user's profile data and learning goals. This content can be provided in various formats, including text, audio, and video.

[0051] Step 5:

[0052] The device presents the user with generated personalized content, providing an interactive learning experience. The user can then use this content to progress through their learning.

[0053] Step 6:

[0054] When a user has a specific question or concern, they enter it through their device. This information is sent to the server, and processing begins immediately.

[0055] Step 7:

[0056] The server generates responses to user questions in real time, including relevant information and solutions. These generated responses are then presented to the user via the terminal.

[0057] Step 8:

[0058] Users contribute to system improvement by providing feedback on the content and responses they receive.

[0059] Step 9:

[0060] The server uses machine learning algorithms based on feedback to improve the system's response accuracy and the usefulness of its content.

[0061] Step 10:

[0062] The server monitors the user's learning progress and provides the user with regular progress reports. These reports can be used to help plan future learning.

[0063] (Example 1)

[0064] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0065] In today's business environment, efficiently supporting the skill development of individual employees is essential for maintaining the competitiveness of the entire organization. However, traditional, uniform training programs have failed to adequately address individual growth needs, resulting in ineffective skill improvement. Furthermore, providing personalized instruction tailored to each employee requires significant human resources, making it difficult to achieve.

[0066] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0067] In this invention, the server includes information processing means for collecting personal characteristic information, analysis means for analyzing the collected characteristic information and evaluating the difference between abilities and goals, and content generation means for generating personalized content based on the evaluation. This makes it possible to provide each employee with optimized learning content and advice.

[0068] "Individual characteristic information" refers to attribute data related to each individual, such as work performance, skill evaluations, and growth goals.

[0069] "Information processing means" refers to devices and programs used to collect, store, and analyze data.

[0070] "Analysis methods" refer to processes and systems for evaluating collected data and analyzing the difference between the current situation and the target.

[0071] "Content generation methods" refer to technologies that automatically create individually optimized learning materials and strategies based on analysis results.

[0072] "Multimedia delivery methods" refer to technologies for providing generated content to users in various formats, such as text, audio, and video.

[0073] A "response generation means" refers to a process or technology for instantly generating and responding to a user's question.

[0074] "Learning methods" refer to methods and techniques for improving the performance of a system based on collected feedback information.

[0075] A "report generation method" is a function that continuously tracks an individual's growth and generates reports on that progress.

[0076] "Improvement measures" refer to methods and techniques for analyzing collected feedback and making future service provision more effective.

[0077] This invention aims to realize a personalized learning support system that efficiently improves the skills of individual employees within organizations such as companies. The embodiments thereof will be described in detail below.

[0078] The server first collects individual employee profile data. This data is retrieved using APIs from databases such as employee management systems and business management systems. The data includes employee work performance, skill assessments, and growth goals based on self-assessment. This information is properly stored and managed using a database management system (DBMS).

[0079] Next, the server analyzes the collected data. This analysis uses analytical libraries such as Python's pandas and scikit-learn, and R's dplyr, to clean the data and assess the gap between employees' current capabilities and their targets. The results are then visualized using visualization tools to present them in a format easily understandable to managers and supervisors.

[0080] Next, the server uses a generative AI model (for example, a model based on natural language processing) to generate personalized content optimized for each employee based on the analysis results. The prompt will be in the following format: "This user (Employee ID: 12345) is aiming to improve their IT skills. They currently have intermediate-level programming skills and want to improve their data analysis skills. Please suggest efficient ways for this user to improve their data analysis skills."

[0081] The terminals are responsible for providing employees with content generated by the server. Through web or mobile applications running on the terminals, users can receive content in various media formats and progress with their learning. Examples include text, presentation slides, and video streaming.

[0082] If a user has a question during their learning process, they can send it to the server via their device. The server, using generative AI, instantly generates a response, providing specific solutions and supplementary materials. This response is provided in real time, ensuring support for deepening the user's understanding.

[0083] Finally, the feedback provided by users on the learning content is collected by the server and used in future content generation processes. This feedback loop continuously improves the system's adaptability and accuracy, enabling more effective learning support.

[0084] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0085] Step 1:

[0086] The server connects to the company's data repository and HR system to collect each employee's profile data. Input data includes employee work performance, skill assessments, and growth goals. The data is retrieved via API and stored in a database. Specifically, it executes queries to extract the necessary data. The output is a structured version of each employee's profile data.

[0087] Step 2:

[0088] The server analyzes the collected profile data and calculates the difference between each employee's abilities and their growth targets. The input for this step is the profile data obtained in step 1. For data analysis, libraries such as Python's pandas are used to clean the data and perform statistical processing. Specifically, numerical evaluation indicators are compared to clarify skill gaps. The output is the analyzed skill gap data for each employee.

[0089] Step 3:

[0090] The server generates personalized content using a generative AI model based on the analysis results. The input is the skill gap data from step 2. By generating prompt sentences and feeding them into the AI ​​model, optimal learning content is generated. Specifically, prompts are sent to the generative AI, and the response is received as text. The output is learning content tailored to each employee.

[0091] Step 4:

[0092] The terminal provides employees with personalized content received from the server. The input is the learning content generated in step 3. The terminal displays the content through a web application or mobile application, allowing users to progress through their learning. Specifically, it plays text and videos using a browser or dedicated application. The output is the learning content displayed to the user.

[0093] Step 5:

[0094] The user sends questions that arise during the learning process to the server via their device. The input is the question text entered by the user. The server receives the question and generates an answer in real time using a generative AI. Specifically, it inputs the question as a prompt into the generative AI model and returns the obtained answer as text. The output is the answer presented to the user.

[0095] Step 6:

[0096] Users provide feedback on learning content to the server via their devices. Input consists of opinions and ratings entered through feedback forms and rating buttons. The server stores this data in a database and uses it for future content creation. Specifically, it performs tasks such as compiling survey data and analyzing feedback data. Output is the feedback data accumulated for improvement.

[0097] (Application Example 1)

[0098] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0099] Providing individualized support to maximize the performance of automated equipment, particularly robots used in factories, is challenging. Furthermore, the lack of mechanisms to efficiently provide optimal instructions in real time hinders the achievement of sufficient productivity improvements.

[0100] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0101] In this invention, the server includes information processing means for generating individual characteristic data, generation means for automatically generating personalized advice based on the generated characteristic data, and performance analysis means for analyzing performance data using a data acquisition device. This makes it possible to provide optimal advice in real time based on the characteristics of each automated device.

[0102] "Individual characteristic data" refers to detailed information about the characteristics and state of a specific individual or device, which is used for performance analysis and optimization of that individual or device.

[0103] "Information processing means" refers to a device or system that has the function of collecting, processing, and analyzing diverse data, and that effectively utilizes data for a specific purpose.

[0104] "Personalized advice" refers to information that provides guidance and suggestions optimized for the characteristics and condition of a specific individual or device, and that contributes to improving the capabilities of that individual or device.

[0105] "Generation means" refers to a device or system that has the function of generating information or advice based on input data.

[0106] A "data collection device" is a device used to collect various types of data for a specific purpose, and the collected data is used for further analysis.

[0107] "Performance analysis means" refers to a device or system for analyzing collected data and evaluating the performance and operation of an individual or device.

[0108] "Instruction supply means" refers to a device or system that has the function of appropriately providing individualized instructions or information to a target.

[0109] "Feedback" is information that evaluates the success and areas for improvement of a specific action or result, and is used to inform future actions and plans.

[0110] "Machine learning techniques" are technologies that use past data and feedback to update models and achieve more accurate information provision and predictions.

[0111] This invention is a system for improving the performance of automated equipment in specific environments, particularly in factories. The system aims to maximize the characteristics of each piece of equipment by providing personalized advice.

[0112] The server is equipped with information processing means to generate individual characteristic data for each automated device, and analyzes the collected data to understand its characteristics. In addition, it acquires performance data in real time from each operating device using a data acquisition device, and analyzes the obtained data in detail using performance analysis means.

[0113] The server generates personalized advice using a generative AI model based on the analyzed data. The generated advice is provided to automated equipment in an appropriate format via an instruction supply means. This allows each piece of equipment to achieve efficient operation in real time.

[0114] Furthermore, the system's accuracy is improved using machine learning techniques based on user feedback. This feedback helps in the subsequent advice generation process, continuously optimizing the performance of each device.

[0115] As a concrete example, in the product assembly process, the server analyzes motion data collected by the robot arm and provides real-time instructions to optimize the movement route, ensuring efficient operation. These instructions are individually customized using a generative AI model, thereby increasing work efficiency.

[0116] An example of a prompt message is: "Analyze the performance data of robot A and generate specific advice for efficiency improvement. Consider the current production line status and provide instructions for optimizing its operation in real time."

[0117] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0118] Step 1:

[0119] The server receives basic information and operational status data of automated equipment from a data acquisition device. Based on this input data, an information processing system processes the data to generate individual characteristic data. This characteristic data includes details about the equipment's performance and operating patterns.

[0120] Step 2:

[0121] The server analyzes the generated individual characteristic data using performance analysis tools. This analysis performs data calculations to identify efficient and inefficient operating points of the equipment. The analysis results identify areas requiring optimization and areas where operation can be improved. The output is the analysis progress and its evaluation.

[0122] Step 3:

[0123] The server uses an AI model based on the analysis results to generate personalized advice. The analysis results are used as input, and the AI ​​model uses them to generate optimal operation sequences and adjustment suggestions. The output consists of specific advice, ready to be supplied as instructions to each device.

[0124] Step 4:

[0125] The server transmits the generated advice to the automated equipment in real time using an instruction supply mechanism. The instructions sent out serve as guidelines for the equipment to operate efficiently and are then executed. The output shows the improvement in the equipment's operation.

[0126] Step 5:

[0127] The user monitors the improvement status of the equipment and inputs feedback into the system. This feedback is processed using machine learning methods to evaluate the degree of effectiveness. Based on the input feedback, the system's AI model is updated, contributing to improved processing accuracy in subsequent processes. The output is the improved AI model.

[0128] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0129] This invention realizes a system for recognizing the emotional state of each employee and providing a personalized learning experience based on that state. The embodiments thereof are described in detail below.

[0130] The server first creates an individual profile for each employee. This is generated by retrieving information such as past work performance, skill evaluations, and growth goals from a database and performing data analysis. This profile data is used as foundational information and is used to generate personalized learning content.

[0131] The emotion engine collects user voice input and facial expression data to analyze the user's emotional state in real time. The device sends this data to the emotion engine, which identifies the user's emotions such as joy, interest, and stress. This information helps to understand the user's current emotional state.

[0132] The server generates learning content based on the user's profile and emotional data. This content is optimized for the user's emotional state at any given time and is delivered in an appropriate format and tone. For example, if the user is feeling stressed, relaxing content will be selected.

[0133] When a user has a specific question or concern, they input it through their device and send it to the server. The server then generates a response that incorporates the sentiment analysis results from the emotion engine and provides it to the user immediately. This allows the user to receive advice and information that is appropriate to their mental state at that time.

[0134] Furthermore, user feedback is collected by the server and used in machine learning algorithms to improve system accuracy and content usefulness. User growth is continuously monitored, and periodic reports are generated. These reports reflect the user's emotional state and learning progress, suggesting the next steps to maintain user motivation and achieve goals.

[0135] As a concrete example, if an employee feels anxious about the progress of a new project and the emotion engine detects "stress," the server will prioritize providing learning content that helps reduce stress and suggest practical actions to improve subsequent work efficiency. In this way, the invention provides a personalized learning environment tailored to each individual's emotional state, embodying a means to maximize employee growth and work efficiency.

[0136] The following describes the processing flow.

[0137] Step 1:

[0138] The server retrieves performance data, skill assessments, and past learning history from the company database to create employee profiles. This data is used to set employee growth goals and identify skill gaps.

[0139] Step 2:

[0140] The device collects emotional data in real time through the user's voice input and facial recognition. It sends basic information to the emotion engine to detect the user's emotional state, such as joy or stress.

[0141] Step 3:

[0142] The emotion engine analyzes acquired voice and facial expression data to identify the user's instantaneous emotional state. This information is used to clearly understand what emotions the user is feeling.

[0143] Step 4:

[0144] The server analyzes user profile information and emotional data to generate the most relevant personalized content for that moment. If the user is feeling stressed, relaxing content or encouraging messages will be provided.

[0145] Step 5:

[0146] The device presents the user with personalized content provided by the server. This content is delivered in various formats, including text, audio, and video, and is presented in an appropriate tone that matches the user's emotional state.

[0147] Step 6:

[0148] If a user has a question or doubt, they can input it using their device. The device then sends the entered information to the server.

[0149] Step 7:

[0150] The server generates answers to user questions that reflect the results of the emotion engine. In particular, it sends back answers to the terminal that are adjusted in tone and content according to the user's emotional state.

[0151] Step 8:

[0152] Users provide feedback on the presented content and answers. This feedback is sent to the server via their device.

[0153] Step 9:

[0154] The server analyzes the collected feedback and uses machine learning algorithms to improve response accuracy based on the data accumulated on the system side. This process increases the quality of the content provided next time and the relevance of the response.

[0155] Step 10:

[0156] The server monitors the user's learning progress and emotional state, and generates regular growth reports based on this data. It also provides the user with suggestions for the next learning steps and recommended action plans.

[0157] (Example 2)

[0158] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0159] Traditional learning systems struggle to provide personalized content tailored to each user's emotional state and characteristics, resulting in insufficient support for efficient learning and growth. Furthermore, they are unable to respond promptly and appropriately to user inquiries, leading to decreased user satisfaction.

[0160] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0161] In this invention, the server includes data processing means for generating individual characteristic data, information generation means for automatically generating specialized information based on the generated individual characteristics, and emotion analysis means for analyzing audio and video data to evaluate the emotional state. This enables the provision of personalized content according to the user's emotional state and the generation of quick and appropriate responses.

[0162] "Individual characteristic data" refers to information about individual users, including past work performance, skill evaluations, and growth goals.

[0163] "Data processing means" refers to a method or apparatus for obtaining information from a database, analyzing characteristic data, and generating a profile.

[0164] "Specialized information" refers to content for learning and information provision that is customized based on the user's current emotional state and characteristic data.

[0165] "Information generation means" refers to a method or apparatus that has the function of automatically generating useful information from data and providing appropriate content to the user.

[0166] "Emotion analysis means" refers to a method or apparatus for evaluating a user's emotions based on audio or video data and performing further processing based on that evaluation.

[0167] "Learning algorithms" refer to algorithms used to improve system performance and response accuracy by utilizing user feedback.

[0168] "Information provision means" refers to a method or apparatus for providing generated specialized information to users in various media formats.

[0169] The following configuration is necessary to carry out this invention.

[0170] The server first retrieves information such as each user's work performance, skill evaluation, and growth goals from the database. This is used to create individual characteristic data. The Pandas library in Python is used to manipulate the data and perform profile analysis. The server also stores the generated characteristic data in physical storage and generates specialized information based on it. A generative AI model is used for the automatic generation of specialized information, optimizing the learning content and information provision.

[0171] The device collects the user's voice and facial expressions in real time through its camera and microphone. The collected data is converted to a specific format and sent to a server. A cloud API is used for voice analysis, and image processing techniques using libraries such as OpenCV are applied to the facial expression data.

[0172] The server performs emotion analysis based on audio and video data. It uses a machine learning model as its emotion engine to classify the user's emotional state into categories such as "joy," "interest," and "stress," which are then used as important elements in creating specialized information.

[0173] Users can input specific questions or inquiries through their device. The input information is sent to a server, where a generating AI model analyzes it and provides an appropriate response. The response is tailored based on the user's emotional data and is provided in real time. An example of a prompt is, "Please suggest the best way for the user to relax if they are feeling stressed."

[0174] Furthermore, user feedback is collected on the server and analyzed by a learning algorithm. This feedback is used to improve the system's accuracy and enhance the quality of specialized information. User progress is also tracked on the server, and reports are generated based on the results, offering suggestions for the next steps.

[0175] For example, if a user feels anxious about starting a new project, the system classifies that emotion as "stress" using emotion analysis tools, and generates and provides related information with a relaxing effect. Through this process, the present invention provides a personalized learning environment for each user.

[0176] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0177] Step 1:

[0178] The server retrieves information such as each user's work performance, skill evaluation, and growth goals from the database. Using this information as input, it generates individual characteristic data using a data processing tool. Specifically, it extracts data using SQL queries, formats the data using the Python Pandas library, and creates a profile. The output of this step is characteristic data for each user.

[0179] Step 2:

[0180] The device collects the user's voice and facial expressions in real time through the microphone and camera. This collected audio and video data becomes the input for step 2. After collection, the data is converted to a specific format and sent to the server. Specifically, the audio is converted to text by a cloud API, and the video data is analyzed using OpenCV and processed into a format necessary for sentiment analysis. The output of this step is the audio and video data necessary for sentiment analysis.

[0181] Step 3:

[0182] The server uses the audio and video data acquired in step 2 as input to evaluate the emotional state using an emotion analysis tool. The emotion engine uses a machine learning model to perform a specific analysis that classifies the data into emotions such as "joy," "interest," and "stress." The model is executed using an ML framework such as TENSORFLOW®, and the emotion results are obtained as output. This output, along with the characteristic data, is used in the next step.

[0183] Step 4:

[0184] The server generates specialized information using information generation means based on individual characteristic data and sentiment analysis results. Considering the characteristic data and emotional state used as input, a generation AI model is used to generate optimal learning content for the user. An example of an AI model used in this process is the Transformer architecture. The output of this step is user-optimized learning content.

[0185] Step 5:

[0186] The user inputs a specific question or inquiry through a terminal. This input is sent to a server and analyzed by a response generation system. A generative AI model generates the optimal response while considering sentiment data. Specifically, it uses a natural language processing model to generate a response sentence based on the input prompt. The output of this step is the response message provided to the user.

[0187] Step 6:

[0188] The server collects user feedback as input. This feedback is analyzed using learning algorithms to improve system accuracy and user experience. The feedback is then used to retrain the model, further enhancing system performance. The output of this step is the improved system performance and evaluation.

[0189] (Application Example 2)

[0190] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0191] In today's work environment, providing real-time work guidance and support tailored to the emotional state of individual workers is challenging. Traditional systems fail to provide individualized guidance based on workers' stress levels and concentration, resulting in insufficient improvement of work efficiency and poor stress management. This can lead to decreased worker productivity and workplace dissatisfaction.

[0192] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0193] In this invention, the server includes data processing means for generating individual configuration information, configuration means for automatically generating personalized content based on the generated individual configuration information, response generation means for immediately generating and supplying responses to user inquiries, learning means for utilizing user feedback to improve the accuracy of responses, and emotion analysis means for analyzing the emotional state of workers and adjusting procedures based on those emotions. This enables individualized guidance and real-time support tailored to the emotional state of workers.

[0194] "Individual configuration information" refers to data generated based on past experiences and skill evaluations accumulated for each user.

[0195] "Data processing means" refers to devices or software that have the function of collecting and analyzing various types of information to create configuration information.

[0196] "Personalized content" refers to customized information and guidance that is appropriately selected based on the user's configuration information and current situation.

[0197] "Configuration means" refers to devices or software that have the function of assembling content based on individual configuration information and providing it in a format suitable for the user.

[0198] "User inquiries" refer to questions and requests that users make to the system.

[0199] A "response generation means" refers to a device or software that has the function of creating and providing appropriate answers in real time to user inquiries.

[0200] "Response" refers to the opinions and feedback that users give regarding the system's responses or the content it provides.

[0201] A "learning tool" is a device or software that analyzes responses and automatically improves the accuracy and efficiency of the entire system.

[0202] "Emotional state" refers to information that indicates the user's current psychological or emotional condition.

[0203] "Emotional analysis tools" refer to devices or software that have the function of evaluating and determining an emotional state using the user's voice and facial expression data.

[0204] In this invention, the server is implemented by integrating multiple means having different functions. First, the server uses data processing means to generate individual configuration information based on the user's past experience, skill evaluation, growth goals, etc. This information is stored for each user and then automatically generated as individualized content by the server's configuration means.

[0205] The terminal receives a query from the user and sends it to the server. The server's response generation mechanism creates an appropriate response to this query in real time and provides it to the user through the terminal. In this process, to further improve the accuracy of the response, the user's response is analyzed by the server's learning mechanism, and the system's accuracy is automatically improved.

[0206] Furthermore, the terminal collects voice and facial expression data and transfers it to the server. The server's emotion analysis system uses this data to evaluate the user's emotional state and adjusts the work procedures based on that emotional state, especially if the worker is feeling fatigued or stressed.

[0207] For example, if a worker feels fatigued on the production line, the emotion analysis system detects this, and the server generates a message such as, "We recommend you take a break." Workers who are highly focused will be instructed, "Let's proceed to the next step." This enables real-time guidance and support tailored to the work environment.

[0208] An example of a prompt message would be, "Please suggest the best course of action based on your current emotional state."

[0209] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0210] Step 1:

[0211] The server collects basic user information and generates individual configuration information using data processing tools. In this process, the user's past experience and skill assessment data are used as input, and the individual configuration information is output through analysis. Specifically, it retrieves relevant information from the database and applies data analysis algorithms.

[0212] Step 2:

[0213] The terminal receives user inquiries as input and sends them to the server. The input queries are parsed through a natural language processing engine and passed to the server's response generation system. Specifically, the user provides voice instructions or text input, which the terminal converts into digital data and sends to the server.

[0214] Step 3:

[0215] The server generates an appropriate response in real time to the received query using a response generation mechanism. In this process, the AI ​​model generates a response using the input query, and the result is sent to the terminal as output. Specifically, the AI ​​model analyzes the query content and constructs the optimal answer.

[0216] Step 4:

[0217] The device collects the user's voice and facial expression data and sends it to a server for emotion analysis. Voice data and video feeds are used as input, which are processed in real time and converted into data indicating the user's emotional state. Specifically, voice recognition software and facial recognition algorithms are executed.

[0218] Step 5:

[0219] The server adjusts work procedures based on the results of emotion analysis. Data indicating emotional state is processed as input, and work procedures and support actions are generated as output based on this data. For example, if stress is indicated, suggestions for reducing the workload will be made.

[0220] Step 6:

[0221] The terminal receives user responses and sends them to the server as feedback. The feedback data is used as input, and the response generation and learning mechanisms are adjusted based on it. Specifically, the user's evaluation is recorded in a database, contributing to improved response accuracy in subsequent interactions.

[0222] Step 7:

[0223] The server improves the overall system functionality based on collected feedback data through learning mechanisms. By analyzing the input feedback and adjusting the generative AI model, it provides more accurate and efficient responses and support as output. Specifically, it applies machine learning algorithms and updates the system parameters.

[0224] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0225] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0226] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0227] [Second Embodiment]

[0228] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0229] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0230] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0231] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0232] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0233] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0234] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0235] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0236] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0237] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0238] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0239] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0240] This invention is a system for realizing a personalized mentorship program that efficiently supports the skill development of each employee within a company. A specific embodiment of this system is described below.

[0241] The server first collects profile data for each employee. This includes past work performance, skill assessments, and growth goals based on self-analysis. After collecting the information, the server analyzes the data and evaluates the gap between the employee's current abilities and their goals.

[0242] Subsequently, the server uses a generation AI to create personalized content best suited to each employee. This content includes specific skill improvement methods, industry success stories, and strategies for achieving goals, and is delivered in various media formats. The terminals present this content to employees, creating an environment where they can learn intuitively and interactively.

[0243] When employees have questions or problems, they send their inquiries to the server via their devices. The server then uses AI to generate immediate responses in real time, providing specific solutions and additional resources. These responses are optimized based on employee profiles and up-to-date data analysis.

[0244] Furthermore, as users provide feedback on each piece of content, the server collects and learns from this data, constantly improving the system's accuracy and usefulness. This allows the quality of the content and responses provided to continuously evolve, making employee learning more efficient and effective.

[0245] For example, if a sales employee is exploring sales methods for a new product, the server will present a customized strategy based on the employee's sales style and market trends. The user can then review this strategy and apply it to their actual sales activities via their terminal. Furthermore, by providing feedback on the results of these activities, the system can achieve greater accuracy and effectiveness in future strategy proposals.

[0246] Thus, the invention provides a means to promote individual growth support for each employee, improve the overall skill level of the organization, and maximize the potential of human resources.

[0247] The following describes the processing flow.

[0248] Step 1:

[0249] The server retrieves each employee's performance data, skill assessments, and learning history information from internal databases and external data sources to create employee profiles.

[0250] Step 2:

[0251] The device allows users to input their personal growth goals and areas of interest, and collects data based on this input. This user input is used to understand individual needs.

[0252] Step 3:

[0253] The server analyzes user input and existing profile data to identify gaps in the skill set needed for growth. Based on this analysis, it prepares to generate personalized content.

[0254] Step 4:

[0255] The server uses generative AI to automatically generate appropriate learning content based on the user's profile data and learning goals. This content can be provided in various formats, including text, audio, and video.

[0256] Step 5:

[0257] The device presents the user with generated personalized content, providing an interactive learning experience. The user can then use this content to progress through their learning.

[0258] Step 6:

[0259] When a user has a specific question or concern, they enter it through their device. This information is sent to the server, and processing begins immediately.

[0260] Step 7:

[0261] The server generates responses to user questions in real time, including relevant information and solutions. These generated responses are then presented to the user via the terminal.

[0262] Step 8:

[0263] Users contribute to system improvement by providing feedback on the content and responses they receive.

[0264] Step 9:

[0265] The server uses machine learning algorithms based on feedback to improve the system's response accuracy and the usefulness of its content.

[0266] Step 10:

[0267] The server monitors the user's learning progress and provides the user with regular progress reports. These reports can be used to help plan future learning.

[0268] (Example 1)

[0269] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0270] In today's business environment, efficiently supporting the skill development of individual employees is essential for maintaining the competitiveness of the entire organization. However, traditional, uniform training programs have failed to adequately address individual growth needs, resulting in ineffective skill improvement. Furthermore, providing personalized instruction tailored to each employee requires significant human resources, making it difficult to achieve.

[0271] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0272] In this invention, the server includes information processing means for collecting personal characteristic information, analysis means for analyzing the collected characteristic information and evaluating the difference between abilities and goals, and content generation means for generating personalized content based on the evaluation. This makes it possible to provide each employee with optimized learning content and advice.

[0273] "Individual characteristic information" refers to attribute data related to each individual, such as work performance, skill evaluations, and growth goals.

[0274] "Information processing means" refers to devices and programs used to collect, store, and analyze data.

[0275] "Analysis methods" refer to processes and systems for evaluating collected data and analyzing the difference between the current situation and the target.

[0276] "Content generation methods" refer to technologies that automatically create individually optimized learning materials and strategies based on analysis results.

[0277] "Multimedia delivery methods" refer to technologies for providing generated content to users in various formats, such as text, audio, and video.

[0278] A "response generation means" refers to a process or technology for instantly generating and responding to a user's question.

[0279] "Learning methods" refer to methods and techniques for improving the performance of a system based on collected feedback information.

[0280] A "report generation method" is a function that continuously tracks an individual's growth and generates reports on that progress.

[0281] "Improvement measures" refer to methods and techniques for analyzing collected feedback and making future service provision more effective.

[0282] This invention realizes a personalized learning support system that efficiently improves the skills of individual employees within an organization such as a company. The embodiments thereof will be described in detail below.

[0283] The server first collects the profile data of individual employees. This data is obtained from databases such as an employee management system or a business management system using an API. The data includes employees' work performance, skill evaluations, and growth goals based on self-analysis. This information is appropriately stored and managed using a database management system (DBMS).

[0284] Next, the server analyzes the collected data. For the analysis, analysis libraries such as pandas and scikit-learn in Python and dplyr in R are used to clean the data and evaluate the difference between the current situation and the goals of employees' capabilities. The results are arranged in a form that can be easily understood by administrators and instructors using visualization tools.

[0285] Subsequently, based on this analysis result, the server uses a generative AI model (for example, a model based on natural language processing) to generate personalized content optimized for each employee. The following format is used as the prompt text. "This user (employee ID: 12345) aims to improve their IT skills. Currently, they have intermediate-level programming skills and want to improve their data analysis skills. Please propose an efficient way to improve the user's data analysis skills."

[0286] The terminal plays the role of providing the content generated by the server to employees. Through a web application or a mobile application operating on the terminal, users can receive the content in various media formats and proceed with their learning. For example, this corresponds to text, presentation slides, video distribution, etc.

[0287] If a user has a question during their learning process, they can send it to the server via their device. The server, using generative AI, instantly generates a response, providing specific solutions and supplementary materials. This response is provided in real time, ensuring support for deepening the user's understanding.

[0288] Finally, the feedback provided by users on the learning content is collected by the server and used in future content generation processes. This feedback loop continuously improves the system's adaptability and accuracy, enabling more effective learning support.

[0289] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0290] Step 1:

[0291] The server connects to the company's data repository and HR system to collect each employee's profile data. Input data includes employee work performance, skill assessments, and growth goals. The data is retrieved via API and stored in a database. Specifically, it executes queries to extract the necessary data. The output is a structured version of each employee's profile data.

[0292] Step 2:

[0293] The server analyzes the collected profile data and calculates the difference between each employee's abilities and their growth targets. The input for this step is the profile data obtained in step 1. For data analysis, libraries such as Python's pandas are used to clean the data and perform statistical processing. Specifically, numerical evaluation indicators are compared to clarify skill gaps. The output is the analyzed skill gap data for each employee.

[0294] Step 3:

[0295] The server generates personalized content using a generative AI model based on the analysis results. The input is the skill gap data from step 2. By generating prompt sentences and feeding them into the AI ​​model, optimal learning content is generated. Specifically, prompts are sent to the generative AI, and the response is received as text. The output is learning content tailored to each employee.

[0296] Step 4:

[0297] The terminal provides employees with personalized content received from the server. The input is the learning content generated in step 3. The terminal displays the content through a web application or mobile application, allowing users to progress through their learning. Specifically, it plays text and videos using a browser or dedicated application. The output is the learning content displayed to the user.

[0298] Step 5:

[0299] The user sends questions that arise during the learning process to the server via their device. The input is the question text entered by the user. The server receives the question and generates an answer in real time using a generative AI. Specifically, it inputs the question as a prompt into the generative AI model and returns the obtained answer as text. The output is the answer presented to the user.

[0300] Step 6:

[0301] Users provide feedback on learning content to the server via their devices. Input consists of opinions and ratings entered through feedback forms and rating buttons. The server stores this data in a database and uses it for future content creation. Specifically, it performs tasks such as compiling survey data and analyzing feedback data. Output is the feedback data accumulated for improvement.

[0302] (Application Example 1)

[0303] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as a "server", and the smart glasses 214 are referred to as a "terminal".

[0304] It is difficult to provide individualized support for automated equipment, especially robots used in factories, to maximize their respective performances. In addition, there is a problem that productivity improvement has not been sufficiently achieved because there is a lack of a mechanism to efficiently provide optimal instructions in real time.

[0305] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0306] In this invention, the server includes information processing means for generating individual characteristic data, generation means for automatically generating individualized advice based on the generated characteristic data, and performance analysis means for analyzing performance data using a data collection device. As a result, it becomes possible to provide optimal advice in real time based on the characteristics of each automated device.

[0307] "Individual characteristic data" is detailed information regarding the characteristics and states of a specific individual or device, and is used for performance analysis and optimization of that individual or device.

[0308] "Information processing means" is a device or system having a function of collecting, processing, and analyzing various data, and effectively utilizes data for a specific purpose.

[0309] "Individualized advice" is information that provides guidance and suggestions optimized for the characteristics and states of a specific individual or device, and contributes to improving the capabilities of that individual or device.

[0310] [[ID=SO]]"Generation means" is a device or system having a function of generating information and advice based on input data.

[0311] A "data collection device" is a device used to collect various types of data for a specific purpose, and the collected data is used for further analysis.

[0312] "Performance analysis means" refers to a device or system for analyzing collected data and evaluating the performance and operation of an individual or device.

[0313] "Instruction supply means" refers to a device or system that has the function of appropriately providing individualized instructions or information to a target.

[0314] "Feedback" is information that evaluates the success and areas for improvement of a specific action or result, and is used to inform future actions and plans.

[0315] "Machine learning techniques" are technologies that use past data and feedback to update models and achieve more accurate information provision and predictions.

[0316] This invention is a system for improving the performance of automated equipment in specific environments, particularly in factories. The system aims to maximize the characteristics of each piece of equipment by providing personalized advice.

[0317] The server is equipped with information processing means to generate individual characteristic data for each automated device, and analyzes the collected data to understand its characteristics. In addition, it acquires performance data in real time from each operating device using a data acquisition device, and analyzes the obtained data in detail using performance analysis means.

[0318] The server generates personalized advice using a generative AI model based on the analyzed data. The generated advice is provided to automated equipment in an appropriate format via an instruction supply means. This allows each piece of equipment to achieve efficient operation in real time.

[0319] Furthermore, the system's accuracy is improved using machine learning techniques based on user feedback. This feedback helps in the subsequent advice generation process, continuously optimizing the performance of each device.

[0320] As a concrete example, in the product assembly process, the server analyzes motion data collected by the robot arm and provides real-time instructions to optimize the movement route, ensuring efficient operation. These instructions are individually customized using a generative AI model, thereby increasing work efficiency.

[0321] An example of a prompt message is: "Analyze the performance data of robot A and generate specific advice for efficiency improvement. Consider the current production line status and provide instructions for optimizing its operation in real time."

[0322] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0323] Step 1:

[0324] The server receives basic information and operational status data of automated equipment from a data acquisition device. Based on this input data, an information processing system processes the data to generate individual characteristic data. This characteristic data includes details about the equipment's performance and operating patterns.

[0325] Step 2:

[0326] The server analyzes the generated individual characteristic data using performance analysis tools. This analysis performs data calculations to identify efficient and inefficient operating points of the equipment. The analysis results identify areas requiring optimization and areas where operation can be improved. The output is the analysis progress and its evaluation.

[0327] Step 3:

[0328] The server uses an AI model based on the analysis results to generate personalized advice. The analysis results are used as input, and the AI ​​model uses them to generate optimal operation sequences and adjustment suggestions. The output consists of specific advice, ready to be supplied as instructions to each device.

[0329] Step 4:

[0330] The server transmits the generated advice to the automated equipment in real time using an instruction supply mechanism. The instructions sent out serve as guidelines for the equipment to operate efficiently and are then executed. The output shows the improvement in the equipment's operation.

[0331] Step 5:

[0332] The user monitors the improvement status of the equipment and inputs feedback into the system. This feedback is processed using machine learning methods to evaluate the degree of effectiveness. Based on the input feedback, the system's AI model is updated, contributing to improved processing accuracy in subsequent processes. The output is the improved AI model.

[0333] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0334] This invention realizes a system for recognizing the emotional state of each employee and providing a personalized learning experience based on that state. The embodiments thereof are described in detail below.

[0335] The server first creates an individual profile for each employee. This is generated by retrieving information such as past work performance, skill evaluations, and growth goals from a database and performing data analysis. This profile data is used as foundational information and is used to generate personalized learning content.

[0336] The emotion engine collects user voice input and facial expression data to analyze the user's emotional state in real time. The device sends this data to the emotion engine, which identifies the user's emotions such as joy, interest, and stress. This information helps to understand the user's current emotional state.

[0337] The server generates learning content based on the user's profile and emotional data. This content is optimized for the user's emotional state at any given time and is delivered in an appropriate format and tone. For example, if the user is feeling stressed, relaxing content will be selected.

[0338] When a user has a specific question or concern, they input it through their device and send it to the server. The server then generates a response that incorporates the sentiment analysis results from the emotion engine and provides it to the user immediately. This allows the user to receive advice and information that is appropriate to their mental state at that time.

[0339] Furthermore, user feedback is collected by the server and used in machine learning algorithms to improve system accuracy and content usefulness. User growth is continuously monitored, and periodic reports are generated. These reports reflect the user's emotional state and learning progress, suggesting the next steps to maintain user motivation and achieve goals.

[0340] As a concrete example, if an employee feels anxious about the progress of a new project and the emotion engine detects "stress," the server will prioritize providing learning content that helps reduce stress and suggest practical actions to improve subsequent work efficiency. In this way, the invention provides a personalized learning environment tailored to each individual's emotional state, embodying a means to maximize employee growth and work efficiency.

[0341] The following describes the processing flow.

[0342] Step 1:

[0343] The server retrieves performance data, skill assessments, and past learning history from the company database to create employee profiles. This data is used to set employee growth goals and identify skill gaps.

[0344] Step 2:

[0345] The device collects emotional data in real time through the user's voice input and facial recognition. It sends basic information to the emotion engine to detect the user's emotional state, such as joy or stress.

[0346] Step 3:

[0347] The emotion engine analyzes acquired voice and facial expression data to identify the user's instantaneous emotional state. This information is used to clearly understand what emotions the user is feeling.

[0348] Step 4:

[0349] The server analyzes user profile information and emotional data to generate the most relevant personalized content for that moment. If the user is feeling stressed, relaxing content or encouraging messages will be provided.

[0350] Step 5:

[0351] The device presents the user with personalized content provided by the server. This content is delivered in various formats, including text, audio, and video, and is presented in an appropriate tone that matches the user's emotional state.

[0352] Step 6:

[0353] If a user has a question or doubt, they can input it using their device. The device then sends the entered information to the server.

[0354] Step 7:

[0355] The server generates answers to user questions that reflect the results of the emotion engine. In particular, it sends back answers to the terminal that are adjusted in tone and content according to the user's emotional state.

[0356] Step 8:

[0357] Users provide feedback on the presented content and answers. This feedback is sent to the server via their device.

[0358] Step 9:

[0359] The server analyzes the collected feedback and uses machine learning algorithms to improve response accuracy based on the data accumulated on the system side. This process increases the quality of the content provided next time and the relevance of the response.

[0360] Step 10:

[0361] The server monitors the user's learning progress and emotional state, and generates regular growth reports based on this data. It also provides the user with suggestions for the next learning steps and recommended action plans.

[0362] (Example 2)

[0363] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0364] Traditional learning systems struggle to provide personalized content tailored to each user's emotional state and characteristics, resulting in insufficient support for efficient learning and growth. Furthermore, they are unable to respond promptly and appropriately to user inquiries, leading to decreased user satisfaction.

[0365] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0366] In this invention, the server includes data processing means for generating individual characteristic data, information generation means for automatically generating specialized information based on the generated individual characteristics, and emotion analysis means for analyzing audio and video data to evaluate the emotional state. This enables the provision of personalized content according to the user's emotional state and the generation of quick and appropriate responses.

[0367] "Individual characteristic data" refers to information about individual users, including past work performance, skill evaluations, and growth goals.

[0368] "Data processing means" refers to a method or apparatus for obtaining information from a database, analyzing characteristic data, and generating a profile.

[0369] "Specialized information" refers to content for learning and information provision that is customized based on the user's current emotional state and characteristic data.

[0370] "Information generation means" refers to a method or apparatus that has the function of automatically generating useful information from data and providing appropriate content to the user.

[0371] "Emotion analysis means" refers to a method or apparatus for evaluating a user's emotions based on audio or video data and performing further processing based on that evaluation.

[0372] "Learning algorithms" refer to algorithms used to improve system performance and response accuracy by utilizing user feedback.

[0373] "Information provision means" refers to a method or apparatus for providing generated specialized information to users in various media formats.

[0374] The following configuration is necessary to carry out this invention.

[0375] The server first retrieves information such as each user's work performance, skill evaluation, and growth goals from the database. This is used to create individual characteristic data. The Pandas library in Python is used to manipulate the data and perform profile analysis. The server also stores the generated characteristic data in physical storage and generates specialized information based on it. A generative AI model is used for the automatic generation of specialized information, optimizing the learning content and information provision.

[0376] The device collects the user's voice and facial expressions in real time through its camera and microphone. The collected data is converted to a specific format and sent to a server. A cloud API is used for voice analysis, and image processing techniques using libraries such as OpenCV are applied to the facial expression data.

[0377] The server performs emotion analysis based on audio and video data. It uses a machine learning model as its emotion engine to classify the user's emotional state into categories such as "joy," "interest," and "stress," which are then used as important elements in creating specialized information.

[0378] Users can input specific questions or inquiries through their device. The input information is sent to a server, where a generating AI model analyzes it and provides an appropriate response. The response is tailored based on the user's emotional data and is provided in real time. An example of a prompt is, "Please suggest the best way for the user to relax if they are feeling stressed."

[0379] Furthermore, user feedback is collected on the server and analyzed by a learning algorithm. This feedback is used to improve the system's accuracy and enhance the quality of specialized information. User progress is also tracked on the server, and reports are generated based on the results, offering suggestions for the next steps.

[0380] For example, if a user feels anxious about starting a new project, the system classifies that emotion as "stress" using emotion analysis tools, and generates and provides related information with a relaxing effect. Through this process, the present invention provides a personalized learning environment for each user.

[0381] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0382] Step 1:

[0383] The server retrieves information such as each user's work performance, skill evaluation, and growth goals from the database. Using this information as input, it generates individual characteristic data using a data processing tool. Specifically, it extracts data using SQL queries, formats the data using the Python Pandas library, and creates a profile. The output of this step is characteristic data for each user.

[0384] Step 2:

[0385] The device collects the user's voice and facial expressions in real time through the microphone and camera. This collected audio and video data becomes the input for step 2. After collection, the data is converted to a specific format and sent to the server. Specifically, the audio is converted to text by a cloud API, and the video data is analyzed using OpenCV and processed into a format necessary for sentiment analysis. The output of this step is the audio and video data necessary for sentiment analysis.

[0386] Step 3:

[0387] The server uses the audio and video data acquired in step 2 as input to evaluate the emotional state using an emotion analysis tool. The emotion engine uses a machine learning model to perform a specific analysis that classifies the data into emotions such as "joy," "interest," and "stress." The model is executed using an ML framework such as TensorFlow, and the emotion results are obtained as output. This output, along with the characteristic data, is used in the next step.

[0388] Step 4:

[0389] The server generates specialized information using information generation means based on individual characteristic data and sentiment analysis results. Considering the characteristic data and emotional state used as input, a generation AI model is used to generate optimal learning content for the user. An example of an AI model used in this process is the Transformer architecture. The output of this step is user-optimized learning content.

[0390] Step 5:

[0391] The user inputs a specific question or inquiry through a terminal. This input is sent to a server and analyzed by a response generation system. A generative AI model generates the optimal response while considering sentiment data. Specifically, it uses a natural language processing model to generate a response sentence based on the input prompt. The output of this step is the response message provided to the user.

[0392] Step 6:

[0393] The server collects user feedback as input. This feedback is analyzed using learning algorithms to improve system accuracy and user experience. The feedback is then used to retrain the model, further enhancing system performance. The output of this step is the improved system performance and evaluation.

[0394] (Application Example 2)

[0395] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0396] In today's work environment, providing real-time work guidance and support tailored to the emotional state of individual workers is challenging. Traditional systems fail to provide individualized guidance based on workers' stress levels and concentration, resulting in insufficient improvement of work efficiency and poor stress management. This can lead to decreased worker productivity and workplace dissatisfaction.

[0397] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0398] In this invention, the server includes data processing means for generating individual configuration information, configuration means for automatically generating personalized content based on the generated individual configuration information, response generation means for immediately generating and supplying responses to user inquiries, learning means for utilizing user feedback to improve the accuracy of responses, and emotion analysis means for analyzing the emotional state of workers and adjusting procedures based on those emotions. This enables individualized guidance and real-time support tailored to the emotional state of workers.

[0399] "Individual configuration information" refers to data generated based on past experiences and skill evaluations accumulated for each user.

[0400] "Data processing means" refers to devices or software that have the function of collecting and analyzing various types of information to create configuration information.

[0401] "Personalized content" refers to customized information and guidance that is appropriately selected based on the user's configuration information and current situation.

[0402] "Configuration means" refers to devices or software that have the function of assembling content based on individual configuration information and providing it in a format suitable for the user.

[0403] "User inquiries" refer to questions and requests that users make to the system.

[0404] A "response generation means" refers to a device or software that has the function of creating and providing appropriate answers in real time to user inquiries.

[0405] "Response" refers to the opinions and feedback that users give regarding the system's responses or the content it provides.

[0406] A "learning tool" is a device or software that analyzes responses and automatically improves the accuracy and efficiency of the entire system.

[0407] "Emotional state" refers to information that indicates the user's current psychological or emotional condition.

[0408] "Emotional analysis tools" refer to devices or software that have the function of evaluating and determining an emotional state using the user's voice and facial expression data.

[0409] In this invention, the server is implemented by integrating multiple means having different functions. First, the server uses data processing means to generate individual configuration information based on the user's past experience, skill evaluation, growth goals, etc. This information is stored for each user and then automatically generated as individualized content by the server's configuration means.

[0410] The terminal receives a query from the user and sends it to the server. The server's response generation mechanism creates an appropriate response to this query in real time and provides it to the user through the terminal. In this process, to further improve the accuracy of the response, the user's response is analyzed by the server's learning mechanism, and the system's accuracy is automatically improved.

[0411] Furthermore, the terminal collects voice and facial expression data and transfers it to the server. The server's emotion analysis system uses this data to evaluate the user's emotional state and adjusts the work procedures based on that emotional state, especially if the worker is feeling fatigued or stressed.

[0412] For example, if a worker feels fatigued on the production line, the emotion analysis system detects this, and the server generates a message such as, "We recommend you take a break." Workers who are highly focused will be instructed, "Let's proceed to the next step." This enables real-time guidance and support tailored to the work environment.

[0413] An example of a prompt message would be, "Please suggest the best course of action based on your current emotional state."

[0414] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0415] Step 1:

[0416] The server collects basic user information and generates individual configuration information using data processing tools. In this process, the user's past experience and skill assessment data are used as input, and the individual configuration information is output through analysis. Specifically, it retrieves relevant information from the database and applies data analysis algorithms.

[0417] Step 2:

[0418] The terminal receives user inquiries as input and sends them to the server. The input queries are parsed through a natural language processing engine and passed to the server's response generation system. Specifically, the user provides voice instructions or text input, which the terminal converts into digital data and sends to the server.

[0419] Step 3:

[0420] The server generates an appropriate response in real time to the received query using a response generation mechanism. In this process, the AI ​​model generates a response using the input query, and the result is sent to the terminal as output. Specifically, the AI ​​model analyzes the query content and constructs the optimal answer.

[0421] Step 4:

[0422] The device collects the user's voice and facial expression data and sends it to a server for emotion analysis. Voice data and video feeds are used as input, which are processed in real time and converted into data indicating the user's emotional state. Specifically, voice recognition software and facial recognition algorithms are executed.

[0423] Step 5:

[0424] The server adjusts work procedures based on the results of emotion analysis. Data indicating emotional state is processed as input, and work procedures and support actions are generated as output based on this data. For example, if stress is indicated, suggestions for reducing the workload will be made.

[0425] Step 6:

[0426] The terminal receives user responses and sends them to the server as feedback. The feedback data is used as input, and the response generation and learning mechanisms are adjusted based on it. Specifically, the user's evaluation is recorded in a database, contributing to improved response accuracy in subsequent interactions.

[0427] Step 7:

[0428] The server improves the overall system functionality based on collected feedback data through learning mechanisms. By analyzing the input feedback and adjusting the generative AI model, it provides more accurate and efficient responses and support as output. Specifically, it applies machine learning algorithms and updates the system parameters.

[0429] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0430] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0431] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0432] [Third Embodiment]

[0433] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0434] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0435] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0436] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0437] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0438] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0439] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0440] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0441] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0442] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0443] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0444] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0445] This invention is a system for realizing a personalized mentorship program that efficiently supports the skill development of each employee within a company. A specific embodiment of this system is described below.

[0446] The server first collects profile data for each employee. This includes past work performance, skill assessments, and growth goals based on self-analysis. After collecting the information, the server analyzes the data and evaluates the gap between the employee's current abilities and their goals.

[0447] Subsequently, the server uses a generation AI to create personalized content best suited to each employee. This content includes specific skill improvement methods, industry success stories, and strategies for achieving goals, and is delivered in various media formats. The terminals present this content to employees, creating an environment where they can learn intuitively and interactively.

[0448] When employees have questions or problems, they send their inquiries to the server via their devices. The server then uses AI to generate immediate responses in real time, providing specific solutions and additional resources. These responses are optimized based on employee profiles and up-to-date data analysis.

[0449] Furthermore, as users provide feedback on each piece of content, the server collects and learns from this data, constantly improving the system's accuracy and usefulness. This allows the quality of the content and responses provided to continuously evolve, making employee learning more efficient and effective.

[0450] For example, if a sales employee is exploring sales methods for a new product, the server will present a customized strategy based on the employee's sales style and market trends. The user can then review this strategy and apply it to their actual sales activities via their terminal. Furthermore, by providing feedback on the results of these activities, the system can achieve greater accuracy and effectiveness in future strategy proposals.

[0451] Thus, the invention provides a means to promote individual growth support for each employee, improve the overall skill level of the organization, and maximize the potential of human resources.

[0452] The following describes the processing flow.

[0453] Step 1:

[0454] The server retrieves each employee's performance data, skill assessments, and learning history information from internal databases and external data sources to create employee profiles.

[0455] Step 2:

[0456] The device allows users to input their personal growth goals and areas of interest, and collects data based on this input. This user input is used to understand individual needs.

[0457] Step 3:

[0458] The server analyzes user input and existing profile data to identify gaps in the skill set needed for growth. Based on this analysis, it prepares to generate personalized content.

[0459] Step 4:

[0460] The server uses generative AI to automatically generate appropriate learning content based on the user's profile data and learning goals. This content can be provided in various formats, including text, audio, and video.

[0461] Step 5:

[0462] The device presents the user with generated personalized content, providing an interactive learning experience. The user can then use this content to progress through their learning.

[0463] Step 6:

[0464] When a user has a specific question or concern, they enter it through their device. This information is sent to the server, and processing begins immediately.

[0465] Step 7:

[0466] The server generates responses to user questions in real time, including relevant information and solutions. These generated responses are then presented to the user via the terminal.

[0467] Step 8:

[0468] Users contribute to system improvement by providing feedback on the content and responses they receive.

[0469] Step 9:

[0470] The server uses machine learning algorithms based on feedback to improve the system's response accuracy and the usefulness of its content.

[0471] Step 10:

[0472] The server monitors the user's learning progress and provides the user with regular progress reports. These reports can be used to help plan future learning.

[0473] (Example 1)

[0474] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0475] In today's business environment, efficiently supporting the skill development of individual employees is essential for maintaining the competitiveness of the entire organization. However, traditional, uniform training programs have failed to adequately address individual growth needs, resulting in ineffective skill improvement. Furthermore, providing personalized instruction tailored to each employee requires significant human resources, making it difficult to achieve.

[0476] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0477] In this invention, the server includes information processing means for collecting personal characteristic information, analysis means for analyzing the collected characteristic information and evaluating the difference between abilities and goals, and content generation means for generating personalized content based on the evaluation. This makes it possible to provide each employee with optimized learning content and advice.

[0478] "Individual characteristic information" refers to attribute data related to each individual, such as work performance, skill evaluations, and growth goals.

[0479] "Information processing means" refers to devices and programs used to collect, store, and analyze data.

[0480] "Analysis methods" refer to processes and systems for evaluating collected data and analyzing the difference between the current situation and the target.

[0481] "Content generation methods" refer to technologies that automatically create individually optimized learning materials and strategies based on analysis results.

[0482] "Multimedia delivery methods" refer to technologies for providing generated content to users in various formats, such as text, audio, and video.

[0483] A "response generation means" refers to a process or technology for instantly generating and responding to a user's question.

[0484] "Learning methods" refer to methods and techniques for improving the performance of a system based on collected feedback information.

[0485] A "report generation method" is a function that continuously tracks an individual's growth and generates reports on that progress.

[0486] "Improvement measures" refer to methods and techniques for analyzing collected feedback and making future service provision more effective.

[0487] This invention aims to realize a personalized learning support system that efficiently improves the skills of individual employees within organizations such as companies. The embodiments thereof will be described in detail below.

[0488] The server first collects individual employee profile data. This data is retrieved using APIs from databases such as employee management systems and business management systems. The data includes employee work performance, skill assessments, and growth goals based on self-assessment. This information is properly stored and managed using a database management system (DBMS).

[0489] Next, the server analyzes the collected data. This analysis uses analytical libraries such as Python's pandas and scikit-learn, and R's dplyr, to clean the data and assess the gap between employees' current capabilities and their targets. The results are then visualized using visualization tools to present them in a format easily understandable to managers and supervisors.

[0490] Next, the server uses a generative AI model (for example, a model based on natural language processing) to generate personalized content optimized for each employee based on the analysis results. The prompt will be in the following format: "This user (Employee ID: 12345) is aiming to improve their IT skills. They currently have intermediate-level programming skills and want to improve their data analysis skills. Please suggest efficient ways for this user to improve their data analysis skills."

[0491] The terminals are responsible for providing employees with content generated by the server. Through web or mobile applications running on the terminals, users can receive content in various media formats and progress with their learning. Examples include text, presentation slides, and video streaming.

[0492] If a user has a question during their learning process, they can send it to the server via their device. The server, using generative AI, instantly generates a response, providing specific solutions and supplementary materials. This response is provided in real time, ensuring support for deepening the user's understanding.

[0493] Finally, the feedback provided by users on the learning content is collected by the server and used in future content generation processes. This feedback loop continuously improves the system's adaptability and accuracy, enabling more effective learning support.

[0494] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0495] Step 1:

[0496] The server connects to the company's data repository and HR system to collect each employee's profile data. Input data includes employee work performance, skill assessments, and growth goals. The data is retrieved via API and stored in a database. Specifically, it executes queries to extract the necessary data. The output is a structured version of each employee's profile data.

[0497] Step 2:

[0498] The server analyzes the collected profile data and calculates the difference between each employee's abilities and their growth targets. The input for this step is the profile data obtained in step 1. For data analysis, libraries such as Python's pandas are used to clean the data and perform statistical processing. Specifically, numerical evaluation indicators are compared to clarify skill gaps. The output is the analyzed skill gap data for each employee.

[0499] Step 3:

[0500] The server generates personalized content using a generative AI model based on the analysis results. The input is the skill gap data from step 2. By generating prompt sentences and feeding them into the AI ​​model, optimal learning content is generated. Specifically, prompts are sent to the generative AI, and the response is received as text. The output is learning content tailored to each employee.

[0501] Step 4:

[0502] The terminal provides employees with personalized content received from the server. The input is the learning content generated in step 3. The terminal displays the content through a web application or mobile application, allowing users to progress through their learning. Specifically, it plays text and videos using a browser or dedicated application. The output is the learning content displayed to the user.

[0503] Step 5:

[0504] The user sends questions that arise during the learning process to the server via their device. The input is the question text entered by the user. The server receives the question and generates an answer in real time using a generative AI. Specifically, it inputs the question as a prompt into the generative AI model and returns the obtained answer as text. The output is the answer presented to the user.

[0505] Step 6:

[0506] Users provide feedback on learning content to the server via their devices. Input consists of opinions and ratings entered through feedback forms and rating buttons. The server stores this data in a database and uses it for future content creation. Specifically, it performs tasks such as compiling survey data and analyzing feedback data. Output is the feedback data accumulated for improvement.

[0507] (Application Example 1)

[0508] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0509] Providing individualized support to maximize the performance of automated equipment, particularly robots used in factories, is challenging. Furthermore, the lack of mechanisms to efficiently provide optimal instructions in real time hinders the achievement of sufficient productivity improvements.

[0510] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0511] In this invention, the server includes information processing means for generating individual characteristic data, generation means for automatically generating personalized advice based on the generated characteristic data, and performance analysis means for analyzing performance data using a data acquisition device. This makes it possible to provide optimal advice in real time based on the characteristics of each automated device.

[0512] "Individual characteristic data" refers to detailed information about the characteristics and state of a specific individual or device, which is used for performance analysis and optimization of that individual or device.

[0513] "Information processing means" refers to a device or system that has the function of collecting, processing, and analyzing diverse data, and that effectively utilizes data for a specific purpose.

[0514] "Personalized advice" refers to information that provides guidance and suggestions optimized for the characteristics and condition of a specific individual or device, and that contributes to improving the capabilities of that individual or device.

[0515] "Generation means" refers to a device or system that has the function of generating information or advice based on input data.

[0516] A "data collection device" is a device used to collect various types of data for a specific purpose, and the collected data is used for further analysis.

[0517] "Performance analysis means" refers to a device or system for analyzing collected data and evaluating the performance and operation of an individual or device.

[0518] "Instruction supply means" refers to a device or system that has the function of appropriately providing individualized instructions or information to a target.

[0519] "Feedback" is information that evaluates the success and areas for improvement of a specific action or result, and is used to inform future actions and plans.

[0520] "Machine learning techniques" are technologies that use past data and feedback to update models and achieve more accurate information provision and predictions.

[0521] This invention is a system for improving the performance of automated equipment in specific environments, particularly in factories. The system aims to maximize the characteristics of each piece of equipment by providing personalized advice.

[0522] The server is equipped with information processing means to generate individual characteristic data for each automated device, and analyzes the collected data to understand its characteristics. In addition, it acquires performance data in real time from each operating device using a data acquisition device, and analyzes the obtained data in detail using performance analysis means.

[0523] The server generates personalized advice using a generative AI model based on the analyzed data. The generated advice is provided to automated equipment in an appropriate format via an instruction supply means. This allows each piece of equipment to achieve efficient operation in real time.

[0524] Furthermore, the system's accuracy is improved using machine learning techniques based on user feedback. This feedback helps in the subsequent advice generation process, continuously optimizing the performance of each device.

[0525] As a concrete example, in the product assembly process, the server analyzes motion data collected by the robot arm and provides real-time instructions to optimize the movement route, ensuring efficient operation. These instructions are individually customized using a generative AI model, thereby increasing work efficiency.

[0526] An example of a prompt message is: "Analyze the performance data of robot A and generate specific advice for efficiency improvement. Consider the current production line status and provide instructions for optimizing its operation in real time."

[0527] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0528] Step 1:

[0529] The server receives basic information and operational status data of automated equipment from a data acquisition device. Based on this input data, an information processing system processes the data to generate individual characteristic data. This characteristic data includes details about the equipment's performance and operating patterns.

[0530] Step 2:

[0531] The server analyzes the generated individual characteristic data using performance analysis tools. This analysis performs data calculations to identify efficient and inefficient operating points of the equipment. The analysis results identify areas requiring optimization and areas where operation can be improved. The output is the analysis progress and its evaluation.

[0532] Step 3:

[0533] The server uses an AI model based on the analysis results to generate personalized advice. The analysis results are used as input, and the AI ​​model uses them to generate optimal operation sequences and adjustment suggestions. The output consists of specific advice, ready to be supplied as instructions to each device.

[0534] Step 4:

[0535] The server transmits the generated advice to the automated equipment in real time using an instruction supply mechanism. The instructions sent out serve as guidelines for the equipment to operate efficiently and are then executed. The output shows the improvement in the equipment's operation.

[0536] Step 5:

[0537] The user monitors the improvement status of the equipment and inputs feedback into the system. This feedback is processed using machine learning methods to evaluate the degree of effectiveness. Based on the input feedback, the system's AI model is updated, contributing to improved processing accuracy in subsequent processes. The output is the improved AI model.

[0538] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0539] This invention realizes a system for recognizing the emotional state of each employee and providing a personalized learning experience based on that state. The embodiments thereof are described in detail below.

[0540] The server first creates an individual profile for each employee. This is generated by retrieving information such as past work performance, skill evaluations, and growth goals from a database and performing data analysis. This profile data is used as foundational information and is used to generate personalized learning content.

[0541] The emotion engine collects user voice input and facial expression data to analyze the user's emotional state in real time. The device sends this data to the emotion engine, which identifies the user's emotions such as joy, interest, and stress. This information helps to understand the user's current emotional state.

[0542] The server generates learning content based on the user's profile and emotional data. This content is optimized for the user's emotional state at any given time and is delivered in an appropriate format and tone. For example, if the user is feeling stressed, relaxing content will be selected.

[0543] When a user has a specific question or concern, they input it through their device and send it to the server. The server then generates a response that incorporates the sentiment analysis results from the emotion engine and provides it to the user immediately. This allows the user to receive advice and information that is appropriate to their mental state at that time.

[0544] Furthermore, user feedback is collected by the server and used in machine learning algorithms to improve system accuracy and content usefulness. User growth is continuously monitored, and periodic reports are generated. These reports reflect the user's emotional state and learning progress, suggesting the next steps to maintain user motivation and achieve goals.

[0545] As a concrete example, if an employee feels anxious about the progress of a new project and the emotion engine detects "stress," the server will prioritize providing learning content that helps reduce stress and suggest practical actions to improve subsequent work efficiency. In this way, the invention provides a personalized learning environment tailored to each individual's emotional state, embodying a means to maximize employee growth and work efficiency.

[0546] The following describes the processing flow.

[0547] Step 1:

[0548] The server retrieves performance data, skill assessments, and past learning history from the company database to create employee profiles. This data is used to set employee growth goals and identify skill gaps.

[0549] Step 2:

[0550] The device collects emotional data in real time through the user's voice input and facial recognition. It sends basic information to the emotion engine to detect the user's emotional state, such as joy or stress.

[0551] Step 3:

[0552] The emotion engine analyzes acquired voice and facial expression data to identify the user's instantaneous emotional state. This information is used to clearly understand what emotions the user is feeling.

[0553] Step 4:

[0554] The server analyzes user profile information and emotional data to generate the most relevant personalized content for that moment. If the user is feeling stressed, relaxing content or encouraging messages will be provided.

[0555] Step 5:

[0556] The device presents the user with personalized content provided by the server. This content is delivered in various formats, including text, audio, and video, and is presented in an appropriate tone that matches the user's emotional state.

[0557] Step 6:

[0558] If a user has a question or doubt, they can input it using their device. The device then sends the entered information to the server.

[0559] Step 7:

[0560] The server generates answers to user questions that reflect the results of the emotion engine. In particular, it sends back answers to the terminal that are adjusted in tone and content according to the user's emotional state.

[0561] Step 8:

[0562] Users provide feedback on the presented content and answers. This feedback is sent to the server via their device.

[0563] Step 9:

[0564] The server analyzes the collected feedback and uses machine learning algorithms to improve response accuracy based on the data accumulated on the system side. This process increases the quality of the content provided next time and the relevance of the response.

[0565] Step 10:

[0566] The server monitors the user's learning progress and emotional state, and generates regular growth reports based on this data. It also provides the user with suggestions for the next learning steps and recommended action plans.

[0567] (Example 2)

[0568] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0569] Traditional learning systems struggle to provide personalized content tailored to each user's emotional state and characteristics, resulting in insufficient support for efficient learning and growth. Furthermore, they are unable to respond promptly and appropriately to user inquiries, leading to decreased user satisfaction.

[0570] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0571] In this invention, the server includes data processing means for generating individual characteristic data, information generation means for automatically generating specialized information based on the generated individual characteristics, and emotion analysis means for analyzing audio and video data to evaluate the emotional state. This enables the provision of personalized content according to the user's emotional state and the generation of quick and appropriate responses.

[0572] "Individual characteristic data" refers to information about individual users, including past work performance, skill evaluations, and growth goals.

[0573] "Data processing means" refers to a method or apparatus for obtaining information from a database, analyzing characteristic data, and generating a profile.

[0574] "Specialized information" refers to content for learning and information provision that is customized based on the user's current emotional state and characteristic data.

[0575] "Information generation means" refers to a method or apparatus that has the function of automatically generating useful information from data and providing appropriate content to the user.

[0576] "Emotion analysis means" refers to a method or apparatus for evaluating a user's emotions based on audio or video data and performing further processing based on that evaluation.

[0577] "Learning algorithms" refer to algorithms used to improve system performance and response accuracy by utilizing user feedback.

[0578] "Information provision means" refers to a method or apparatus for providing generated specialized information to users in various media formats.

[0579] The following configuration is necessary to carry out this invention.

[0580] The server first retrieves information such as each user's work performance, skill evaluation, and growth goals from the database. This is used to create individual characteristic data. The Pandas library in Python is used to manipulate the data and perform profile analysis. The server also stores the generated characteristic data in physical storage and generates specialized information based on it. A generative AI model is used for the automatic generation of specialized information, optimizing the learning content and information provision.

[0581] The device collects the user's voice and facial expressions in real time through its camera and microphone. The collected data is converted to a specific format and sent to a server. A cloud API is used for voice analysis, and image processing techniques using libraries such as OpenCV are applied to the facial expression data.

[0582] The server performs emotion analysis based on audio and video data. It uses a machine learning model as its emotion engine to classify the user's emotional state into categories such as "joy," "interest," and "stress," which are then used as important elements in creating specialized information.

[0583] Users can input specific questions or inquiries through their device. The input information is sent to a server, where a generating AI model analyzes it and provides an appropriate response. The response is tailored based on the user's emotional data and is provided in real time. An example of a prompt is, "Please suggest the best way for the user to relax if they are feeling stressed."

[0584] Furthermore, user feedback is collected on the server and analyzed by a learning algorithm. This feedback is used to improve the system's accuracy and enhance the quality of specialized information. User progress is also tracked on the server, and reports are generated based on the results, offering suggestions for the next steps.

[0585] For example, if a user feels anxious about starting a new project, the system classifies that emotion as "stress" using emotion analysis tools, and generates and provides related information with a relaxing effect. Through this process, the present invention provides a personalized learning environment for each user.

[0586] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0587] Step 1:

[0588] The server retrieves information such as each user's work performance, skill evaluation, and growth goals from the database. Using this information as input, it generates individual characteristic data using a data processing tool. Specifically, it extracts data using SQL queries, formats the data using the Python Pandas library, and creates a profile. The output of this step is characteristic data for each user.

[0589] Step 2:

[0590] The device collects the user's voice and facial expressions in real time through the microphone and camera. This collected audio and video data becomes the input for step 2. After collection, the data is converted to a specific format and sent to the server. Specifically, the audio is converted to text by a cloud API, and the video data is analyzed using OpenCV and processed into a format necessary for sentiment analysis. The output of this step is the audio and video data necessary for sentiment analysis.

[0591] Step 3:

[0592] The server uses the audio and video data acquired in step 2 as input to evaluate the emotional state using an emotion analysis tool. The emotion engine uses a machine learning model to perform a specific analysis that classifies the data into emotions such as "joy," "interest," and "stress." The model is executed using an ML framework such as TensorFlow, and the emotion results are obtained as output. This output, along with the characteristic data, is used in the next step.

[0593] Step 4:

[0594] The server generates specialized information using information generation means based on individual characteristic data and sentiment analysis results. Considering the characteristic data and emotional state used as input, a generation AI model is used to generate optimal learning content for the user. An example of an AI model used in this process is the Transformer architecture. The output of this step is user-optimized learning content.

[0595] Step 5:

[0596] The user inputs a specific question or inquiry through a terminal. This input is sent to a server and analyzed by a response generation system. A generative AI model generates the optimal response while considering sentiment data. Specifically, it uses a natural language processing model to generate a response sentence based on the input prompt. The output of this step is the response message provided to the user.

[0597] Step 6:

[0598] The server collects user feedback as input. This feedback is analyzed using learning algorithms to improve system accuracy and user experience. The feedback is then used to retrain the model, further enhancing system performance. The output of this step is the improved system performance and evaluation.

[0599] (Application Example 2)

[0600] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0601] In today's work environment, providing real-time work guidance and support tailored to the emotional state of individual workers is challenging. Traditional systems fail to provide individualized guidance based on workers' stress levels and concentration, resulting in insufficient improvement of work efficiency and poor stress management. This can lead to decreased worker productivity and workplace dissatisfaction.

[0602] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0603] In this invention, the server includes data processing means for generating individual configuration information, configuration means for automatically generating personalized content based on the generated individual configuration information, response generation means for immediately generating and supplying responses to user inquiries, learning means for utilizing user feedback to improve the accuracy of responses, and emotion analysis means for analyzing the emotional state of workers and adjusting procedures based on those emotions. This enables individualized guidance and real-time support tailored to the emotional state of workers.

[0604] "Individual configuration information" refers to data generated based on past experiences and skill evaluations accumulated for each user.

[0605] "Data processing means" refers to devices or software that have the function of collecting and analyzing various types of information to create configuration information.

[0606] "Personalized content" refers to customized information and guidance that is appropriately selected based on the user's configuration information and current situation.

[0607] "Configuration means" refers to devices or software that have the function of assembling content based on individual configuration information and providing it in a format suitable for the user.

[0608] "User inquiries" refer to questions and requests that users make to the system.

[0609] A "response generation means" refers to a device or software that has the function of creating and providing appropriate answers in real time to user inquiries.

[0610] "Response" refers to the opinions and feedback that users give regarding the system's responses or the content it provides.

[0611] A "learning tool" is a device or software that analyzes responses and automatically improves the accuracy and efficiency of the entire system.

[0612] "Emotional state" refers to information that indicates the user's current psychological or emotional condition.

[0613] "Emotional analysis tools" refer to devices or software that have the function of evaluating and determining an emotional state using the user's voice and facial expression data.

[0614] In this invention, the server is implemented by integrating multiple means having different functions. First, the server uses data processing means to generate individual configuration information based on the user's past experience, skill evaluation, growth goals, etc. This information is stored for each user and then automatically generated as individualized content by the server's configuration means.

[0615] The terminal receives a query from the user and sends it to the server. The server's response generation mechanism creates an appropriate response to this query in real time and provides it to the user through the terminal. In this process, to further improve the accuracy of the response, the user's response is analyzed by the server's learning mechanism, and the system's accuracy is automatically improved.

[0616] Furthermore, the terminal collects voice and facial expression data and transfers it to the server. The server's emotion analysis system uses this data to evaluate the user's emotional state and adjusts the work procedures based on that emotional state, especially if the worker is feeling fatigued or stressed.

[0617] For example, if a worker feels fatigued on the production line, the emotion analysis system detects this, and the server generates a message such as, "We recommend you take a break." Workers who are highly focused will be instructed, "Let's proceed to the next step." This enables real-time guidance and support tailored to the work environment.

[0618] An example of a prompt message would be, "Please suggest the best course of action based on your current emotional state."

[0619] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0620] Step 1:

[0621] The server collects basic user information and generates individual configuration information using data processing tools. In this process, the user's past experience and skill assessment data are used as input, and the individual configuration information is output through analysis. Specifically, it retrieves relevant information from the database and applies data analysis algorithms.

[0622] Step 2:

[0623] The terminal receives user inquiries as input and sends them to the server. The input queries are parsed through a natural language processing engine and passed to the server's response generation system. Specifically, the user provides voice instructions or text input, which the terminal converts into digital data and sends to the server.

[0624] Step 3:

[0625] The server generates an appropriate response in real time to the received query using a response generation mechanism. In this process, the AI ​​model generates a response using the input query, and the result is sent to the terminal as output. Specifically, the AI ​​model analyzes the query content and constructs the optimal answer.

[0626] Step 4:

[0627] The device collects the user's voice and facial expression data and sends it to a server for emotion analysis. Voice data and video feeds are used as input, which are processed in real time and converted into data indicating the user's emotional state. Specifically, voice recognition software and facial recognition algorithms are executed.

[0628] Step 5:

[0629] The server adjusts work procedures based on the results of emotion analysis. Data indicating emotional state is processed as input, and work procedures and support actions are generated as output based on this data. For example, if stress is indicated, suggestions for reducing the workload will be made.

[0630] Step 6:

[0631] The terminal receives user responses and sends them to the server as feedback. The feedback data is used as input, and the response generation and learning mechanisms are adjusted based on it. Specifically, the user's evaluation is recorded in a database, contributing to improved response accuracy in subsequent interactions.

[0632] Step 7:

[0633] The server improves the overall system functionality based on collected feedback data through learning mechanisms. By analyzing the input feedback and adjusting the generative AI model, it provides more accurate and efficient responses and support as output. Specifically, it applies machine learning algorithms and updates the system parameters.

[0634] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0635] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0636] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0637] [Fourth Embodiment]

[0638] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0639] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0640] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0641] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0642] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0643] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0644] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0645] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0646] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0647] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0648] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0649] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0650] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0651] This invention is a system for realizing a personalized mentorship program that efficiently supports the skill development of each employee within a company. A specific embodiment of this system is described below.

[0652] The server first collects profile data for each employee. This includes past work performance, skill assessments, and growth goals based on self-analysis. After collecting the information, the server analyzes the data and evaluates the gap between the employee's current abilities and their goals.

[0653] Subsequently, the server uses a generation AI to create personalized content best suited to each employee. This content includes specific skill improvement methods, industry success stories, and strategies for achieving goals, and is delivered in various media formats. The terminals present this content to employees, creating an environment where they can learn intuitively and interactively.

[0654] When employees have questions or problems, they send their inquiries to the server via their devices. The server then uses AI to generate immediate responses in real time, providing specific solutions and additional resources. These responses are optimized based on employee profiles and up-to-date data analysis.

[0655] Furthermore, as users provide feedback on each piece of content, the server collects and learns from this data, constantly improving the system's accuracy and usefulness. This allows the quality of the content and responses provided to continuously evolve, making employee learning more efficient and effective.

[0656] For example, if a sales employee is exploring sales methods for a new product, the server will present a customized strategy based on the employee's sales style and market trends. The user can then review this strategy and apply it to their actual sales activities via their terminal. Furthermore, by providing feedback on the results of these activities, the system can achieve greater accuracy and effectiveness in future strategy proposals.

[0657] Thus, the invention provides a means to promote individual growth support for each employee, improve the overall skill level of the organization, and maximize the potential of human resources.

[0658] The following describes the processing flow.

[0659] Step 1:

[0660] The server retrieves each employee's performance data, skill assessments, and learning history information from internal databases and external data sources to create employee profiles.

[0661] Step 2:

[0662] The device allows users to input their personal growth goals and areas of interest, and collects data based on this input. This user input is used to understand individual needs.

[0663] Step 3:

[0664] The server analyzes user input and existing profile data to identify gaps in the skill set needed for growth. Based on this analysis, it prepares to generate personalized content.

[0665] Step 4:

[0666] The server uses generative AI to automatically generate appropriate learning content based on the user's profile data and learning goals. This content can be provided in various formats, including text, audio, and video.

[0667] Step 5:

[0668] The device presents the user with generated personalized content, providing an interactive learning experience. The user can then use this content to progress through their learning.

[0669] Step 6:

[0670] When a user has a specific question or concern, they enter it through their device. This information is sent to the server, and processing begins immediately.

[0671] Step 7:

[0672] The server generates responses to user questions in real time, including relevant information and solutions. These generated responses are then presented to the user via the terminal.

[0673] Step 8:

[0674] Users contribute to system improvement by providing feedback on the content and responses they receive.

[0675] Step 9:

[0676] The server uses machine learning algorithms based on feedback to improve the system's response accuracy and the usefulness of its content.

[0677] Step 10:

[0678] The server monitors the user's learning progress and provides the user with regular progress reports. These reports can be used to help plan future learning.

[0679] (Example 1)

[0680] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0681] In today's business environment, efficiently supporting the skill development of individual employees is essential for maintaining the competitiveness of the entire organization. However, traditional, uniform training programs have failed to adequately address individual growth needs, resulting in ineffective skill improvement. Furthermore, providing personalized instruction tailored to each employee requires significant human resources, making it difficult to achieve.

[0682] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0683] In this invention, the server includes information processing means for collecting personal characteristic information, analysis means for analyzing the collected characteristic information and evaluating the difference between abilities and goals, and content generation means for generating personalized content based on the evaluation. This makes it possible to provide each employee with optimized learning content and advice.

[0684] "Individual characteristic information" refers to attribute data related to each individual, such as work performance, skill evaluations, and growth goals.

[0685] "Information processing means" refers to devices and programs used to collect, store, and analyze data.

[0686] "Analysis methods" refer to processes and systems for evaluating collected data and analyzing the difference between the current situation and the target.

[0687] "Content generation methods" refer to technologies that automatically create individually optimized learning materials and strategies based on analysis results.

[0688] "Multimedia delivery methods" refer to technologies for providing generated content to users in various formats, such as text, audio, and video.

[0689] A "response generation means" refers to a process or technology for instantly generating and responding to a user's question.

[0690] "Learning methods" refer to methods and techniques for improving the performance of a system based on collected feedback information.

[0691] A "report generation method" is a function that continuously tracks an individual's growth and generates reports on that progress.

[0692] "Improvement measures" refer to methods and techniques for analyzing collected feedback and making future service provision more effective.

[0693] This invention aims to realize a personalized learning support system that efficiently improves the skills of individual employees within organizations such as companies. The embodiments thereof will be described in detail below.

[0694] The server first collects individual employee profile data. This data is retrieved using APIs from databases such as employee management systems and business management systems. The data includes employee work performance, skill assessments, and growth goals based on self-assessment. This information is properly stored and managed using a database management system (DBMS).

[0695] Next, the server analyzes the collected data. This analysis uses analytical libraries such as Python's pandas and scikit-learn, and R's dplyr, to clean the data and assess the gap between employees' current capabilities and their targets. The results are then visualized using visualization tools to present them in a format easily understandable to managers and supervisors.

[0696] Next, the server uses a generative AI model (for example, a model based on natural language processing) to generate personalized content optimized for each employee based on the analysis results. The prompt will be in the following format: "This user (Employee ID: 12345) is aiming to improve their IT skills. They currently have intermediate-level programming skills and want to improve their data analysis skills. Please suggest efficient ways for this user to improve their data analysis skills."

[0697] The terminals are responsible for providing employees with content generated by the server. Through web or mobile applications running on the terminals, users can receive content in various media formats and progress with their learning. Examples include text, presentation slides, and video streaming.

[0698] If a user has a question during their learning process, they can send it to the server via their device. The server, using generative AI, instantly generates a response, providing specific solutions and supplementary materials. This response is provided in real time, ensuring support for deepening the user's understanding.

[0699] Finally, the feedback provided by users on the learning content is collected by the server and used in future content generation processes. This feedback loop continuously improves the system's adaptability and accuracy, enabling more effective learning support.

[0700] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0701] Step 1:

[0702] The server connects to the company's data repository and HR system to collect each employee's profile data. Input data includes employee work performance, skill assessments, and growth goals. The data is retrieved via API and stored in a database. Specifically, it executes queries to extract the necessary data. The output is a structured version of each employee's profile data.

[0703] Step 2:

[0704] The server analyzes the collected profile data and calculates the difference between each employee's abilities and their growth targets. The input for this step is the profile data obtained in step 1. For data analysis, libraries such as Python's pandas are used to clean the data and perform statistical processing. Specifically, numerical evaluation indicators are compared to clarify skill gaps. The output is the analyzed skill gap data for each employee.

[0705] Step 3:

[0706] The server generates personalized content using a generative AI model based on the analysis results. The input is the skill gap data from step 2. By generating prompt sentences and feeding them into the AI ​​model, optimal learning content is generated. Specifically, prompts are sent to the generative AI, and the response is received as text. The output is learning content tailored to each employee.

[0707] Step 4:

[0708] The terminal provides employees with personalized content received from the server. The input is the learning content generated in step 3. The terminal displays the content through a web application or mobile application, allowing users to progress through their learning. Specifically, it plays text and videos using a browser or dedicated application. The output is the learning content displayed to the user.

[0709] Step 5:

[0710] The user sends questions that arise during the learning process to the server via their device. The input is the question text entered by the user. The server receives the question and generates an answer in real time using a generative AI. Specifically, it inputs the question as a prompt into the generative AI model and returns the obtained answer as text. The output is the answer presented to the user.

[0711] Step 6:

[0712] Users provide feedback on learning content to the server via their devices. Input consists of opinions and ratings entered through feedback forms and rating buttons. The server stores this data in a database and uses it for future content creation. Specifically, it performs tasks such as compiling survey data and analyzing feedback data. Output is the feedback data accumulated for improvement.

[0713] (Application Example 1)

[0714] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0715] Providing individualized support to maximize the performance of automated equipment, particularly robots used in factories, is challenging. Furthermore, the lack of mechanisms to efficiently provide optimal instructions in real time hinders the achievement of sufficient productivity improvements.

[0716] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0717] In this invention, the server includes information processing means for generating individual characteristic data, generation means for automatically generating personalized advice based on the generated characteristic data, and performance analysis means for analyzing performance data using a data acquisition device. This makes it possible to provide optimal advice in real time based on the characteristics of each automated device.

[0718] "Individual characteristic data" refers to detailed information about the characteristics and state of a specific individual or device, which is used for performance analysis and optimization of that individual or device.

[0719] "Information processing means" refers to a device or system that has the function of collecting, processing, and analyzing diverse data, and that effectively utilizes data for a specific purpose.

[0720] "Personalized advice" refers to information that provides guidance and suggestions optimized for the characteristics and condition of a specific individual or device, and that contributes to improving the capabilities of that individual or device.

[0721] "Generation means" refers to a device or system that has the function of generating information or advice based on input data.

[0722] A "data collection device" is a device used to collect various types of data for a specific purpose, and the collected data is used for further analysis.

[0723] "Performance analysis means" refers to a device or system for analyzing collected data and evaluating the performance and operation of an individual or device.

[0724] "Instruction supply means" refers to a device or system that has the function of appropriately providing individualized instructions or information to a target.

[0725] "Feedback" is information that evaluates the success and areas for improvement of a specific action or result, and is used to inform future actions and plans.

[0726] "Machine learning techniques" are technologies that use past data and feedback to update models and achieve more accurate information provision and predictions.

[0727] This invention is a system for improving the performance of automated equipment in specific environments, particularly in factories. The system aims to maximize the characteristics of each piece of equipment by providing personalized advice.

[0728] The server is equipped with information processing means to generate individual characteristic data for each automated device, and analyzes the collected data to understand its characteristics. In addition, it acquires performance data in real time from each operating device using a data acquisition device, and analyzes the obtained data in detail using performance analysis means.

[0729] The server generates personalized advice using a generative AI model based on the analyzed data. The generated advice is provided to automated equipment in an appropriate format via an instruction supply means. This allows each piece of equipment to achieve efficient operation in real time.

[0730] Furthermore, the system's accuracy is improved using machine learning techniques based on user feedback. This feedback helps in the subsequent advice generation process, continuously optimizing the performance of each device.

[0731] As a concrete example, in the product assembly process, the server analyzes motion data collected by the robot arm and provides real-time instructions to optimize the movement route, ensuring efficient operation. These instructions are individually customized using a generative AI model, thereby increasing work efficiency.

[0732] An example of a prompt message is: "Analyze the performance data of robot A and generate specific advice for efficiency improvement. Consider the current production line status and provide instructions for optimizing its operation in real time."

[0733] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0734] Step 1:

[0735] The server receives basic information and operational status data of automated equipment from a data acquisition device. Based on this input data, an information processing system processes the data to generate individual characteristic data. This characteristic data includes details about the equipment's performance and operating patterns.

[0736] Step 2:

[0737] The server analyzes the generated individual characteristic data using performance analysis tools. This analysis performs data calculations to identify efficient and inefficient operating points of the equipment. The analysis results identify areas requiring optimization and areas where operation can be improved. The output is the analysis progress and its evaluation.

[0738] Step 3:

[0739] The server uses an AI model based on the analysis results to generate personalized advice. The analysis results are used as input, and the AI ​​model uses them to generate optimal operation sequences and adjustment suggestions. The output consists of specific advice, ready to be supplied as instructions to each device.

[0740] Step 4:

[0741] The server transmits the generated advice to the automated equipment in real time using an instruction supply mechanism. The instructions sent out serve as guidelines for the equipment to operate efficiently and are then executed. The output shows the improvement in the equipment's operation.

[0742] Step 5:

[0743] The user monitors the improvement status of the equipment and inputs feedback into the system. This feedback is processed using machine learning methods to evaluate the degree of effectiveness. Based on the input feedback, the system's AI model is updated, contributing to improved processing accuracy in subsequent processes. The output is the improved AI model.

[0744] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0745] This invention realizes a system for recognizing the emotional state of each employee and providing a personalized learning experience based on that state. The embodiments thereof are described in detail below.

[0746] The server first creates an individual profile for each employee. This is generated by retrieving information such as past work performance, skill evaluations, and growth goals from a database and performing data analysis. This profile data is used as foundational information and is used to generate personalized learning content.

[0747] The emotion engine collects user voice input and facial expression data to analyze the user's emotional state in real time. The device sends this data to the emotion engine, which identifies the user's emotions such as joy, interest, and stress. This information helps to understand the user's current emotional state.

[0748] The server generates learning content based on the user's profile and emotional data. This content is optimized for the user's emotional state at any given time and is delivered in an appropriate format and tone. For example, if the user is feeling stressed, relaxing content will be selected.

[0749] When a user has a specific question or concern, they input it through their device and send it to the server. The server then generates a response that incorporates the sentiment analysis results from the emotion engine and provides it to the user immediately. This allows the user to receive advice and information that is appropriate to their mental state at that time.

[0750] Furthermore, user feedback is collected by the server and used in machine learning algorithms to improve system accuracy and content usefulness. User growth is continuously monitored, and periodic reports are generated. These reports reflect the user's emotional state and learning progress, suggesting the next steps to maintain user motivation and achieve goals.

[0751] As a concrete example, if an employee feels anxious about the progress of a new project and the emotion engine detects "stress," the server will prioritize providing learning content that helps reduce stress and suggest practical actions to improve subsequent work efficiency. In this way, the invention provides a personalized learning environment tailored to each individual's emotional state, embodying a means to maximize employee growth and work efficiency.

[0752] The following describes the processing flow.

[0753] Step 1:

[0754] The server retrieves performance data, skill assessments, and past learning history from the company database to create employee profiles. This data is used to set employee growth goals and identify skill gaps.

[0755] Step 2:

[0756] The device collects emotional data in real time through the user's voice input and facial recognition. It sends basic information to the emotion engine to detect the user's emotional state, such as joy or stress.

[0757] Step 3:

[0758] The emotion engine analyzes acquired voice and facial expression data to identify the user's instantaneous emotional state. This information is used to clearly understand what emotions the user is feeling.

[0759] Step 4:

[0760] The server analyzes user profile information and emotional data to generate the most relevant personalized content for that moment. If the user is feeling stressed, relaxing content or encouraging messages will be provided.

[0761] Step 5:

[0762] The device presents the user with personalized content provided by the server. This content is delivered in various formats, including text, audio, and video, and is presented in an appropriate tone that matches the user's emotional state.

[0763] Step 6:

[0764] If a user has a question or doubt, they can input it using their device. The device then sends the entered information to the server.

[0765] Step 7:

[0766] The server generates answers to user questions that reflect the results of the emotion engine. In particular, it sends back answers to the terminal that are adjusted in tone and content according to the user's emotional state.

[0767] Step 8:

[0768] Users provide feedback on the presented content and answers. This feedback is sent to the server via their device.

[0769] Step 9:

[0770] The server analyzes the collected feedback and uses machine learning algorithms to improve response accuracy based on the data accumulated on the system side. This process increases the quality of the content provided next time and the relevance of the response.

[0771] Step 10:

[0772] The server monitors the user's learning progress and emotional state, and generates regular growth reports based on this data. It also provides the user with suggestions for the next learning steps and recommended action plans.

[0773] (Example 2)

[0774] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0775] Traditional learning systems struggle to provide personalized content tailored to each user's emotional state and characteristics, resulting in insufficient support for efficient learning and growth. Furthermore, they are unable to respond promptly and appropriately to user inquiries, leading to decreased user satisfaction.

[0776] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0777] In this invention, the server includes data processing means for generating individual characteristic data, information generation means for automatically generating specialized information based on the generated individual characteristics, and emotion analysis means for analyzing audio and video data to evaluate the emotional state. This enables the provision of personalized content according to the user's emotional state and the generation of quick and appropriate responses.

[0778] "Individual characteristic data" refers to information about individual users, including past work performance, skill evaluations, and growth goals.

[0779] "Data processing means" refers to a method or apparatus for obtaining information from a database, analyzing characteristic data, and generating a profile.

[0780] "Specialized information" refers to content for learning and information provision that is customized based on the user's current emotional state and characteristic data.

[0781] "Information generation means" refers to a method or apparatus that has the function of automatically generating useful information from data and providing appropriate content to the user.

[0782] "Emotion analysis means" refers to a method or apparatus for evaluating a user's emotions based on audio or video data and performing further processing based on that evaluation.

[0783] "Learning algorithms" refer to algorithms used to improve system performance and response accuracy by utilizing user feedback.

[0784] "Information provision means" refers to a method or apparatus for providing generated specialized information to users in various media formats.

[0785] The following configuration is necessary to carry out this invention.

[0786] The server first retrieves information such as each user's work performance, skill evaluation, and growth goals from the database. This is used to create individual characteristic data. The Pandas library in Python is used to manipulate the data and perform profile analysis. The server also stores the generated characteristic data in physical storage and generates specialized information based on it. A generative AI model is used for the automatic generation of specialized information, optimizing the learning content and information provision.

[0787] The device collects the user's voice and facial expressions in real time through its camera and microphone. The collected data is converted to a specific format and sent to a server. A cloud API is used for voice analysis, and image processing techniques using libraries such as OpenCV are applied to the facial expression data.

[0788] The server performs emotion analysis based on audio and video data. It uses a machine learning model as its emotion engine to classify the user's emotional state into categories such as "joy," "interest," and "stress," which are then used as important elements in creating specialized information.

[0789] Users can input specific questions or inquiries through their device. The input information is sent to a server, where a generating AI model analyzes it and provides an appropriate response. The response is tailored based on the user's emotional data and is provided in real time. An example of a prompt is, "Please suggest the best way for the user to relax if they are feeling stressed."

[0790] Furthermore, user feedback is collected on the server and analyzed by a learning algorithm. This feedback is used to improve the system's accuracy and enhance the quality of specialized information. User progress is also tracked on the server, and reports are generated based on the results, offering suggestions for the next steps.

[0791] For example, if a user feels anxious about starting a new project, the system classifies that emotion as "stress" using emotion analysis tools, and generates and provides related information with a relaxing effect. Through this process, the present invention provides a personalized learning environment for each user.

[0792] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0793] Step 1:

[0794] The server retrieves information such as each user's work performance, skill evaluation, and growth goals from the database. Using this information as input, it generates individual characteristic data using a data processing tool. Specifically, it extracts data using SQL queries, formats the data using the Python Pandas library, and creates a profile. The output of this step is characteristic data for each user.

[0795] Step 2:

[0796] The device collects the user's voice and facial expressions in real time through the microphone and camera. This collected audio and video data becomes the input for step 2. After collection, the data is converted to a specific format and sent to the server. Specifically, the audio is converted to text by a cloud API, and the video data is analyzed using OpenCV and processed into a format necessary for sentiment analysis. The output of this step is the audio and video data necessary for sentiment analysis.

[0797] Step 3:

[0798] The server uses the audio and video data acquired in step 2 as input to evaluate the emotional state using an emotion analysis tool. The emotion engine uses a machine learning model to perform a specific analysis that classifies the data into emotions such as "joy," "interest," and "stress." The model is executed using an ML framework such as TensorFlow, and the emotion results are obtained as output. This output, along with the characteristic data, is used in the next step.

[0799] Step 4:

[0800] The server generates specialized information using information generation means based on individual characteristic data and sentiment analysis results. Considering the characteristic data and emotional state used as input, a generation AI model is used to generate optimal learning content for the user. An example of an AI model used in this process is the Transformer architecture. The output of this step is user-optimized learning content.

[0801] Step 5:

[0802] The user inputs a specific question or inquiry through a terminal. This input is sent to a server and analyzed by a response generation system. A generative AI model generates the optimal response while considering sentiment data. Specifically, it uses a natural language processing model to generate a response sentence based on the input prompt. The output of this step is the response message provided to the user.

[0803] Step 6:

[0804] The server collects user feedback as input. This feedback is analyzed using learning algorithms to improve system accuracy and user experience. The feedback is then used to retrain the model, further enhancing system performance. The output of this step is the improved system performance and evaluation.

[0805] (Application Example 2)

[0806] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0807] In today's work environment, providing real-time work guidance and support tailored to the emotional state of individual workers is challenging. Traditional systems fail to provide individualized guidance based on workers' stress levels and concentration, resulting in insufficient improvement of work efficiency and poor stress management. This can lead to decreased worker productivity and workplace dissatisfaction.

[0808] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0809] In this invention, the server includes data processing means for generating individual configuration information, configuration means for automatically generating personalized content based on the generated individual configuration information, response generation means for immediately generating and supplying responses to user inquiries, learning means for utilizing user feedback to improve the accuracy of responses, and emotion analysis means for analyzing the emotional state of workers and adjusting procedures based on those emotions. This enables individualized guidance and real-time support tailored to the emotional state of workers.

[0810] "Individual configuration information" refers to data generated based on past experiences and skill evaluations accumulated for each user.

[0811] "Data processing means" refers to devices or software that have the function of collecting and analyzing various types of information to create configuration information.

[0812] "Personalized content" refers to customized information and guidance that is appropriately selected based on the user's configuration information and current situation.

[0813] "Configuration means" refers to devices or software that have the function of assembling content based on individual configuration information and providing it in a format suitable for the user.

[0814] "User inquiries" refer to questions and requests that users make to the system.

[0815] A "response generation means" refers to a device or software that has the function of creating and providing appropriate answers in real time to user inquiries.

[0816] "Response" refers to the opinions and feedback that users give regarding the system's responses or the content it provides.

[0817] A "learning tool" is a device or software that analyzes responses and automatically improves the accuracy and efficiency of the entire system.

[0818] "Emotional state" refers to information that indicates the user's current psychological or emotional condition.

[0819] "Emotional analysis tools" refer to devices or software that have the function of evaluating and determining an emotional state using the user's voice and facial expression data.

[0820] In this invention, the server is implemented by integrating multiple means having different functions. First, the server uses data processing means to generate individual configuration information based on the user's past experience, skill evaluation, growth goals, etc. This information is stored for each user and then automatically generated as individualized content by the server's configuration means.

[0821] The terminal receives a query from the user and sends it to the server. The server's response generation mechanism creates an appropriate response to this query in real time and provides it to the user through the terminal. In this process, to further improve the accuracy of the response, the user's response is analyzed by the server's learning mechanism, and the system's accuracy is automatically improved.

[0822] Furthermore, the terminal collects voice and facial expression data and transfers it to the server. The server's emotion analysis system uses this data to evaluate the user's emotional state and adjusts the work procedures based on that emotional state, especially if the worker is feeling fatigued or stressed.

[0823] For example, if a worker feels fatigued on the production line, the emotion analysis system detects this, and the server generates a message such as, "We recommend you take a break." Workers who are highly focused will be instructed, "Let's proceed to the next step." This enables real-time guidance and support tailored to the work environment.

[0824] An example of a prompt message would be, "Please suggest the best course of action based on your current emotional state."

[0825] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0826] Step 1:

[0827] The server collects basic user information and generates individual configuration information using data processing tools. In this process, the user's past experience and skill assessment data are used as input, and the individual configuration information is output through analysis. Specifically, it retrieves relevant information from the database and applies data analysis algorithms.

[0828] Step 2:

[0829] The terminal receives user inquiries as input and sends them to the server. The input queries are parsed through a natural language processing engine and passed to the server's response generation system. Specifically, the user provides voice instructions or text input, which the terminal converts into digital data and sends to the server.

[0830] Step 3:

[0831] The server generates an appropriate response in real time to the received query using a response generation mechanism. In this process, the AI ​​model generates a response using the input query, and the result is sent to the terminal as output. Specifically, the AI ​​model analyzes the query content and constructs the optimal answer.

[0832] Step 4:

[0833] The device collects the user's voice and facial expression data and sends it to a server for emotion analysis. Voice data and video feeds are used as input, which are processed in real time and converted into data indicating the user's emotional state. Specifically, voice recognition software and facial recognition algorithms are executed.

[0834] Step 5:

[0835] The server adjusts work procedures based on the results of emotion analysis. Data indicating emotional state is processed as input, and work procedures and support actions are generated as output based on this data. For example, if stress is indicated, suggestions for reducing the workload will be made.

[0836] Step 6:

[0837] The terminal receives user responses and sends them to the server as feedback. The feedback data is used as input, and the response generation and learning mechanisms are adjusted based on it. Specifically, the user's evaluation is recorded in a database, contributing to improved response accuracy in subsequent interactions.

[0838] Step 7:

[0839] The server improves the overall system functionality based on collected feedback data through learning mechanisms. By analyzing the input feedback and adjusting the generative AI model, it provides more accurate and efficient responses and support as output. Specifically, it applies machine learning algorithms and updates the system parameters.

[0840] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0841] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0842] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0843] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0844] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. In the upper and lower directions of the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. Also, the upper side of the concentric circles is where "pleasant" emotions are located, and the lower side is where "unpleasant" emotions are located. In this way, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0845] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0846] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0847] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0848] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0849] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0850] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0851] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0852] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0853] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0854] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0855] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0856] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0857] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0858] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0859] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0860] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0861] The following is further disclosed regarding the embodiments described above.

[0862] (Claim 1)

[0863] Information processing means for generating individual profile data,

[0864] A generation means for automatically generating personalized content based on the generated individual profile,

[0865] A response generation means that generates and provides responses in real time to user questions,

[0866] A machine learning method that improves the system's response accuracy by receiving feedback,

[0867] A system that includes this.

[0868] (Claim 2)

[0869] The system according to claim 1, comprising a report generation means for monitoring the user's growth status and generating reports corresponding to that growth.

[0870] (Claim 3)

[0871] The system according to claim 1, comprising multimedia delivery means that enables the generated personalized content to be provided in a variety of media formats.

[0872] "Example 1"

[0873] (Claim 1)

[0874] Information processing means for collecting personal characteristic information,

[0875] An analytical means for analyzing collected characteristic information and evaluating the difference between ability and goal,

[0876] A content generation means that generates personalized content based on evaluation,

[0877] A multimedia delivery method that provides generated personalized content in various media formats,

[0878] A response generation means that generates and provides responses in real time to user questions,

[0879] A learning method that improves the system's response accuracy by receiving feedback,

[0880] A system that includes this.

[0881] (Claim 2)

[0882] The system according to claim 1, comprising a report generation means for monitoring the user's growth status and creating a report corresponding to that growth.

[0883] (Claim 3)

[0884] The system according to claim 1, further comprising means for accumulating collected feedback data and reflecting it in future content generation.

[0885] "Application Example 1"

[0886] (Claim 1)

[0887] Information processing means for generating individual characteristic data,

[0888] A generation means that automatically generates personalized advice based on generated characteristic data,

[0889] A performance analysis means that analyzes performance data using a data collection device,

[0890] An instruction supply means that provides real-time optimized instructions to automated equipment,

[0891] A machine learning method that improves the system's optimization accuracy by receiving feedback,

[0892] A system that includes this.

[0893] (Claim 2)

[0894] The system according to claim 1, comprising a report generation means for evaluating the effectiveness of individualized guidance and generating an improvement report based on that evaluation.

[0895] (Claim 3)

[0896] The system according to claim 1, comprising a multilayer media supply means capable of providing the generated personalized advice in a variety of output formats.

[0897] "Example 2 of combining an emotion engine"

[0898] (Claim 1)

[0899] A data processing means for generating individual characteristic data,

[0900] Information generation means that automatically generates specialized information based on the generated individual characteristics,

[0901] An emotion analysis means for analyzing audio and video data to evaluate emotional states,

[0902] A response generation means that generates and provides responses in real time to user questions,

[0903] A learning algorithm that improves the system's response accuracy by receiving feedback,

[0904] A system that includes this.

[0905] (Claim 2)

[0906] The system according to claim 1, comprising data generation means for monitoring the user's progress and generating analysis results according to the progress.

[0907] (Claim 3)

[0908] The system according to claim 1, comprising information provision means that enables the generated specialized information to be provided in various media formats.

[0909] "Application example 2 when combining with an emotional engine"

[0910] (Claim 1)

[0911] A data processing means for generating individual configuration information,

[0912] A configuration means that automatically generates personalized content based on the generated individual configuration information,

[0913] A response generation means that immediately generates and supplies a response to an inquiry from a user,

[0914] A learning method that utilizes user feedback to improve the accuracy of responses,

[0915] An emotion analysis tool that analyzes the emotional state of workers and adjusts procedures based on those emotions,

[0916] A system that includes this.

[0917] (Claim 2)

[0918] The system according to claim 1, comprising a report generation means for observing the user's growth progress and generating reports according to that progress.

[0919] (Claim 3)

[0920] The system according to claim 1, comprising media supply means capable of supplying the generated individualized content in different media formats. [Explanation of Symbols]

[0921] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. Information processing means for generating individual profile data, A generation means for automatically generating personalized content based on the generated individual profile, A response generation means that generates and provides responses in real time to user questions, A machine learning method that improves the system's response accuracy by receiving feedback, A system that includes this.

2. The system according to claim 1, comprising a report generation means for monitoring the user's growth status and generating reports corresponding to that growth.

3. The system according to claim 1, comprising multimedia delivery means that enables the generated personalized content to be provided in a variety of media formats.

Images