system

Abstract

We provide the system. [Solution] A means of inputting information about the crew's skills, strengths, and weaknesses, A means of dividing the crew into clusters based on the input information, A means for automatically generating a training plan for each crew member based on the aforementioned cluster, The means for displaying the aforementioned training plan, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor and includes steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In conventional crew training, it has been difficult to formulate an effective training plan based on the skills and individual characteristics of each member. In addition, creating a training plan requires a great deal of labor and time, and especially when managing a large number of crew members, the work becomes complicated. Furthermore, in creating a training plan based on artificial judgment, the consistency and effectiveness of the plan are often not guaranteed. There is a need for a system that solves these problems and realizes efficient and effective crew training.

Means for Solving the Problems

[0005] The present invention provides a crew training system that includes means for inputting information on crew members' skills, strengths, and weaknesses; means for dividing crew members into clusters based on the input information; means for automatically generating training plans for each crew member based on the clusters; and means for displaying the training plans. This system enables the rapid and accurate creation of effective training plans that take into account the characteristics of each crew member. Specifically, by using means for vectorizing and clustering crew members' skills, crew members with similar skill sets are grouped together, and training plans optimized for each group are provided. Furthermore, by including means for outputting the training plans in JSON format, the system facilitates integration with other systems and applications.

[0006] A "crew" is a group of members involved in a specific project or task.

[0007] "Skills" refer to the knowledge, techniques, and abilities necessary to effectively perform a particular task or job.

[0008] "Strengths" refer to the unique strengths and abilities that each crew member possesses.

[0009] "Weaknesses" refer to characteristics that individual crew members possess that need improvement or are weak points.

[0010] A "cluster" is a group of similar crews that are organized based on specific criteria.

[0011] A "training plan" is a plan that includes specific goals and action plans for improving the crew's skills.

[0012] "Vectorization" is the process of converting text data and other non-numeric data into numerical vectors.

[0013] Clustering is a technique for dividing data into several clusters (groups) based on similarity.

[0014] JSON format is a data exchange format that structures data and represents it in text format. [Brief explanation of the drawing]

[0015] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2, which combines an emotion engine.

Embodiments for Carrying Out the Invention

[0016] An example of an embodiment of the system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a processor with a reference numeral (hereinafter simply referred to as "processor") may be one arithmetic unit or a combination of a plurality of arithmetic units. Also, the processor may be one type of arithmetic unit or a combination of a plurality of types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0019] In the following embodiments, a RAM (Random Access Memory) with a reference numeral is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0020] In the following embodiments, a storage with a reference numeral is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, and the like.

[0021] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0023] [First Embodiment]

[0024] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0025] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0028] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0031] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0035] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0036] The present invention provides a system for automatically generating crew development plans, comprising means for inputting crew skills, strengths, and weaknesses, means for analysis, means for generating clusters, means for creating development plans, and means for displaying or outputting those plans. The system is implemented as follows:

[0037] 1. Means of entering crew information

[0038] Subject: User

[0039] Users input information such as crew IDs, strengths, weaknesses, and skills into the system. This is done through a dedicated form or user interface. This input information is sent to and stored on the server.

[0040] 2. Data Analysis Methods

[0041] Subject: Server

[0042] The server analyzes the data based on the entered crew information. In particular, it vectorizes the crew's skills and uses this to perform clustering. This process classifies crew members into groups with similar skill sets.

[0043] For example, if Crew A's skills are "Python, Management" and Crew B's skills are "Java(registered trademark), Hadoop," these skills are converted into numerical vectors, and clustered using feature vectors calculated with the TF-IDF (Term Frequency-Inverse Document Frequency) method.

[0044] 3. Means for generating clusters

[0045] Subject: Server

[0046] The server performs clustering using the previously analyzed data. It uses a clustering algorithm (e.g., K-Means) to group crew members with similar skills.

[0047] 4. Means of creating a training plan

[0048] Subject: Server

[0049] The server automatically generates training plans based on the clustering results. Since each cluster corresponds to a specific training goal, individual training plans are generated based on the cluster to which each crew member belongs.

[0050] Specifically, crews belonging to Cluster 0 are set with goals such as "improving Skill A" and "practicing project management," while crews belonging to Cluster 1 are set with goals such as "technical training in Skill B" and "promoting collaborative work."

[0051] 5. Means of displaying the training plan

[0052] Subject: terminal

[0053] The terminal displays the training plan generated by the server. This allows users to check and implement the specific training plan for each crew member. The training plan is output in JSON format, making it easy to integrate with other systems and tools.

[0054] For example, the following training plan will be generated:

[0055] Crew ID: C001

[0056] Short-term goals: Improve Skill A, practice project management.

[0057] Long-term goal: Leadership training

[0058] Duration: 6 months

[0059] Users can verify this and support the development of each crew member.

[0060] As described above, the system of the present invention can automatically create an efficient and effective crew training plan, thereby improving crew skills and operational efficiency.

[0061] The following describes the processing flow.

[0062] Step 1:

[0063] Subject: User

[0064] Users input crew information. Specifically, they enter crew IDs, strengths, weaknesses, skills, etc., through a dedicated form or user interface, and send this data to the server.

[0065] Step 2:

[0066] Subject: Server

[0067] The server stores the received crew information in a database. This includes fields for crew ID, strengths, weaknesses, and skills.

[0068] Step 3:

[0069] Subject: Server

[0070] The server vectorizes the saved crew skill information. This vectorization uses TfidfVectorizer to convert each crew member's skill into a numerical vector.

[0071] Step 4:

[0072] Subject: Server

[0073] The server performs clustering based on vectorized skill information. A clustering algorithm (e.g., K-Means) is used to group crews with similar skill sets.

[0074] Step 5:

[0075] Subject: Server

[0076] Based on the clustering results, the server determines the cluster to which each crew member belongs and stores this information in the database. Each crew member is assigned information about which cluster they belong to.

[0077] Step 6:

[0078] Subject: Server

[0079] The server automatically generates a growth plan based on each cluster. Specifically, it applies pre-configured growth goals and action plans for each cluster.

[0080] Step 7:

[0081] Subject: Server

[0082] The server converts the generated training plan into JSON format and saves it. This JSON data will be provided to the user in a later step.

[0083] Step 8:

[0084] Subject: terminal

[0085] The terminal sends a training plan request to the server. This is the procedure for users to check the training plans for each crew member.

[0086] Step 9:

[0087] Subject: Server

[0088] The server receives the request and sends the saved breeding plan in JSON format to the terminal.

[0089] Step 10:

[0090] Subject: terminal

[0091] The terminal displays the training plan received from the server on the screen. The user reviews this and takes specific actions to support the training of each crew member.

[0092] (Example 1)

[0093] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0094] The creation of conventional crew training plans has been difficult to implement efficiently and effectively due to the significant time and effort required to consider individual skills and characteristics. Furthermore, manual input and analysis can lead to errors and biases, making it challenging to design appropriate training plans for each crew member. This invention solves these problems by providing a system that efficiently inputs and analyzes crew information and automatically generates training plans.

[0095] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0096] In this invention, the server includes means for inputting information on the crew's skills, strengths, and weaknesses; means for vectorizing the crew data and generating feature vectors using the TF-IDF method; and means for dividing the crew into clusters using the feature vectors. This makes it possible to efficiently and automatically generate training plans for each crew member and output and display the training plans in JSON format.

[0097] "Crew skills" refer to the abilities and knowledge necessary to perform specific tasks or roles.

[0098] "Strengths" refer to the strengths and outstanding characteristics that the crew possesses.

[0099] "Weaknesses" refer to characteristics or shortcomings that the crew possesses that could be improved.

[0100] "Input method" refers to an interface for recording and transmitting information about crew members' skills, strengths, and weaknesses to the system.

[0101] "Vectorization" refers to the process of converting crew skill information into mathematical vectors and treating it as numerical data.

[0102] The "TF-IDF method" is a technique for evaluating the importance of words in text data, and is an abbreviation for Term Frequency-Inverse Document Frequency.

[0103] A "feature vector" is a numerical vector generated based on the crew's skills and is used as input data for clustering.

[0104] "Clustering" refers to the process of dividing data into groups based on similarity.

[0105] The "K-Means algorithm" refers to a clustering method for dividing data into K clusters.

[0106] A "development plan" refers to a document that includes goals and action plans for improving the crew's abilities and skills.

[0107] "JSON format" is an abbreviation for JavaScript (registered trademark) Object Notation, and refers to a lightweight data format for structuring, storing, and transferring data.

[0108] A "database" refers to a system for efficiently storing, managing, and retrieving structured data.

[0109] "Means of display" refers to an interface for receiving crew training plans from the server and displaying them in a format viewable by the user.

[0110] The system of the present invention is for automatically generating crew training plans and includes the following means.

[0111] 1. Means of entering crew information

[0112] Subject: User

[0113] Users enter information such as crew IDs, strengths, weaknesses, and skills through a dedicated form or user interface. This involves users using a browser to input information into the UI and clicking a submit button. This input information is sent to the server and stored in a database.

[0114] 2. A method for vectorizing data and generating feature vectors using the TF-IDF method.

[0115] Subject: Server

[0116] The server analyzes crew data retrieved from the database. Specifically, it converts crew skill information into numerical vectors and generates feature vectors using the TF-IDF (Term Frequency-Inverse Document Frequency) method. These feature vectors are then used as input data for clustering.

[0117] 3. Methods for dividing the crew into clusters

[0118] Subject: Server

[0119] The server applies a clustering algorithm (e.g., K-Means) based on the generated feature vectors to group crews with similar skill sets. This automatically classifies crews into specific clusters.

[0120] 4. Means for automatically generating training plans

[0121] Subject: Server

[0122] The server automatically generates training plans for each crew member based on the clustering results. Each cluster has specific training goals, and individual training plans are generated according to the cluster to which each crew member belongs. Specifically, each crew member is given a detailed training plan (short-term goals, long-term goals, and duration).

[0123] 5. A means of outputting and displaying the training plan in JSON format.

[0124] Subject: terminal

[0125] The terminal displays the training plan generated by the server to the user. The training plan is sent from the server to the terminal in JSON format and displayed on the web interface. The user can check the training plan using a browser.

[0126] Hardware and software to be used

[0127] Server: Database system (e.g., MySQL®), machine learning library (e.g., Scikit-learn)

[0128] Device: Web browser

[0129] Specific example

[0130] The user opens a web browser, accesses the specified URL, enters "Crew ID: C001, Skills: Python, Management, Strengths: Team Leadership, Weaknesses: Lack of Presentation Skills" into the input form, and clicks the "Submit" button. This information is sent to the server, vectorized using the TF-IDF method, and clustered using the K-Means algorithm. Finally, an automatically generated training plan is output in JSON format and displayed on the user's terminal.

[0131] Example of a prompt

[0132] Please enter the crew information: [Crew ID, Strengths, Weaknesses, Skills]

[0133] This will generate a training plan. Please click "Generate" once you have finished entering the information.

[0134] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0135] Step 1:

[0136] Subject: User

[0137] Users input and submit crew information (crew ID, strengths, weaknesses, skills) into the system through a dedicated form or user interface.

[0138] Input: Crew ID, Strengths, Weaknesses, Skills

[0139] Output: Crew information in JSON format is sent to the server.

[0140] Specific actions: The user opens a web browser and accesses the specified URL. They enter "Crew ID: C001, Skills: Python, Management, Strengths: Team leadership, Weaknesses: Lack of presentation skills" into the displayed input form and click the "Submit" button.

[0141] Step 2:

[0142] Subject: Server

[0143] The server receives crew information in JSON format sent by the user and stores it in the database.

[0144] Input: Crew information submitted by the user (in JSON format)

[0145] Output: Crew information stored in the database

[0146] Specific operation: The server receives an HTTP request sent by the user and parses the JSON data contained in its body. Then, it inserts the parsed data into a database (e.g., MySQL).

[0147] Step 3:

[0148] Subject: Server

[0149] The server retrieves crew information from the database and vectorizes the skill information. Feature vectors are generated using the TF-IDF method.

[0150] Input: Crew skill information retrieved from the database

[0151] Output: Vectorized skill information (feature vectors)

[0152] Specific operation: The server executes a script (e.g., a Python program) to retrieve skill information from the database. This skill information is converted into a numerical vector, and TF-IDF calculations are performed.

[0153] Step 4:

[0154] Subject: Server

[0155] The server applies the K-Means algorithm based on the generated feature vectors to classify the crew into clusters.

[0156] Input: Feature vector

[0157] Output: Cluster ID for each crew

[0158] Specific operation: The server uses a machine learning library (e.g., Scikit-learn) to perform K-Means clustering on the feature vectors. The result of the calculation is the cluster ID for each crew.

[0159] Step 5:

[0160] Subject: Server

[0161] The server automatically generates training plans for each crew member based on the clustering results. Each cluster has set training goals, which are used to create individual plans.

[0162] Input: Cluster ID and growth objectives

[0163] Output: Training plan for each crew member (JSON format)

[0164] Specific operation: Based on the clustering results, the server retrieves training objectives corresponding to each cluster. Next, it generates specific training plans (short-term objectives, long-term objectives, and duration) for each crew member.

[0165] Step 6:

[0166] Subject: terminal

[0167] The terminal displays the training plan generated by the server to the user. The training plan is sent from the server in JSON format and displayed via a web interface.

[0168] Input: Training plan (JSON format) sent from the server

[0169] Output: Displayed training plan

[0170] Specific operation: The user reopens their web browser and accesses the training plan display page. The browser retrieves JSON data from the server and displays it in a user-friendly format (e.g., an HTML table).

[0171] Example: Crew ID: C001, Short-term goal: Improve Skill A, practice project management, Long-term goal: Leadership training, Duration: 6 months

[0172] (Application Example 1)

[0173] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0174] The problem lies in the lack of a system that effectively and efficiently manages the skill sets and training plans of crew members working in the factory, and provides optimal training plans for each crew member in real time. Traditional manual management and planning are time-consuming and labor-intensive, and it was difficult to create optimal training plans tailored to the individual skills of each crew member. In addition, there was a problem in that training plans became mere formalities and did not lead to actual work efficiency improvements or skill development.

[0175] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0176] In this invention, the server includes means for inputting information on the skills, strengths, and weaknesses of crew members, means for dividing the crew into clusters based on the input information, and means for automatically generating a training plan for each crew member based on the clusters. This makes it possible to effectively manage the skill sets of crew members and provide optimal training plans in real time. Furthermore, by providing the training plans in JSON format and supplying them to factory robots, and by providing means for allowing crew members to check their training plans via tablet terminals, it becomes possible to propose crew placement and training plans in real time.

[0177] "Crew" refers to the workers and staff who work in a factory.

[0178] "Skills" is a general term for the specialized knowledge and techniques that a crew member possesses.

[0179] "Strengths" refers to the distinctive strengths and advantages of each crew member.

[0180] "Weaknesses" refer to the crew's shortcomings or areas that need improvement.

[0181] "Means of inputting information" refers to interfaces or devices used to input data on crew members' skills, strengths, and weaknesses.

[0182] "Methods for dividing crews into clusters" refers to algorithms or processes that group similar crews together based on their skills and characteristics.

[0183] "Means for automatically generating training plans" refers to systems or programs that automatically create training goals and methods for crew members in each cluster.

[0184] "Means for displaying training plans" refers to devices or software that visually display automatically generated training plans.

[0185] "Method for outputting in JSON format" refers to the function that exports the training plan in JSON (JavaScript Object Notation) format.

[0186] "Factory robots" refer to robots that assist crew members or perform automated tasks within a factory.

[0187] "A means of proposing crew placement and training plans in real time" refers to a system or algorithm in which factory robots instantly propose the optimal placement and training plan based on the current status and skills of the crew.

[0188] A "tablet device" refers to a portable computer device used to review training plans.

[0189] The system for realizing this application generates, manages, and implements effective crew training plans through an application installed on factory robots. The system consists of multiple hardware and software components.

[0190] Hardware and software to be used:

[0191] Hardware: Factory robots, servers, tablet devices

[0192] Software: Python (for data analysis and clustering), Django (web interface), MySQL (database management)

[0193] System processing procedures and functions:

[0194] 1. Enter and save crew information:

[0195] Users enter information about crew members, including their ID, strengths, weaknesses, and skills, via a tablet device. This information is sent to the server through a dedicated form or user interface and stored in a MySQL database.

[0196] 2. Data Analysis and Clustering:

[0197] The server uses Python to analyze the input crew information. Specifically, it uses the TF-IDF (Term Frequency-Inverse Document Frequency) method to create numerical vectors of crew skills and then performs clustering using the K-Means clustering algorithm.

[0198] 3. Automatic generation of training plans:

[0199] The server automatically generates a growth plan for each cluster based on the clustering results. The growth plan includes short-term and long-term goals and is saved in JSON format.

[0200] 4. Display and implementation of the training plan:

[0201] The factory robot receives the generated training plan in real time and displays it via a tablet device. Users can review the displayed training plan and implement specific training plans for their crew. The robot also handles crew placement and provides real-time training plan suggestions.

[0202] Specific example:

[0203] For example, let's say there are crew members A, B, and C working on a factory assembly line.

[0204] Crew A's skills: "Mechanical Engineering, Basic Programming"

[0205] Crew B's skills: "Basic Electrical Engineering, Applied Programming"

[0206] Crew C's skills: "Mechanical maintenance, Electrical maintenance"

[0207] The system converts these skills into numerical vectors and performs clustering. As a result, for example, Crew A and Crew B are classified into the same cluster, and training plans are set for this cluster: "Improvement of programming skills" and "Basic training in machine maintenance." A different training plan is generated for Crew C.

[0208] Examples of prompt statements for generated AI models:

[0209] Based on the crew skill information provided below, generate an optimal training plan. Crew skills have been numerically vectorized using the TF-IDF method, and K-Means is used for clustering. Propose specific training goals for each cluster.

[0210] Crew information:

[0211] 1. Crew ID: C001, Skills: "Mechanical Engineering, Basic Programming"

[0212] 2. Crew ID: C002, Skills: "Basic Electrical Engineering, Applied Programming"

[0213] 3. Crew ID: C003, Skills: "Machine Maintenance, Electrical Maintenance"

[0214] "

[0215] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0216] Step 1:

[0217] Users input crew information using a tablet device. Specifically, they enter the crew's ID, strengths, weaknesses, and skills into a dedicated input form and send that data to the server. The entered data is stored in a MySQL database on the server.

[0218] Input: Crew ID, strengths, weaknesses, skills

[0219] Output: Crew information stored in the database on the server

[0220] Step 2:

[0221] The server performs data analysis based on the stored crew information. Specifically, it uses Python to numerically vectorize crew skills using the TF-IDF (Term Frequency-Inverse Document Frequency) method. This quantifies each crew member's skills as feature vectors.

[0222] Input: Crew skill information stored in the database

[0223] Output: Numerically vectorized crew skill data

[0224] Step 3:

[0225] The server performs clustering based on vectorized skill data. Using the K-Means clustering algorithm, crews with similar skills are divided into clusters. The clustering results are stored in the database as the cluster ID for each crew member.

[0226] Input: Numerically vectorized crew skill data

[0227] Output: Crew information with assigned cluster IDs

[0228] Step 4:

[0229] The server automatically generates training plans based on the clustering results. Specifically, it assigns predefined training objectives (short-term and long-term goals) to each cluster and creates training plans for each crew member. The generated training plans are stored in the database in JSON format.

[0230] Input: Crew information with a cluster ID assigned.

[0231] Output: Training plan in JSON format

[0232] Step 5:

[0233] Factory robots receive training plans transmitted from a server in real time and display the crew training plans via tablet devices. This allows users to review and implement training plans for each crew member. The robots also suggest crew assignments and training plans.

[0234] Input: Training plan in JSON format

[0235] Output: Training plan displayed on the tablet device

[0236] Through these steps, the system will be able to efficiently manage crew skill information and automatically generate and provide optimal training plans in real time.

[0237] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0238] The present invention provides a system for automatically generating crew training plans, which combines means for inputting crew skills, strengths, and weaknesses, means for analyzing them, means for generating clusters, means for creating training plans, and means for displaying or outputting those plans, with an emotion engine that recognizes user emotions. This system is implemented as follows:

[0239] 1. Means of entering crew information

[0240] Subject: User

[0241] Users input information such as crew IDs, strengths, weaknesses, and skills into the system. This is done through a dedicated form or user interface. This input information is sent to and stored on the server.

[0242] 2. Data Analysis Methods

[0243] Subject: Server

[0244] The server analyzes the data based on the entered crew information. In particular, it vectorizes the crew's skills and uses this to perform clustering. This process classifies crew members into groups with similar skill sets.

[0245] For example, if Crew A's skills are "Python, Management" and Crew B's skills are "Java, Hadoop," these skills are converted into numerical vectors, and clustered using feature vectors calculated with the TF-IDF (Term Frequency-Inverse Document Frequency) method.

[0246] 3. Means for generating clusters

[0247] Subject: Server

[0248] The server performs clustering using the previously analyzed data. It uses a clustering algorithm (e.g., K-Means) to group crew members with similar skills.

[0249] 4. Means of creating a training plan

[0250] Subject: Server

[0251] The server automatically generates training plans based on the clustering results. Each cluster has specific training goals and action plans set, and individual training plans are generated based on the cluster to which each crew member belongs.

[0252] For example, crews belonging to Cluster 0 are set with goals such as "improving Skill A" and "practicing project management," while crews belonging to Cluster 1 are set with goals such as "technical training in Skill B" and "promoting collaborative work."

[0253] 5. Emotion recognition means using an emotion engine

[0254] Subject: terminal

[0255] Using the camera and microphone built into the device, the system analyzes the user's facial expressions and voice tone in real time to recognize the user's emotions. The emotion engine processes this data and generates emotion data.

[0256] 6. Methods for collecting emotional data

[0257] Subject: Server

[0258] The server receives emotional data transmitted from the emotion engine and stores it in a database. This data is used to adapt the crew's training plan.

[0259] 7. Means of adjusting development plans to reflect emotions

[0260] Subject: Server

[0261] The server dynamically adjusts the content and progress of the training plan based on the collected user emotion data. For example, if a user is experiencing stress, the plan is adjusted to reduce stress; conversely, if the user is showing positive emotions, the plan is adjusted to increase the number of challenging tasks.

[0262] 8. Means of displaying the training plan

[0263] Subject: terminal

[0264] The terminal displays the training plan generated and adjusted by the server. This allows users to review and implement the specific training plan for each crew member. The training plan is output in JSON format, making it easy to integrate with other systems and tools.

[0265] For example, the following training plan will be generated:

[0266] Crew ID: C001

[0267] Short-term goals: Improve Skill A, practice project management.

[0268] Long-term goal: Leadership training

[0269] Duration: 6 months

[0270] Users can verify this and support the development of each crew member.

[0271] The system of the present invention can automatically create efficient and effective crew training plans and, by taking into account the user's feelings, can achieve more personalized training support.

[0272] The following describes the processing flow.

[0273] Step 1:

[0274] Subject: User

[0275] Users input crew information. Specifically, they enter information such as crew ID, strengths, weaknesses, and skills through a dedicated form or user interface, and send this data to the server.

[0276] Step 2:

[0277] Subject: Server

[0278] The server stores the received crew information in the database. Each field of crew ID, strengths, weaknesses, and skills is stored in the database.

[0279] Step 3:

[0280] Subject: Server

[0281] The server retrieves the crew's skill information from the database and vectorizes it. TfidfVectorizer is used to convert the skills into a numerical vector.

[0282] Step 4:

[0283] Subject: Server

[0284] Clustering is performed based on the vectorized skills. A clustering algorithm (such as K-Means) is used to group crews with similar skill sets.

[0285] Step 5:

[0286] Subject: Server

[0287] Analyze the clustering results to determine the cluster to which each crew belongs. Add the determined cluster information to the crew's data and save it in the database.

[0288] Step 6:

[0289] Subject: Server

[0290] Automatically generate a training plan based on the clusters. Apply the specific training goals and action plans set for each cluster to generate individual training plans.

[0291] Step 7:

[0292] Subject: Terminal

[0293] The terminal analyzes the user's expression and voice tone in real time to recognize the user's emotion. An emotion engine is used to generate emotion data.

[0294] Step 8:

[0295] Subject: Server

[0296] The server receives the emotion data sent from the terminal and stores it in the database. This emotion data is reflected in the crew's training plan.

[0297] Step 9:

[0298] <​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​The terminal displays the received training plan on its screen. The user reviews the displayed training plan and takes specific actions to support the training of each crew member.

[0309] (Example 2)

[0310] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0311] Traditional crew training systems typically provided uniform training plans without considering the emotional state of each individual crew member. This led to problems such as crew members experiencing stress or, conversely, a lack of stimulation resulting in decreased motivation. Furthermore, the manual generation of training plans was inefficient and time-consuming. Additionally, data format issues arose when integrating the generated training plans with other systems, resulting in low versatility.

[0312] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0313] In this invention, the server includes means for inputting information on the skills, strengths, and weaknesses of crew members; means for dividing the crew into clusters based on the input information; means for automatically generating a training plan for each crew member based on the clusters; means for dynamically adjusting the training plan using an emotion engine that recognizes the user's emotions; and means for displaying the training plan. This makes it possible to provide more personalized training plans that take into account the individual emotional state of each crew member, thereby resolving issues of crew stress and decreased motivation. Furthermore, since the training plan is output in JSON format, it is easy to link with other systems, improving versatility.

[0314] "Crew" refers to individual members who are engaged in a specific task or project.

[0315] "Skills" refer to the knowledge and abilities that a crew member possesses in order to perform a specific job or task.

[0316] "Strengths" refers to the individual strengths and outstanding characteristics of each crew member.

[0317] "Weaknesses" refer to the individual weaknesses or areas for improvement of each crew member.

[0318] "Means of input" refers to methods of entering data into a system through user interfaces or forms.

[0319] A "cluster" refers to a group of crew members who share similar skills or characteristics.

[0320] "Clustering" refers to the process of classifying crews based on their similarity using data analysis techniques.

[0321] A "training plan" refers to specific goals and action plans aimed at improving the skills and characteristics of the crew.

[0322] An "emotion engine" refers to a combination of hardware and software used to recognize and analyze a user's emotional state.

[0323] "Dynamic adjustment methods" refer to methods of adapting the plan based on real-time sentiment data.

[0324] "JSON format" refers to JavaScript Object Notation, a lightweight data format for data exchange.

[0325] "Means of display" refers to methods that allow users to visually confirm their training plan.

[0326] System Overview

[0327] The present invention relates to a system for automatically generating crew training plans. This system includes means for inputting crew skills, strengths, and weaknesses, means for analyzing data, means for generating clusters, means for creating training plans, and means for displaying or outputting those plans, and further incorporates an emotion engine that recognizes user emotions.

[0328] Hardware and software to be used

[0329] Hardware: Server, terminal (with camera and microphone), network connection

[0330] Software: Database management system, clustering algorithms (e.g., K-Means), Python libraries (scikit-learn, OpenCV), sentiment analysis engine, display interface, JSON output function.

[0331] Specific operation of the system

[0332] 1. Enter crew information

[0333] Users input information such as crew IDs, strengths, weaknesses, and skills into the system. This information is collected through dedicated forms and user interfaces. For example, a user might use a web browser to enter information such as "Crew ID: C001," "Strengths: Honesty," "Weaknesses: Communication," and "Skills: Java, Spring framework," and then click the submit button.

[0334] 2. Saving input data

[0335] The server receives information sent by the user and stores it in the database. For example, it can use SQL queries to store data in the database.

[0336] 3. Data Analysis

[0337] The server retrieves crew information from the database and converts the crew's skills into text vectors. Specifically, it vectorizes the data using the TF-IDF method and performs clustering using the K-Means algorithm. For example, the scikit-learn library in Python is used to convert the skill data into numerical vectors and then perform clustering.

[0338] 4. Cluster generation

[0339] The server classifies crew members into groups (clusters) with similar skill sets based on the clustering results. The generated clustering results are stored in a database.

[0340] 5. Automatic generation of training plans

[0341] The server automatically generates a training plan tailored to each cluster based on the clustering results. For example, it might create a plan for cluster 0 that includes "improving Skill A" and "practicing project management." This plan is then stored in the database.

[0342] 6. Emotion recognition by an emotion engine

[0343] When reviewing the training plan, the device's camera and microphone are used to recognize the user's emotions in real time. The emotion engine analyzes the user's facial expressions and voice tone to generate emotion data. For example, OpenCV is used to analyze facial expressions and estimate emotional states.

[0344] 7. Collection of emotional data

[0345] The server receives emotion data from the emotion engine and stores it in a database. For example, if a user is feeling stressed, that emotion data is sent to the server.

[0346] 8. Dynamic adjustment of the training plan

[0347] The server dynamically adjusts the content of the training plan based on the collected emotional data. For example, if stress levels are high, the training plan's workload is reduced, and if motivation is high, challenging goals are added.

[0348] 9. Display of the training plan

[0349] The finalized training plan is sent to the device and displayed for the user to review. Since the training plan is output in JSON format, it can be easily integrated with other systems.

[0350] Example of a prompt

[0351] Please enter the information for Crew C001 (ID, Strengths: Honesty, Weaknesses: Communication, Skills: Java, Spring framework) and generate a training plan.

[0352] This system allows users to easily generate personalized training plans for each crew member, taking emotional data into consideration to support more effective training.

[0353] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0354] Step 1: The user enters the crew information.

[0355] The user accesses a dedicated form on the system. They enter information such as the crew member's ID, strengths, weaknesses, and skills into the form and click the submit button. This input information is sent to the server. Specifically, the user enters "Crew ID: C001", "Strengths: Honesty", "Weaknesses: Communication", and "Skills: Java, Spring framework" into the form using a web browser. Input data: Crew member ID / Strengths / Weaknesses / Skills. Output data: Once the user completes the input and clicks the submit button, the user's input information is sent to the server.

[0356] Step 2: The server saves the input data.

[0357] The server receives information sent by the user and stores it in the database. Based on the received data, the server uses SQL queries to save information to the database. Specifically, the following SQL query is executed: INSERT INTO crew_data (ID, strengths, weaknesses, skills) VALUES ('C001', 'Sincerity', 'Communication', 'Java, Spring framework'). Input data: Crew information sent by the user. Output data: Crew information saved by the server in the database.

[0358] Step 3: The server analyzes the data.

[0359] The server retrieves crew information from the database and converts the crew's skills into text vectors. The server performs vectorization using the TF-IDF method and then clustering. Specifically, it uses the Python scikit-learn library to convert the crew's skills into numerical vectors and performs clustering using the KMeans(n_clusters=3) algorithm. Input data: Crew information stored in the database. Output data: Vectorized skills and clustering results.

[0360] Step 4: The server generates the cluster.

[0361] The server classifies the crew into groups based on the clustering results. The classification results are then saved back to the database. Specifically, assuming crew C001 belongs to cluster 1, the SQL query INSERT INTO cluster_data (crew_id, cluster_id) VALUES ('C001', 1) is executed. Input data: Clustering results. Output data: Cluster information saved in the database.

[0362] Step 5: The server automatically generates a training plan.

[0363] The server automatically generates a training plan suitable for each cluster based on the clustering results. Specifically, tasks such as "Improving Skill A" and "Practicing Project Management" are assigned to Cluster 1. The generated training plans are saved in the database. Input data: Clustering results. Output data: Training plans.

[0364] Step 6: The device recognizes the user's emotions.

[0365] While the training plan is being reviewed, the device's built-in camera and microphone are used to recognize the user's emotions in real time. The device's emotion engine analyzes the user's facial expressions and voice tone to generate emotion data. Specifically, it uses OpenCV to analyze facial expressions and estimate the emotional state. Input data: User's facial expressions and voice captured by the camera and microphone. Output data: Generated emotion data.

[0366] Step 7: The server collects emotion data.

[0367] The server receives emotion data sent from the emotion engine and stores it in the database. Specifically, an SQL query, INSERT INTO emotion_data (user_id, stress_level) VALUES ('U001', 70), is executed based on the emotion data. Input data: Emotion data sent from the terminal. Output data: Emotion data stored in the database.

[0368] Step 8: The server dynamically adjusts the growth plan.

[0369] The server dynamically adjusts the training plan based on the collected emotional data. Specifically, if the stress level is high, the workload of the plan is reduced, and conversely, if motivation is high, challenging goals are added. For example, the following SQL query is executed: UPDATE training_plans SET adjustments = 'Load Reduction' WHERE user_id = 'U001' AND stress_level > 60. Input data: Emotional data stored in the database. Output data: Adjusted training plan.

[0370] Step 9: The device displays the training plan.

[0371] The terminal receives the training plan prepared by the server and displays it on the user interface. For example, for crew member C001, information such as the short-term goal "Improve Skill A", the long-term goal "Leadership Training", and the duration "6 months" is displayed. Input data: Prepared training plan. Output data: Training plan displayed to the user.

[0372] (Application Example 2)

[0373] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0374] In conventional systems, crew training plans are automatically generated based on skills and characteristics, but they fail to take into account specific situations and emotional aspects, resulting in insufficient personalization for each crew member. In particular, in the training of robot operators within factories, plans that ignore individual emotions and stress levels hinder efficient training. Therefore, dynamic adjustments that respond to the user's emotional state are necessary to enhance the effectiveness of training plans.

[0375] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0376] In this invention, the server includes means for inputting information on the skills, strengths, and weaknesses of crew members; means for dividing the crew into clusters based on the input information; means for automatically generating a training plan for each crew member based on the clusters; means for dynamically adjusting the plan using emotion recognition data when creating the plan; and means for displaying the training plan. This makes it possible to generate flexible training plans that take into account the emotional state of the user.

[0377] A "crew" refers to a group of people assembled to perform a specific task or job.

[0378] "Skills" refer to the specialized abilities and techniques necessary to perform a particular task or job.

[0379] "Strengths" refer to the outstanding characteristics or advantages that an individual or group possesses.

[0380] A "weakness" refers to a weakness or flaw that an individual or group possesses.

[0381] A "cluster" is a group of multiple data points that share similar characteristics or features.

[0382] A "training plan" outlines specific procedures and guidelines for improving individual skills and abilities in order to achieve particular goals.

[0383] "Emotion recognition data" refers to information that identifies a user's emotional state by analyzing their facial expressions, tone of voice, behavior, etc.

[0384] "Dynamic adjustment" refers to a process that automatically changes content and settings in response to changes in circumstances or data.

[0385] "Display means" refers to tools and interfaces that present generated information and data to users in an easily understandable way.

[0386] The present invention provides a system to support the training of robot operators in factories. This system provides an interface in which users can input the skills, strengths, and weaknesses of operators, and also has the function of creating a personalized training plan that reflects the user's emotional state in real time by combining it with an emotion recognition engine.

[0387] Means for entering crew information

[0388] Users input information such as the operator's ID, strengths, weaknesses, and skills into the system through a dedicated form or user interface. This information is transmitted to and stored on the server via the client terminal.

[0389] Data analysis methods

[0390] The server analyzes the data based on the input operator information. Specifically, it converts the operator's skills into numerical vectors and performs clustering based on these vectors. It calculates feature vectors using the TF-IDF (Term Frequency-Inverse Document Frequency) method and classifies operators into groups with similar skill sets using the K-Means algorithm.

[0391] Means for generating clusters

[0392] The server uses the above analysis data to perform clustering and classify operators into appropriate clusters. This categorizes operators with other operators who have similar skill sets.

[0393] Means for creating a training plan

[0394] The server automatically generates training plans based on the clustering results. Each cluster has specific training goals and action plans, and individual training plans are generated based on the cluster to which the operator belongs. For example, operators belonging to cluster 0 will have goals such as "improvement of specific skills" and "practical project management skills."

[0395] Adjustment methods based on emotion recognition

[0396] The system uses the camera and microphone built into the device to analyze the user's facial expressions and voice tone, recognizing their emotions. This emotion recognition engine operates in real time, generating emotion data. The server receives this emotion data and stores it in a database.

[0397] Dynamic adjustment method based on emotional data

[0398] The server dynamically adjusts the content and progress of the training plan based on emotion recognition data. For example, if a user is experiencing stress, the plan is adjusted to alleviate that stress; conversely, if the user is showing positive emotions, the plan is adjusted to increase the number of challenging tasks.

[0399] Means of displaying training plans

[0400] The terminal displays the training plan generated and adjusted by the server for user review. This allows users to check and implement the specific training plan for each operator. The training plan is output in JSON format, making it easy to integrate with other systems and tools.

[0401] Hardware and software to be used

[0402] Camera and microphone: Used to recognize the operator's emotions.

[0403] Tkinter: A library for creating GUIs.

[0404] Requests: Used for sending data.

[0405] Scikit-learn: Used for TF-IDF vectorization and KMeans clustering.

[0406] EmotionEngine: A custom emotion recognition engine.

[0407] Specific example

[0408] Skill set input: Enter specific skills such as "Java, Python".

[0409] Emotion Recognition: If a user is experiencing stress, the training plan automatically adjusts to a "reduced training plan."

[0410] Example of a prompt

[0411] "We are developing an application to support training plans for robot operators in factories. Please implement a function that analyzes the operator's emotional state in real time and dynamically adjusts the training plan based on their skill set."

[0412] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0413] Step 1:

[0414] Users input information such as operator ID, strengths, weaknesses, and skills through a dedicated form or user interface. This input data is sent from the terminal to the server and stored in a database. The input data consists of ID, strengths, weaknesses, skill set, etc. The stored data is used for subsequent analysis processing.

[0415] Step 2:

[0416] The server analyzes the data based on the input operator information. In particular, it converts the operator's skill set into a numerical vector. Feature vectors are calculated using the TF-IDF (Term Frequency-Inverse Document Frequency) method and used as data for clustering. In this step, skill data is used as input, and feature vectors are generated as output.

[0417] Step 3:

[0418] The server performs clustering using the K-Means algorithm based on feature vectors. This classifies operators into the same cluster as other operators with similar skill sets. Feature vectors are used as input, and the clustering results are obtained as output. The clustering results are data indicating which cluster each operator belongs to.

[0419] Step 4:

[0420] The server automatically generates a training plan based on the clustering results. Each cluster has specific training goals and action plans pre-configured, and an appropriate training plan is created based on the clustering results. The clustering results are used as input, and a training plan is generated as output. The specific training goals and action plans generated include both short-term and long-term goals.

[0421] Step 5:

[0422] The device uses its built-in camera and microphone to analyze the user's facial expressions and voice tone in real time and recognize their emotions. An emotion recognition engine processes this data to identify the user's emotional state. Facial expression data and voice data are used as input, and recognized emotion data is generated as output.

[0423] Step 6:

[0424] The server receives emotion recognition data and stores it in a database. This emotion data is also used for dynamic adjustments to the development plan. Emotion data is used as input, and the output stored in the database is data indicating each emotional state.

[0425] Step 7:

[0426] The server dynamically adjusts the content and progress of the training plan based on emotional data. For example, if a user is experiencing stress, the training plan is adjusted to alleviate that stress; if they are showing positive emotions, challenging tasks are added. The input is emotional data and the existing training plan, and the output is the adjusted training plan.

[0427] Step 8:

[0428] The adjusted training plan is displayed to the user via a terminal. This allows the user to review and implement a specific training plan for each operator. The adjusted training plan is used as input, and the output is the training plan displayed on the terminal. The training plan is also output in JSON format, allowing for easy integration with other systems and tools.

[0429] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0430] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0431] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0432] [Second Embodiment]

[0433] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0434] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0435] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0436] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0437] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0438] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0439] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0440] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0441] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0442] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0443] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0444] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0445] The present invention provides a system for automatically generating crew development plans, comprising means for inputting crew skills, strengths, and weaknesses, means for analysis, means for generating clusters, means for creating development plans, and means for displaying or outputting those plans. The system is implemented as follows:

[0446] 1. Means of entering crew information

[0447] Subject: User

[0448] Users input information such as crew IDs, strengths, weaknesses, and skills into the system. This is done through a dedicated form or user interface. This input information is sent to and stored on the server.

[0449] 2. Data Analysis Methods

[0450] Subject: Server

[0451] The server analyzes the data based on the entered crew information. In particular, it vectorizes the crew's skills and uses this to perform clustering. This process classifies crew members into groups with similar skill sets.

[0452] For example, if Crew A's skills are "Python, Management" and Crew B's skills are "Java, Hadoop," these skills are converted into numerical vectors, and clustered using feature vectors calculated with the TF-IDF (Term Frequency-Inverse Document Frequency) method.

[0453] 3. Means for generating clusters

[0454] Subject: Server

[0455] The server performs clustering using the previously analyzed data. It uses a clustering algorithm (e.g., K-Means) to group crew members with similar skills.

[0456] 4. Means of creating a training plan

[0457] Subject: Server

[0458] The server automatically generates training plans based on the clustering results. Since each cluster corresponds to a specific training goal, individual training plans are generated based on the cluster to which each crew member belongs.

[0459] Specifically, crews belonging to Cluster 0 are set with goals such as "improving Skill A" and "practicing project management," while crews belonging to Cluster 1 are set with goals such as "technical training in Skill B" and "promoting collaborative work."

[0460] 5. Means of displaying the training plan

[0461] Subject: terminal

[0462] The terminal displays the training plan generated by the server. This allows users to check and implement the specific training plan for each crew member. The training plan is output in JSON format, making it easy to integrate with other systems and tools.

[0463] For example, the following training plan will be generated:

[0464] Crew ID: C001

[0465] Short-term goals: Improve Skill A, practice project management.

[0466] Long-term goal: Leadership training

[0467] Duration: 6 months

[0468] Users can verify this and support the development of each crew member.

[0469] As described above, the system of the present invention can automatically create an efficient and effective crew training plan, thereby improving crew skills and operational efficiency.

[0470] The following describes the processing flow.

[0471] Step 1:

[0472] Subject: User

[0473] Users input crew information. Specifically, they enter crew IDs, strengths, weaknesses, skills, etc., through a dedicated form or user interface, and send this data to the server.

[0474] Step 2:

[0475] Subject: Server

[0476] The server stores the received crew information in a database. This includes fields for crew ID, strengths, weaknesses, and skills.

[0477] Step 3:

[0478] Subject: Server

[0479] The server vectorizes the saved crew skill information. This vectorization uses TfidfVectorizer to convert each crew member's skill into a numerical vector.

[0480] Step 4:

[0481] Subject: Server

[0482] The server performs clustering based on vectorized skill information. A clustering algorithm (e.g., K-Means) is used to group crews with similar skill sets.

[0483] Step 5:

[0484] Subject: Server

[0485] Based on the clustering results, the server determines the cluster to which each crew member belongs and stores this information in the database. Each crew member is assigned information about which cluster they belong to.

[0486] Step 6:

[0487] Subject: Server

[0488] The server automatically generates a growth plan based on each cluster. Specifically, it applies pre-configured growth goals and action plans for each cluster.

[0489] Step 7:

[0490] Subject: Server

[0491] The server converts the generated training plan into JSON format and saves it. This JSON data will be provided to the user in a later step.

[0492] Step 8:

[0493] Subject: terminal

[0494] The terminal sends a training plan request to the server. This is the procedure for users to check the training plans for each crew member.

[0495] Step 9:

[0496] Subject: Server

[0497] The server receives the request and sends the saved breeding plan in JSON format to the terminal.

[0498] Step 10:

[0499] Subject: terminal

[0500] The terminal displays the training plan received from the server on the screen. The user reviews this and takes specific actions to support the training of each crew member.

[0501] (Example 1)

[0502] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0503] The creation of conventional crew training plans has been difficult to implement efficiently and effectively due to the significant time and effort required to consider individual skills and characteristics. Furthermore, manual input and analysis can lead to errors and biases, making it challenging to design appropriate training plans for each crew member. This invention solves these problems by providing a system that efficiently inputs and analyzes crew information and automatically generates training plans.

[0504] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0505] In this invention, the server includes means for inputting information on the crew's skills, strengths, and weaknesses; means for vectorizing the crew data and generating feature vectors using the TF-IDF method; and means for dividing the crew into clusters using the feature vectors. This makes it possible to efficiently and automatically generate training plans for each crew member and output and display the training plans in JSON format.

[0506] "Crew skills" refer to the abilities and knowledge necessary to perform specific tasks or roles.

[0507] "Strengths" refer to the strengths and outstanding characteristics that the crew possesses.

[0508] "Weaknesses" refer to characteristics or shortcomings that the crew possesses that could be improved.

[0509] "Input method" refers to an interface for recording and transmitting information about crew members' skills, strengths, and weaknesses to the system.

[0510] "Vectorization" refers to the process of converting crew skill information into mathematical vectors and treating it as numerical data.

[0511] The "TF-IDF method" is a technique for evaluating the importance of words in text data, and is an abbreviation for Term Frequency-Inverse Document Frequency.

[0512] A "feature vector" is a numerical vector generated based on the crew's skills and is used as input data for clustering.

[0513] "Clustering" refers to the process of dividing data into groups based on similarity.

[0514] The "K-Means algorithm" refers to a clustering method for dividing data into K clusters.

[0515] A "development plan" refers to a document that includes goals and action plans for improving the crew's abilities and skills.

[0516] "JSON format" is an abbreviation for JavaScript Object Notation, and refers to a lightweight data format for structuring, storing, and transferring data.

[0517] A "database" refers to a system for efficiently storing, managing, and retrieving structured data.

[0518] "Means of display" refers to an interface for receiving crew training plans from the server and displaying them in a format viewable by the user.

[0519] The system of the present invention is for automatically generating crew training plans and includes the following means.

[0520] 1. Means of entering crew information

[0521] Subject: User

[0522] Users enter information such as crew IDs, strengths, weaknesses, and skills through a dedicated form or user interface. This involves users using a browser to input information into the UI and clicking a submit button. This input information is sent to the server and stored in a database.

[0523] 2. A method for vectorizing data and generating feature vectors using the TF-IDF method.

[0524] Subject: Server

[0525] The server analyzes crew data retrieved from the database. Specifically, it converts crew skill information into numerical vectors and generates feature vectors using the TF-IDF (Term Frequency-Inverse Document Frequency) method. These feature vectors are then used as input data for clustering.

[0526] 3. Methods for dividing the crew into clusters

[0527] Subject: Server

[0528] The server applies a clustering algorithm (e.g., K-Means) based on the generated feature vectors to group crews with similar skill sets. This automatically classifies crews into specific clusters.

[0529] 4. Means for automatically generating training plans

[0530] Subject: Server

[0531] The server automatically generates training plans for each crew member based on the clustering results. Each cluster has specific training goals, and individual training plans are generated according to the cluster to which each crew member belongs. Specifically, each crew member is given a detailed training plan (short-term goals, long-term goals, and duration).

[0532] 5. A means of outputting and displaying the training plan in JSON format.

[0533] Subject: terminal

[0534] The terminal displays the training plan generated by the server to the user. The training plan is sent from the server to the terminal in JSON format and displayed on the web interface. The user can check the training plan using a browser.

[0535] Hardware and software to be used

[0536] Server: Database system (e.g., MySQL), machine learning library (e.g., Scikit-learn)

[0537] Device: Web browser

[0538] Specific example

[0539] The user opens a web browser, accesses the specified URL, enters "Crew ID: C001, Skills: Python, Management, Strengths: Team Leadership, Weaknesses: Lack of Presentation Skills" into the input form, and clicks the "Submit" button. This information is sent to the server, vectorized using the TF-IDF method, and clustered using the K-Means algorithm. Finally, an automatically generated training plan is output in JSON format and displayed on the user's terminal.

[0540] Example of a prompt

[0541] Please enter the crew information: [Crew ID, Strengths, Weaknesses, Skills]

[0542] This will generate a training plan. Please click "Generate" once you have finished entering the information.

[0543] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0544] Step 1:

[0545] Subject: User

[0546] Users input and submit crew information (crew ID, strengths, weaknesses, skills) into the system through a dedicated form or user interface.

[0547] Input: Crew ID, Strengths, Weaknesses, Skills

[0548] Output: Crew information in JSON format is sent to the server.

[0549] Specific actions: The user opens a web browser and accesses the specified URL. They enter "Crew ID: C001, Skills: Python, Management, Strengths: Team leadership, Weaknesses: Lack of presentation skills" into the displayed input form and click the "Submit" button.

[0550] Step 2:

[0551] Subject: Server

[0552] The server receives crew information in JSON format sent by the user and stores it in the database.

[0553] Input: Crew information submitted by the user (in JSON format)

[0554] Output: Crew information stored in the database

[0555] Specific operation: The server receives an HTTP request sent by the user and parses the JSON data contained in its body. Then, it inserts the parsed data into a database (e.g., MySQL).

[0556] Step 3:

[0557] Subject: Server

[0558] The server retrieves crew information from the database and vectorizes the skill information. Feature vectors are generated using the TF-IDF method.

[0559] Input: Crew skill information retrieved from the database

[0560] Output: Vectorized skill information (feature vectors)

[0561] Specific operation: The server executes a script (e.g., a Python program) to retrieve skill information from the database. This skill information is converted into a numerical vector, and TF-IDF calculations are performed.

[0562] Step 4:

[0563] Subject: Server

[0564] The server applies the K-Means algorithm based on the generated feature vectors to classify the crew into clusters.

[0565] Input: Feature vector

[0566] Output: Cluster ID for each crew

[0567] Specific operation: The server uses a machine learning library (e.g., Scikit-learn) to perform K-Means clustering on the feature vectors. The result of the calculation is the cluster ID for each crew.

[0568] Step 5:

[0569] Subject: Server

[0570] The server automatically generates training plans for each crew member based on the clustering results. Each cluster has set training goals, which are used to create individual plans.

[0571] Input: Cluster ID and growth objectives

[0572] Output: Training plan for each crew member (JSON format)

[0573] Specific operation: Based on the clustering results, the server retrieves training objectives corresponding to each cluster. Next, it generates specific training plans (short-term objectives, long-term objectives, and duration) for each crew member.

[0574] Step 6:

[0575] Subject: terminal

[0576] The terminal displays the training plan generated by the server to the user. The training plan is sent from the server in JSON format and displayed via a web interface.

[0577] Input: Training plan (JSON format) sent from the server

[0578] Output: Displayed training plan

[0579] Specific operation: The user reopens their web browser and accesses the training plan display page. The browser retrieves JSON data from the server and displays it in a user-friendly format (e.g., an HTML table).

[0580] Example: Crew ID: C001, Short-term goal: Improve Skill A, practice project management, Long-term goal: Leadership training, Duration: 6 months

[0581] (Application Example 1)

[0582] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0583] The problem lies in the lack of a system that effectively and efficiently manages the skill sets and training plans of crew members working in the factory, and provides optimal training plans for each crew member in real time. Traditional manual management and planning are time-consuming and labor-intensive, and it was difficult to create optimal training plans tailored to the individual skills of each crew member. In addition, there was a problem in that training plans became mere formalities and did not lead to actual work efficiency improvements or skill development.

[0584] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0585] In this invention, the server includes means for inputting information on the skills, strengths, and weaknesses of crew members, means for dividing the crew into clusters based on the input information, and means for automatically generating a training plan for each crew member based on the clusters. This makes it possible to effectively manage the skill sets of crew members and provide optimal training plans in real time. Furthermore, by providing the training plans in JSON format and supplying them to factory robots, and by providing means for allowing crew members to check their training plans via tablet terminals, it becomes possible to propose crew placement and training plans in real time.

[0586] "Crew" refers to the workers and staff who work in a factory.

[0587] "Skills" is a general term for the specialized knowledge and techniques that a crew member possesses.

[0588] "Strengths" refers to the distinctive strengths and advantages of each crew member.

[0589] "Weaknesses" refer to the crew's shortcomings or areas that need improvement.

[0590] "Means of inputting information" refers to interfaces or devices used to input data on crew members' skills, strengths, and weaknesses.

[0591] "Methods for dividing crews into clusters" refers to algorithms or processes that group similar crews together based on their skills and characteristics.

[0592] "Means for automatically generating training plans" refers to systems or programs that automatically create training goals and methods for crew members in each cluster.

[0593] "Means for displaying training plans" refers to devices or software that visually display automatically generated training plans.

[0594] "Method for outputting in JSON format" refers to the function that exports the training plan in JSON (JavaScript Object Notation) format.

[0595] "Factory robots" refer to robots that assist crew members or perform automated tasks within a factory.

[0596] "A means of proposing crew placement and training plans in real time" refers to a system or algorithm in which factory robots instantly propose the optimal placement and training plan based on the current status and skills of the crew.

[0597] A "tablet device" refers to a portable computer device used to review training plans.

[0598] The system for realizing this application generates, manages, and implements effective crew training plans through an application installed on factory robots. The system consists of multiple hardware and software components.

[0599] Hardware and software to be used:

[0600] Hardware: Factory robots, servers, tablet devices

[0601] Software: Python (for data analysis and clustering), Django (web interface), MySQL (database management)

[0602] System processing procedures and functions:

[0603] 1. Enter and save crew information:

[0604] Users enter information about crew members, including their ID, strengths, weaknesses, and skills, via a tablet device. This information is sent to the server through a dedicated form or user interface and stored in a MySQL database.

[0605] 2. Data Analysis and Clustering:

[0606] The server uses Python to analyze the input crew information. Specifically, it uses the TF-IDF (Term Frequency-Inverse Document Frequency) method to create numerical vectors of crew skills and then performs clustering using the K-Means clustering algorithm.

[0607] 3. Automatic generation of training plans:

[0608] The server automatically generates a growth plan for each cluster based on the clustering results. The growth plan includes short-term and long-term goals and is saved in JSON format.

[0609] 4. Display and implementation of the training plan:

[0610] The factory robot receives the generated training plan in real time and displays it via a tablet device. Users can review the displayed training plan and implement specific training plans for their crew. The robot also handles crew placement and provides real-time training plan suggestions.

[0611] Specific example:

[0612] For example, let's say there are crew members A, B, and C working on a factory assembly line.

[0613] Crew A's skills: "Mechanical Engineering, Basic Programming"

[0614] Crew B's skills: "Basic Electrical Engineering, Applied Programming"

[0615] Crew C's skills: "Mechanical maintenance, Electrical maintenance"

[0616] The system converts these skills into numerical vectors and performs clustering. As a result, for example, Crew A and Crew B are classified into the same cluster, and training plans are set for this cluster: "Improvement of programming skills" and "Basic training in machine maintenance." A different training plan is generated for Crew C.

[0617] Examples of prompt statements for generated AI models:

[0618] Based on the crew skill information provided below, generate an optimal training plan. Crew skills have been numerically vectorized using the TF-IDF method, and K-Means is used for clustering. Propose specific training goals for each cluster.

[0619] Crew information:

[0620] 1. Crew ID: C001, Skills: "Mechanical Engineering, Basic Programming"

[0621] 2. Crew ID: C002, Skills: "Basic Electrical Engineering, Applied Programming"

[0622] 3. Crew ID: C003, Skills: "Machine Maintenance, Electrical Maintenance"

[0623] "

[0624] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0625] Step 1:

[0626] Users input crew information using a tablet device. Specifically, they enter the crew's ID, strengths, weaknesses, and skills into a dedicated input form and send that data to the server. The entered data is stored in a MySQL database on the server.

[0627] Input: Crew ID, strengths, weaknesses, skills

[0628] Output: Crew information stored in the database on the server

[0629] Step 2:

[0630] The server performs data analysis based on the stored crew information. Specifically, it uses Python to numerically vectorize crew skills using the TF-IDF (Term Frequency-Inverse Document Frequency) method. This quantifies each crew member's skills as feature vectors.

[0631] Input: Crew skill information stored in the database

[0632] Output: Numerically vectorized crew skill data

[0633] Step 3:

[0634] The server performs clustering based on vectorized skill data. Using the K-Means clustering algorithm, crews with similar skills are divided into clusters. The clustering results are stored in the database as the cluster ID for each crew member.

[0635] Input: Numerically vectorized crew skill data

[0636] Output: Crew information with assigned cluster IDs

[0637] Step 4:

[0638] The server automatically generates training plans based on the clustering results. Specifically, it assigns predefined training objectives (short-term and long-term goals) to each cluster and creates training plans for each crew member. The generated training plans are stored in the database in JSON format.

[0639] Input: Crew information with a cluster ID assigned.

[0640] Output: Training plan in JSON format

[0641] Step 5:

[0642] Factory robots receive training plans transmitted from a server in real time and display the crew training plans via tablet devices. This allows users to review and implement training plans for each crew member. The robots also suggest crew assignments and training plans.

[0643] Input: Training plan in JSON format

[0644] Output: Training plan displayed on the tablet device

[0645] Through these steps, the system will be able to efficiently manage crew skill information and automatically generate and provide optimal training plans in real time.

[0646] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0647] The present invention provides a system for automatically generating crew training plans, which combines means for inputting crew skills, strengths, and weaknesses, means for analyzing them, means for generating clusters, means for creating training plans, and means for displaying or outputting those plans, with an emotion engine that recognizes user emotions. This system is implemented as follows:

[0648] 1. Means of entering crew information

[0649] Subject: User

[0650] Users input information such as crew IDs, strengths, weaknesses, and skills into the system. This is done through a dedicated form or user interface. This input information is sent to and stored on the server.

[0651] 2. Data Analysis Methods

[0652] Subject: Server

[0653] The server analyzes the data based on the entered crew information. In particular, it vectorizes the crew's skills and uses this to perform clustering. This process classifies crew members into groups with similar skill sets.

[0654] For example, if Crew A's skills are "Python, Management" and Crew B's skills are "Java, Hadoop," these skills are converted into numerical vectors, and clustered using feature vectors calculated with the TF-IDF (Term Frequency-Inverse Document Frequency) method.

[0655] 3. Means for generating clusters

[0656] Subject: Server

[0657] The server performs clustering using the previously analyzed data. It uses a clustering algorithm (e.g., K-Means) to group crew members with similar skills.

[0658] 4. Means of creating a training plan

[0659] Subject: Server

[0660] The server automatically generates training plans based on the clustering results. Each cluster has specific training goals and action plans set, and individual training plans are generated based on the cluster to which each crew member belongs.

[0661] For example, crews belonging to Cluster 0 are set with goals such as "improving Skill A" and "practicing project management," while crews belonging to Cluster 1 are set with goals such as "technical training in Skill B" and "promoting collaborative work."

[0662] 5. Emotion recognition means using an emotion engine

[0663] Subject: terminal

[0664] Using the camera and microphone built into the device, the system analyzes the user's facial expressions and voice tone in real time to recognize the user's emotions. The emotion engine processes this data and generates emotion data.

[0665] 6. Methods for collecting emotional data

[0666] Subject: Server

[0667] The server receives emotional data transmitted from the emotion engine and stores it in a database. This data is used to adapt the crew's training plan.

[0668] 7. Means of adjusting development plans to reflect emotions

[0669] Subject: Server

[0670] The server dynamically adjusts the content and progress of the training plan based on the collected user emotion data. For example, if a user is experiencing stress, the plan is adjusted to reduce stress; conversely, if the user is showing positive emotions, the plan is adjusted to increase the number of challenging tasks.

[0671] 8. Means of displaying the training plan

[0672] Subject: terminal

[0673] The terminal displays the training plan generated and adjusted by the server. This allows users to review and implement the specific training plan for each crew member. The training plan is output in JSON format, making it easy to integrate with other systems and tools.

[0674] For example, the following training plan will be generated:

[0675] Crew ID: C001

[0676] Short-term goals: Improve Skill A, practice project management.

[0677] Long-term goal: Leadership training

[0678] Duration: 6 months

[0679] Users can verify this and support the development of each crew member.

[0680] The system of the present invention can automatically create efficient and effective crew training plans and, by taking into account the user's feelings, can achieve more personalized training support.

[0681] The following describes the processing flow.

[0682] Step 1:

[0683] Subject: User

[0684] Users input crew information. Specifically, they enter information such as crew ID, strengths, weaknesses, and skills through a dedicated form or user interface, and send this data to the server.

[0685] Step 2:

[0686] Subject: Server

[0687] The server stores the received crew information in a database. The database contains fields for Crew ID, Strengths, Weaknesses, and Skills.

[0688] Step 3:

[0689] Subject: Server

[0690] The server retrieves crew skill information from the database and vectorizes it. It uses TfidfVectorizer to convert skills into numerical vectors.

[0691] Step 4:

[0692] Subject: Server

[0693] Clustering is performed based on vectorized skills. A clustering algorithm (e.g., K-Means) is used to group crews with similar skill sets.

[0694] Step 5:

[0695] Subject: Server

[0696] The clustering results are analyzed to determine the cluster to which each crew member belongs. The determined cluster information is added to the crew member's data and saved to the database.

[0697] Step 6:

[0698] Subject: Server

[0699] The system automatically generates training plans based on clusters. It applies specific training goals and action plans set for each cluster to generate individual training plans.

[0700] Step 7:

[0701] Subject: terminal

[0702] The device analyzes the user's facial expressions and tone of voice in real time to recognize the user's emotions. It then uses an emotion engine to generate emotion data.

[0703] Step 8:

[0704] Subject: Server

[0705] The server receives emotional data sent from the terminals and stores it in a database. This emotional data is then used to inform the crew's training plans.

[0706] Step 9:

[0707] Subject: Server

[0708] The server analyzes emotional data and dynamically adjusts the training plan based on the collected user emotions. For example, if it determines that the user is experiencing stress, it adjusts the training plan to reduce the workload.

[0709] Step 10:

[0710] Subject: Server

[0711] The adjusted training plan is output in JSON format and saved back to the database. This format facilitates data integration with other systems.

[0712] Step 11:

[0713] Subject: terminal

[0714] The terminal sends a request for a training plan to the server and receives a customized training plan from the server.

[0715] Step 12:

[0716] Subject: terminal

[0717] The terminal displays the received training plan on its screen. The user reviews the displayed training plan and takes specific actions to support the training of each crew member.

[0718] (Example 2)

[0719] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0720] Traditional crew training systems typically provided uniform training plans without considering the emotional state of each individual crew member. This led to problems such as crew members experiencing stress or, conversely, a lack of stimulation resulting in decreased motivation. Furthermore, the manual generation of training plans was inefficient and time-consuming. Additionally, data format issues arose when integrating the generated training plans with other systems, resulting in low versatility.

[0721] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0722] In this invention, the server includes means for inputting information on the skills, strengths, and weaknesses of crew members; means for dividing the crew into clusters based on the input information; means for automatically generating a training plan for each crew member based on the clusters; means for dynamically adjusting the training plan using an emotion engine that recognizes the user's emotions; and means for displaying the training plan. This makes it possible to provide more personalized training plans that take into account the individual emotional state of each crew member, thereby resolving issues of crew stress and decreased motivation. Furthermore, since the training plan is output in JSON format, it is easy to link with other systems, improving versatility.

[0723] "Crew" refers to individual members who are engaged in a specific task or project.

[0724] "Skills" refer to the knowledge and abilities that a crew member possesses in order to perform a specific job or task.

[0725] "Strengths" refers to the individual strengths and outstanding characteristics of each crew member.

[0726] "Weaknesses" refer to the individual weaknesses or areas for improvement of each crew member.

[0727] "Means of input" refers to methods of entering data into a system through user interfaces or forms.

[0728] A "cluster" refers to a group of crew members who share similar skills or characteristics.

[0729] "Clustering" refers to the process of classifying crews based on their similarity using data analysis techniques.

[0730] A "training plan" refers to specific goals and action plans aimed at improving the skills and characteristics of the crew.

[0731] An "emotion engine" refers to a combination of hardware and software used to recognize and analyze a user's emotional state.

[0732] "Dynamic adjustment methods" refer to methods of adapting the plan based on real-time sentiment data.

[0733] "JSON format" refers to JavaScript Object Notation, a lightweight data format for data exchange.

[0734] "Means of display" refers to methods that allow users to visually confirm their training plan.

[0735] System Overview

[0736] The present invention relates to a system for automatically generating crew training plans. This system includes means for inputting crew skills, strengths, and weaknesses, means for analyzing data, means for generating clusters, means for creating training plans, and means for displaying or outputting those plans, and further incorporates an emotion engine that recognizes user emotions.

[0737] Hardware and software to be used

[0738] Hardware: Server, terminal (with camera and microphone), network connection

[0739] Software: Database management system, clustering algorithms (e.g., K-Means), Python libraries (scikit-learn, OpenCV), sentiment analysis engine, display interface, JSON output function.

[0740] Specific operation of the system

[0741] 1. Enter crew information

[0742] Users input information such as crew IDs, strengths, weaknesses, and skills into the system. This information is collected through dedicated forms and user interfaces. For example, a user might use a web browser to enter information such as "Crew ID: C001," "Strengths: Honesty," "Weaknesses: Communication," and "Skills: Java, Spring framework," and then click the submit button.

[0743] 2. Saving input data

[0744] The server receives information sent by the user and stores it in the database. For example, it can use SQL queries to store data in the database.

[0745] 3. Data Analysis

[0746] The server retrieves crew information from the database and converts the crew's skills into text vectors. Specifically, it vectorizes the data using the TF-IDF method and performs clustering using the K-Means algorithm. For example, the scikit-learn library in Python is used to convert the skill data into numerical vectors and then perform clustering.

[0747] 4. Cluster generation

[0748] The server classifies crew members into groups (clusters) with similar skill sets based on the clustering results. The generated clustering results are stored in a database.

[0749] 5. Automatic generation of training plans

[0750] The server automatically generates a training plan tailored to each cluster based on the clustering results. For example, it might create a plan for cluster 0 that includes "improving Skill A" and "practicing project management." This plan is then stored in the database.

[0751] 6. Emotion recognition by an emotion engine

[0752] When reviewing the training plan, the device's camera and microphone are used to recognize the user's emotions in real time. The emotion engine analyzes the user's facial expressions and voice tone to generate emotion data. For example, OpenCV is used to analyze facial expressions and estimate emotional states.

[0753] 7. Collection of emotional data

[0754] The server receives emotion data from the emotion engine and stores it in a database. For example, if a user is feeling stressed, that emotion data is sent to the server.

[0755] 8. Dynamic adjustment of the training plan

[0756] The server dynamically adjusts the content of the training plan based on the collected emotional data. For example, if stress levels are high, the training plan's workload is reduced, and if motivation is high, challenging goals are added.

[0757] 9. Display of the training plan

[0758] The finalized training plan is sent to the device and displayed for the user to review. Since the training plan is output in JSON format, it can be easily integrated with other systems.

[0759] Example of a prompt

[0760] Please enter the information for Crew C001 (ID, Strengths: Honesty, Weaknesses: Communication, Skills: Java, Spring framework) and generate a training plan.

[0761] This system allows users to easily generate personalized training plans for each crew member, taking emotional data into consideration to support more effective training.

[0762] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0763] Step 1: The user enters the crew information.

[0764] The user accesses a dedicated form on the system. They enter information such as the crew member's ID, strengths, weaknesses, and skills into the form and click the submit button. This input information is sent to the server. Specifically, the user enters "Crew ID: C001", "Strengths: Honesty", "Weaknesses: Communication", and "Skills: Java, Spring framework" into the form using a web browser. Input data: Crew member ID / Strengths / Weaknesses / Skills. Output data: Once the user completes the input and clicks the submit button, the user's input information is sent to the server.

[0765] Step 2: The server saves the input data.

[0766] The server receives information sent by the user and stores it in the database. Based on the received data, the server uses SQL queries to save information to the database. Specifically, the following SQL query is executed: INSERT INTO crew_data (ID, strengths, weaknesses, skills) VALUES ('C001', 'Sincerity', 'Communication', 'Java, Spring framework'). Input data: Crew information sent by the user. Output data: Crew information saved by the server in the database.

[0767] Step 3: The server analyzes the data.

[0768] The server retrieves crew information from the database and converts the crew's skills into text vectors. The server performs vectorization using the TF-IDF method and then clustering. Specifically, it uses the Python scikit-learn library to convert the crew's skills into numerical vectors and performs clustering using the KMeans(n_clusters=3) algorithm. Input data: Crew information stored in the database. Output data: Vectorized skills and clustering results.

[0769] Step 4: The server generates the cluster.

[0770] The server classifies the crew into groups based on the clustering results. The classification results are then saved back to the database. Specifically, assuming crew C001 belongs to cluster 1, the SQL query INSERT INTO cluster_data (crew_id, cluster_id) VALUES ('C001', 1) is executed. Input data: Clustering results. Output data: Cluster information saved in the database.

[0771] Step 5: The server automatically generates a training plan.

[0772] The server automatically generates a training plan suitable for each cluster based on the clustering results. Specifically, tasks such as "Improving Skill A" and "Practicing Project Management" are assigned to Cluster 1. The generated training plans are saved in the database. Input data: Clustering results. Output data: Training plans.

[0773] Step 6: The device recognizes the user's emotions.

[0774] While the training plan is being reviewed, the device's built-in camera and microphone are used to recognize the user's emotions in real time. The device's emotion engine analyzes the user's facial expressions and voice tone to generate emotion data. Specifically, it uses OpenCV to analyze facial expressions and estimate the emotional state. Input data: User's facial expressions and voice captured by the camera and microphone. Output data: Generated emotion data.

[0775] Step 7: The server collects emotion data.

[0776] The server receives emotion data sent from the emotion engine and stores it in the database. Specifically, an SQL query, INSERT INTO emotion_data (user_id, stress_level) VALUES ('U001', 70), is executed based on the emotion data. Input data: Emotion data sent from the terminal. Output data: Emotion data stored in the database.

[0777] Step 8: The server dynamically adjusts the growth plan.

[0778] The server dynamically adjusts the training plan based on the collected emotional data. Specifically, if the stress level is high, the workload of the plan is reduced, and conversely, if motivation is high, challenging goals are added. For example, the following SQL query is executed: UPDATE training_plans SET adjustments = 'Load Reduction' WHERE user_id = 'U001' AND stress_level > 60. Input data: Emotional data stored in the database. Output data: Adjusted training plan.

[0779] Step 9: The device displays the training plan.

[0780] The terminal receives the training plan prepared by the server and displays it on the user interface. For example, for crew member C001, information such as the short-term goal "Improve Skill A", the long-term goal "Leadership Training", and the duration "6 months" is displayed. Input data: Prepared training plan. Output data: Training plan displayed to the user.

[0781] (Application Example 2)

[0782] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0783] In conventional systems, crew training plans are automatically generated based on skills and characteristics, but they fail to take into account specific situations and emotional aspects, resulting in insufficient personalization for each crew member. In particular, in the training of robot operators within factories, plans that ignore individual emotions and stress levels hinder efficient training. Therefore, dynamic adjustments that respond to the user's emotional state are necessary to enhance the effectiveness of training plans.

[0784] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0785] In this invention, the server includes means for inputting information on the skills, strengths, and weaknesses of crew members; means for dividing the crew into clusters based on the input information; means for automatically generating a training plan for each crew member based on the clusters; means for dynamically adjusting the plan using emotion recognition data when creating the plan; and means for displaying the training plan. This makes it possible to generate flexible training plans that take into account the emotional state of the user.

[0786] A "crew" refers to a group of people assembled to perform a specific task or job.

[0787] "Skills" refer to the specialized abilities and techniques necessary to perform a particular task or job.

[0788] "Strengths" refer to the outstanding characteristics or advantages that an individual or group possesses.

[0789] A "weakness" refers to a weakness or flaw that an individual or group possesses.

[0790] A "cluster" is a group of multiple data points that share similar characteristics or features.

[0791] A "training plan" outlines specific procedures and guidelines for improving individual skills and abilities in order to achieve particular goals.

[0792] "Emotion recognition data" refers to information that identifies a user's emotional state by analyzing their facial expressions, tone of voice, behavior, etc.

[0793] "Dynamic adjustment" refers to a process that automatically changes content and settings in response to changes in circumstances or data.

[0794] "Display means" refers to tools and interfaces that present generated information and data to users in an easily understandable way.

[0795] The present invention provides a system to support the training of robot operators in factories. This system provides an interface in which users can input the skills, strengths, and weaknesses of operators, and also has the function of creating a personalized training plan that reflects the user's emotional state in real time by combining it with an emotion recognition engine.

[0796] Means for entering crew information

[0797] Users input information such as the operator's ID, strengths, weaknesses, and skills into the system through a dedicated form or user interface. This information is transmitted to and stored on the server via the client terminal.

[0798] Data analysis methods

[0799] The server analyzes the data based on the input operator information. Specifically, it converts the operator's skills into numerical vectors and performs clustering based on these vectors. It calculates feature vectors using the TF-IDF (Term Frequency-Inverse Document Frequency) method and classifies operators into groups with similar skill sets using the K-Means algorithm.

[0800] Means for generating clusters

[0801] The server uses the above analysis data to perform clustering and classify operators into appropriate clusters. This categorizes operators with other operators who have similar skill sets.

[0802] Means for creating a training plan

[0803] The server automatically generates training plans based on the clustering results. Each cluster has specific training goals and action plans, and individual training plans are generated based on the cluster to which the operator belongs. For example, operators belonging to cluster 0 will have goals such as "improvement of specific skills" and "practical project management skills."

[0804] Adjustment methods based on emotion recognition

[0805] The system uses the camera and microphone built into the device to analyze the user's facial expressions and voice tone, recognizing their emotions. This emotion recognition engine operates in real time, generating emotion data. The server receives this emotion data and stores it in a database.

[0806] Dynamic adjustment method based on emotional data

[0807] The server dynamically adjusts the content and progress of the training plan based on emotion recognition data. For example, if a user is experiencing stress, the plan is adjusted to alleviate that stress; conversely, if the user is showing positive emotions, the plan is adjusted to increase the number of challenging tasks.

[0808] Means of displaying training plans

[0809] The terminal displays the training plan generated and adjusted by the server for user review. This allows users to check and implement the specific training plan for each operator. The training plan is output in JSON format, making it easy to integrate with other systems and tools.

[0810] Hardware and software to be used

[0811] Camera and microphone: Used to recognize the operator's emotions.

[0812] Tkinter: A library for creating GUIs.

[0813] Requests: Used for sending data.

[0814] Scikit-learn: Used for TF-IDF vectorization and KMeans clustering.

[0815] EmotionEngine: A custom emotion recognition engine.

[0816] Specific example

[0817] Skill set input: Enter specific skills such as "Java, Python".

[0818] Emotion Recognition: If a user is experiencing stress, the training plan automatically adjusts to a "reduced training plan."

[0819] Example of a prompt

[0820] "We are developing an application to support training plans for robot operators in factories. Please implement a function that analyzes the operator's emotional state in real time and dynamically adjusts the training plan based on their skill set."

[0821] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0822] Step 1:

[0823] Users input information such as operator ID, strengths, weaknesses, and skills through a dedicated form or user interface. This input data is sent from the terminal to the server and stored in a database. The input data consists of ID, strengths, weaknesses, skill set, etc. The stored data is used for subsequent analysis processing.

[0824] Step 2:

[0825] The server analyzes the data based on the input operator information. In particular, it converts the operator's skill set into a numerical vector. Feature vectors are calculated using the TF-IDF (Term Frequency-Inverse Document Frequency) method and used as data for clustering. In this step, skill data is used as input, and feature vectors are generated as output.

[0826] Step 3:

[0827] The server performs clustering using the K-Means algorithm based on feature vectors. This classifies operators into the same cluster as other operators with similar skill sets. Feature vectors are used as input, and the clustering results are obtained as output. The clustering results are data indicating which cluster each operator belongs to.

[0828] Step 4:

[0829] The server automatically generates a training plan based on the clustering results. Each cluster has specific training goals and action plans pre-configured, and an appropriate training plan is created based on the clustering results. The clustering results are used as input, and a training plan is generated as output. The specific training goals and action plans generated include both short-term and long-term goals.

[0830] Step 5:

[0831] The device uses its built-in camera and microphone to analyze the user's facial expressions and voice tone in real time and recognize their emotions. An emotion recognition engine processes this data to identify the user's emotional state. Facial expression data and voice data are used as input, and recognized emotion data is generated as output.

[0832] Step 6:

[0833] The server receives emotion recognition data and stores it in a database. This emotion data is also used for dynamic adjustments to the development plan. Emotion data is used as input, and the output stored in the database is data indicating each emotional state.

[0834] Step 7:

[0835] The server dynamically adjusts the content and progress of the training plan based on emotional data. For example, if a user is experiencing stress, the training plan is adjusted to alleviate that stress; if they are showing positive emotions, challenging tasks are added. The input is emotional data and the existing training plan, and the output is the adjusted training plan.

[0836] Step 8:

[0837] The adjusted training plan is displayed to the user via a terminal. This allows the user to review and implement a specific training plan for each operator. The adjusted training plan is used as input, and the output is the training plan displayed on the terminal. The training plan is also output in JSON format, allowing for easy integration with other systems and tools.

[0838] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0839] The data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of the data generation model 58 is ChatGPT (Internet Search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0840] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0841] [Third Embodiment]

[0842] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0843] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0844] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0845] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0846] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0847] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0848] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0849] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0850] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0851] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0852] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0853] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0854] The present invention provides a system for automatically generating crew development plans, comprising means for inputting crew skills, strengths, and weaknesses, means for analysis, means for generating clusters, means for creating development plans, and means for displaying or outputting those plans. The system is implemented as follows:

[0855] 1. Means of entering crew information

[0856] Subject: User

[0857] Users input information such as crew IDs, strengths, weaknesses, and skills into the system. This is done through a dedicated form or user interface. This input information is sent to and stored on the server.

[0858] 2. Data Analysis Methods

[0859] Subject: Server

[0860] The server analyzes the data based on the entered crew information. In particular, it vectorizes the crew's skills and uses this to perform clustering. This process classifies crew members into groups with similar skill sets.

[0861] For example, if Crew A's skills are "Python, Management" and Crew B's skills are "Java, Hadoop," these skills are converted into numerical vectors, and clustered using feature vectors calculated with the TF-IDF (Term Frequency-Inverse Document Frequency) method.

[0862] 3. Means for generating clusters

[0863] Subject: Server

[0864] The server performs clustering using the previously analyzed data. It uses a clustering algorithm (e.g., K-Means) to group crew members with similar skills.

[0865] 4. Means of creating a training plan

[0866] Subject: Server

[0867] The server automatically generates training plans based on the clustering results. Since each cluster corresponds to a specific training goal, individual training plans are generated based on the cluster to which each crew member belongs.

[0868] Specifically, crews belonging to Cluster 0 are set with goals such as "improving Skill A" and "practicing project management," while crews belonging to Cluster 1 are set with goals such as "technical training in Skill B" and "promoting collaborative work."

[0869] 5. Means of displaying the training plan

[0870] Subject: terminal

[0871] The terminal displays the training plan generated by the server. This allows users to check and implement the specific training plan for each crew member. The training plan is output in JSON format, making it easy to integrate with other systems and tools.

[0872] For example, the following training plan will be generated:

[0873] Crew ID: C001

[0874] Short-term goals: Improve Skill A, practice project management.

[0875] Long-term goal: Leadership training

[0876] Duration: 6 months

[0877] Users can verify this and support the development of each crew member.

[0878] As described above, the system of the present invention can automatically create an efficient and effective crew training plan, thereby improving crew skills and operational efficiency.

[0879] The following describes the processing flow.

[0880] Step 1:

[0881] Subject: User

[0882] Users input crew information. Specifically, they enter crew IDs, strengths, weaknesses, skills, etc., through a dedicated form or user interface, and send this data to the server.

[0883] Step 2:

[0884] Subject: Server

[0885] The server stores the received crew information in a database. This includes fields for crew ID, strengths, weaknesses, and skills.

[0886] Step 3:

[0887] Subject: Server

[0888] The server vectorizes the saved crew skill information. This vectorization uses TfidfVectorizer to convert each crew member's skill into a numerical vector.

[0889] Step 4:

[0890] Subject: Server

[0891] The server performs clustering based on vectorized skill information. A clustering algorithm (e.g., K-Means) is used to group crews with similar skill sets.

[0892] Step 5:

[0893] Subject: Server

[0894] Based on the clustering results, the server determines the cluster to which each crew member belongs and stores this information in the database. Each crew member is assigned information about which cluster they belong to.

[0895] Step 6:

[0896] Subject: Server

[0897] The server automatically generates a growth plan based on each cluster. Specifically, it applies pre-configured growth goals and action plans for each cluster.

[0898] Step 7:

[0899] Subject: Server

[0900] The server converts the generated training plan into JSON format and saves it. This JSON data will be provided to the user in a later step.

[0901] Step 8:

[0902] Subject: terminal

[0903] The terminal sends a training plan request to the server. This is the procedure for users to check the training plans for each crew member.

[0904] Step 9:

[0905] Subject: Server

[0906] The server receives the request and sends the saved breeding plan in JSON format to the terminal.

[0907] Step 10:

[0908] Subject: terminal

[0909] The terminal displays the training plan received from the server on the screen. The user reviews this and takes specific actions to support the training of each crew member.

[0910] (Example 1)

[0911] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0912] The creation of conventional crew training plans has been difficult to implement efficiently and effectively due to the significant time and effort required to consider individual skills and characteristics. Furthermore, manual input and analysis can lead to errors and biases, making it challenging to design appropriate training plans for each crew member. This invention solves these problems by providing a system that efficiently inputs and analyzes crew information and automatically generates training plans.

[0913] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0914] In this invention, the server includes means for inputting information on the crew's skills, strengths, and weaknesses; means for vectorizing the crew data and generating feature vectors using the TF-IDF method; and means for dividing the crew into clusters using the feature vectors. This makes it possible to efficiently and automatically generate training plans for each crew member and output and display the training plans in JSON format.

[0915] "Crew skills" refer to the abilities and knowledge necessary to perform specific tasks or roles.

[0916] "Strengths" refer to the strengths and outstanding characteristics that the crew possesses.

[0917] "Weaknesses" refer to characteristics or shortcomings that the crew possesses that could be improved.

[0918] "Input method" refers to an interface for recording and transmitting information about crew members' skills, strengths, and weaknesses to the system.

[0919] "Vectorization" refers to the process of converting crew skill information into mathematical vectors and treating it as numerical data.

[0920] The "TF-IDF method" is a technique for evaluating the importance of words in text data, and is an abbreviation for Term Frequency-Inverse Document Frequency.

[0921] A "feature vector" is a numerical vector generated based on the crew's skills and is used as input data for clustering.

[0922] "Clustering" refers to the process of dividing data into groups based on similarity.

[0923] The "K-Means algorithm" refers to a clustering method for dividing data into K clusters.

[0924] A "development plan" refers to a document that includes goals and action plans for improving the crew's abilities and skills.

[0925] "JSON format" is an abbreviation for JavaScript Object Notation, and refers to a lightweight data format for structuring, storing, and transferring data.

[0926] A "database" refers to a system for efficiently storing, managing, and retrieving structured data.

[0927] "Means of display" refers to an interface for receiving crew training plans from the server and displaying them in a format viewable by the user.

[0928] The system of the present invention is for automatically generating crew training plans and includes the following means.

[0929] 1. Means of entering crew information

[0930] Subject: User

[0931] Users enter information such as crew IDs, strengths, weaknesses, and skills through a dedicated form or user interface. This involves users using a browser to input information into the UI and clicking a submit button. This input information is sent to the server and stored in a database.

[0932] 2. A method for vectorizing data and generating feature vectors using the TF-IDF method.

[0933] Subject: Server

[0934] The server analyzes crew data retrieved from the database. Specifically, it converts crew skill information into numerical vectors and generates feature vectors using the TF-IDF (Term Frequency-Inverse Document Frequency) method. These feature vectors are then used as input data for clustering.

[0935] 3. Methods for dividing the crew into clusters

[0936] Subject: Server

[0937] The server applies a clustering algorithm (e.g., K-Means) based on the generated feature vectors to group crews with similar skill sets. This automatically classifies crews into specific clusters.

[0938] 4. Means for automatically generating training plans

[0939] Subject: Server

[0940] The server automatically generates training plans for each crew member based on the clustering results. Each cluster has specific training goals, and individual training plans are generated according to the cluster to which each crew member belongs. Specifically, each crew member is given a detailed training plan (short-term goals, long-term goals, and duration).

[0941] 5. A means of outputting and displaying the training plan in JSON format.

[0942] Subject: terminal

[0943] The terminal displays the training plan generated by the server to the user. The training plan is sent from the server to the terminal in JSON format and displayed on the web interface. The user can check the training plan using a browser.

[0944] Hardware and software to be used

[0945] Server: Database system (e.g., MySQL), machine learning library (e.g., Scikit-learn)

[0946] Device: Web browser

[0947] Specific example

[0948] The user opens a web browser, accesses the specified URL, enters "Crew ID: C001, Skills: Python, Management, Strengths: Team Leadership, Weaknesses: Lack of Presentation Skills" into the input form, and clicks the "Submit" button. This information is sent to the server, vectorized using the TF-IDF method, and clustered using the K-Means algorithm. Finally, an automatically generated training plan is output in JSON format and displayed on the user's terminal.

[0949] Example of a prompt

[0950] Please enter the crew information: [Crew ID, Strengths, Weaknesses, Skills]

[0951] This will generate a training plan. Please click "Generate" once you have finished entering the information.

[0952] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0953] Step 1:

[0954] Subject: User

[0955] Users input and submit crew information (crew ID, strengths, weaknesses, skills) into the system through a dedicated form or user interface.

[0956] Input: Crew ID, Strengths, Weaknesses, Skills

[0957] Output: Crew information in JSON format is sent to the server.

[0958] Specific actions: The user opens a web browser and accesses the specified URL. They enter "Crew ID: C001, Skills: Python, Management, Strengths: Team leadership, Weaknesses: Lack of presentation skills" into the displayed input form and click the "Submit" button.

[0959] Step 2:

[0960] Subject: Server

[0961] The server receives crew information in JSON format sent by the user and stores it in the database.

[0962] Input: Crew information submitted by the user (in JSON format)

[0963] Output: Crew information stored in the database

[0964] Specific operation: The server receives an HTTP request sent by the user and parses the JSON data contained in its body. Then, it inserts the parsed data into a database (e.g., MySQL).

[0965] Step 3:

[0966] Subject: Server

[0967] The server retrieves crew information from the database and vectorizes the skill information. Feature vectors are generated using the TF-IDF method.

[0968] Input: Crew skill information retrieved from the database

[0969] Output: Vectorized skill information (feature vectors)

[0970] Specific operation: The server executes a script (e.g., a Python program) to retrieve skill information from the database. This skill information is converted into a numerical vector, and TF-IDF calculations are performed.

[0971] Step 4:

[0972] Subject: Server

[0973] The server applies the K-Means algorithm based on the generated feature vectors to classify the crew into clusters.

[0974] Input: Feature vector

[0975] Output: Cluster ID for each crew

[0976] Specific operation: The server uses a machine learning library (e.g., Scikit-learn) to perform K-Means clustering on the feature vectors. The result of the calculation is the cluster ID for each crew.

[0977] Step 5:

[0978] Subject: Server

[0979] The server automatically generates training plans for each crew member based on the clustering results. Each cluster has set training goals, which are used to create individual plans.

[0980] Input: Cluster ID and growth objectives

[0981] Output: Training plan for each crew member (JSON format)

[0982] Specific operation: Based on the clustering results, the server retrieves training objectives corresponding to each cluster. Next, it generates specific training plans (short-term objectives, long-term objectives, and duration) for each crew member.

[0983] Step 6:

[0984] Subject: terminal

[0985] The terminal displays the training plan generated by the server to the user. The training plan is sent from the server in JSON format and displayed via a web interface.

[0986] Input: Training plan (JSON format) sent from the server

[0987] Output: Displayed training plan

[0988] Specific operation: The user reopens their web browser and accesses the training plan display page. The browser retrieves JSON data from the server and displays it in a user-friendly format (e.g., an HTML table).

[0989] Example: Crew ID: C001, Short-term goal: Improve Skill A, practice project management, Long-term goal: Leadership training, Duration: 6 months

[0990] (Application Example 1)

[0991] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0992] The problem lies in the lack of a system that effectively and efficiently manages the skill sets and training plans of crew members working in the factory, and provides optimal training plans for each crew member in real time. Traditional manual management and planning are time-consuming and labor-intensive, and it was difficult to create optimal training plans tailored to the individual skills of each crew member. In addition, there was a problem in that training plans became mere formalities and did not lead to actual work efficiency improvements or skill development.

[0993] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0994] In this invention, the server includes means for inputting information on the skills, strengths, and weaknesses of crew members, means for dividing the crew into clusters based on the input information, and means for automatically generating a training plan for each crew member based on the clusters. This makes it possible to effectively manage the skill sets of crew members and provide optimal training plans in real time. Furthermore, by providing the training plans in JSON format and supplying them to factory robots, and by providing means for allowing crew members to check their training plans via tablet terminals, it becomes possible to propose crew placement and training plans in real time.

[0995] "Crew" refers to the workers and staff who work in a factory.

[0996] "Skills" is a general term for the specialized knowledge and techniques that a crew member possesses.

[0997] "Strengths" refers to the distinctive strengths and advantages of each crew member.

[0998] "Weaknesses" refer to the crew's shortcomings or areas that need improvement.

[0999] "Means of inputting information" refers to interfaces or devices used to input data on crew members' skills, strengths, and weaknesses.

[1000] "Methods for dividing crews into clusters" refers to algorithms or processes that group similar crews together based on their skills and characteristics.

[1001] "Means for automatically generating training plans" refers to systems or programs that automatically create training goals and methods for crew members in each cluster.

[1002] "Means for displaying training plans" refers to devices or software that visually display automatically generated training plans.

[1003] "Method for outputting in JSON format" refers to the function that exports the training plan in JSON (JavaScript Object Notation) format.

[1004] "Factory robots" refer to robots that assist crew members or perform automated tasks within a factory.

[1005] "A means of proposing crew placement and training plans in real time" refers to a system or algorithm in which factory robots instantly propose the optimal placement and training plan based on the current status and skills of the crew.

[1006] A "tablet device" refers to a portable computer device used to review training plans.

[1007] The system for realizing this application generates, manages, and implements effective crew training plans through an application installed on factory robots. The system consists of multiple hardware and software components.

[1008] Hardware and software to be used:

[1009] Hardware: Factory robots, servers, tablet devices

[1010] Software: Python (for data analysis and clustering), Django (web interface), MySQL (database management)

[1011] System processing procedures and functions:

[1012] 1. Enter and save crew information:

[1013] Users enter information about crew members, including their ID, strengths, weaknesses, and skills, via a tablet device. This information is sent to the server through a dedicated form or user interface and stored in a MySQL database.

[1014] 2. Data Analysis and Clustering:

[1015] The server uses Python to analyze the input crew information. Specifically, it uses the TF-IDF (Term Frequency-Inverse Document Frequency) method to create numerical vectors of crew skills and then performs clustering using the K-Means clustering algorithm.

[1016] 3. Automatic generation of training plans:

[1017] The server automatically generates a growth plan for each cluster based on the clustering results. The growth plan includes short-term and long-term goals and is saved in JSON format.

[1018] 4. Display and implementation of the training plan:

[1019] The factory robot receives the generated training plan in real time and displays it via a tablet device. Users can review the displayed training plan and implement specific training plans for their crew. The robot also handles crew placement and provides real-time training plan suggestions.

[1020] Specific example:

[1021] For example, let's say there are crew members A, B, and C working on a factory assembly line.

[1022] Crew A's skills: "Mechanical Engineering, Basic Programming"

[1023] Crew B's skills: "Basic Electrical Engineering, Applied Programming"

[1024] Crew C's skills: "Mechanical maintenance, Electrical maintenance"

[1025] The system converts these skills into numerical vectors and performs clustering. As a result, for example, Crew A and Crew B are classified into the same cluster, and training plans are set for this cluster: "Improvement of programming skills" and "Basic training in machine maintenance." A different training plan is generated for Crew C.

[1026] Examples of prompt statements for generated AI models:

[1027] Based on the crew skill information provided below, generate an optimal training plan. Crew skills have been numerically vectorized using the TF-IDF method, and K-Means is used for clustering. Propose specific training goals for each cluster.

[1028] Crew information:

[1029] 1. Crew ID: C001, Skills: "Mechanical Engineering, Basic Programming"

[1030] 2. Crew ID: C002, Skills: "Basic Electrical Engineering, Applied Programming"

[1031] 3. Crew ID: C003, Skills: "Machine Maintenance, Electrical Maintenance"

[1032] "

[1033] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1034] Step 1:

[1035] Users input crew information using a tablet device. Specifically, they enter the crew's ID, strengths, weaknesses, and skills into a dedicated input form and send that data to the server. The entered data is stored in a MySQL database on the server.

[1036] Input: Crew ID, strengths, weaknesses, skills

[1037] Output: Crew information stored in the database on the server

[1038] Step 2:

[1039] The server performs data analysis based on the stored crew information. Specifically, it uses Python to numerically vectorize crew skills using the TF-IDF (Term Frequency-Inverse Document Frequency) method. This quantifies each crew member's skills as feature vectors.

[1040] Input: Crew skill information stored in the database

[1041] Output: Numerically vectorized crew skill data

[1042] Step 3:

[1043] The server performs clustering based on vectorized skill data. Using the K-Means clustering algorithm, crews with similar skills are divided into clusters. The clustering results are stored in the database as the cluster ID for each crew member.

[1044] Input: Numerically vectorized crew skill data

[1045] Output: Crew information with assigned cluster IDs

[1046] Step 4:

[1047] The server automatically generates training plans based on the clustering results. Specifically, it assigns predefined training objectives (short-term and long-term goals) to each cluster and creates training plans for each crew member. The generated training plans are stored in the database in JSON format.

[1048] Input: Crew information with a cluster ID assigned.

[1049] Output: Training plan in JSON format

[1050] Step 5:

[1051] Factory robots receive training plans transmitted from a server in real time and display the crew training plans via tablet devices. This allows users to review and implement training plans for each crew member. The robots also suggest crew assignments and training plans.

[1052] Input: Training plan in JSON format

[1053] Output: Training plan displayed on the tablet device

[1054] Through these steps, the system will be able to efficiently manage crew skill information and automatically generate and provide optimal training plans in real time.

[1055] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1056] The present invention provides a system for automatically generating crew training plans, which combines means for inputting crew skills, strengths, and weaknesses, means for analyzing them, means for generating clusters, means for creating training plans, and means for displaying or outputting those plans, with an emotion engine that recognizes user emotions. This system is implemented as follows:

[1057] 1. Means of entering crew information

[1058] Subject: User

[1059] Users input information such as crew IDs, strengths, weaknesses, and skills into the system. This is done through a dedicated form or user interface. This input information is sent to and stored on the server.

[1060] 2. Data Analysis Methods

[1061] Subject: Server

[1062] The server analyzes the data based on the entered crew information. In particular, it vectorizes the crew's skills and uses this to perform clustering. This process classifies crew members into groups with similar skill sets.

[1063] For example, if Crew A's skills are "Python, Management" and Crew B's skills are "Java, Hadoop," these skills are converted into numerical vectors, and clustered using feature vectors calculated with the TF-IDF (Term Frequency-Inverse Document Frequency) method.

[1064] 3. Means for generating clusters

[1065] Subject: Server

[1066] The server performs clustering using the previously analyzed data. It uses a clustering algorithm (e.g., K-Means) to group crew members with similar skills.

[1067] 4. Means of creating a training plan

[1068] Subject: Server

[1069] The server automatically generates training plans based on the clustering results. Each cluster has specific training goals and action plans set, and individual training plans are generated based on the cluster to which each crew member belongs.

[1070] For example, crews belonging to Cluster 0 are set with goals such as "improving Skill A" and "practicing project management," while crews belonging to Cluster 1 are set with goals such as "technical training in Skill B" and "promoting collaborative work."

[1071] 5. Emotion recognition means using an emotion engine

[1072] Subject: terminal

[1073] Using the camera and microphone built into the device, the system analyzes the user's facial expressions and voice tone in real time to recognize the user's emotions. The emotion engine processes this data and generates emotion data.

[1074] 6. Methods for collecting emotional data

[1075] Subject: Server

[1076] The server receives emotional data transmitted from the emotion engine and stores it in a database. This data is used to adapt the crew's training plan.

[1077] 7. Means of adjusting development plans to reflect emotions

[1078] Subject: Server

[1079] The server dynamically adjusts the content and progress of the training plan based on the collected user emotion data. For example, if a user is experiencing stress, the plan is adjusted to reduce stress; conversely, if the user is showing positive emotions, the plan is adjusted to increase the number of challenging tasks.

[1080] 8. Means of displaying the training plan

[1081] Subject: terminal

[1082] The terminal displays the training plan generated and adjusted by the server. This allows users to review and implement the specific training plan for each crew member. The training plan is output in JSON format, making it easy to integrate with other systems and tools.

[1083] For example, the following training plan will be generated:

[1084] Crew ID: C001

[1085] Short-term goals: Improve Skill A, practice project management.

[1086] Long-term goal: Leadership training

[1087] Duration: 6 months

[1088] Users can verify this and support the development of each crew member.

[1089] The system of the present invention can automatically create efficient and effective crew training plans and, by taking into account the user's feelings, can achieve more personalized training support.

[1090] The following describes the processing flow.

[1091] Step 1:

[1092] Subject: User

[1093] Users input crew information. Specifically, they enter information such as crew ID, strengths, weaknesses, and skills through a dedicated form or user interface, and send this data to the server.

[1094] Step 2:

[1095] Subject: Server

[1096] The server stores the received crew information in a database. The database contains fields for Crew ID, Strengths, Weaknesses, and Skills.

[1097] Step 3:

[1098] Subject: Server

[1099] The server retrieves crew skill information from the database and vectorizes it. It uses TfidfVectorizer to convert skills into numerical vectors.

[1100] Step 4:

[1101] Subject: Server

[1102] Clustering is performed based on vectorized skills. A clustering algorithm (e.g., K-Means) is used to group crews with similar skill sets.

[1103] Step 5:

[1104] Subject: Server

[1105] The clustering results are analyzed to determine the cluster to which each crew member belongs. The determined cluster information is added to the crew member's data and saved to the database.

[1106] Step 6:

[1107] Subject: Server

[1108] The system automatically generates training plans based on clusters. It applies specific training goals and action plans set for each cluster to generate individual training plans.

[1109] Step 7:

[1110] Subject: terminal

[1111] The device analyzes the user's facial expressions and tone of voice in real time to recognize the user's emotions. It then uses an emotion engine to generate emotion data.

[1112] Step 8:

[1113] Subject: Server

[1114] The server receives emotional data sent from the terminals and stores it in a database. This emotional data is then used to inform the crew's training plans.

[1115] Step 9:

[1116] Subject: Server

[1117] The server analyzes emotional data and dynamically adjusts the training plan based on the collected user emotions. For example, if it determines that the user is experiencing stress, it adjusts the training plan to reduce the workload.

[1118] Step 10:

[1119] Subject: Server

[1120] The adjusted training plan is output in JSON format and saved back to the database. This format facilitates data integration with other systems.

[1121] Step 11:

[1122] Subject: terminal

[1123] The terminal sends a request for a training plan to the server and receives a customized training plan from the server.

[1124] Step 12:

[1125] Subject: terminal

[1126] The terminal displays the received training plan on its screen. The user reviews the displayed training plan and takes specific actions to support the training of each crew member.

[1127] (Example 2)

[1128] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1129] Traditional crew training systems typically provided uniform training plans without considering the emotional state of each individual crew member. This led to problems such as crew members experiencing stress or, conversely, a lack of stimulation resulting in decreased motivation. Furthermore, the manual generation of training plans was inefficient and time-consuming. Additionally, data format issues arose when integrating the generated training plans with other systems, resulting in low versatility.

[1130] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1131] In this invention, the server includes means for inputting information on the skills, strengths, and weaknesses of crew members; means for dividing the crew into clusters based on the input information; means for automatically generating a training plan for each crew member based on the clusters; means for dynamically adjusting the training plan using an emotion engine that recognizes the user's emotions; and means for displaying the training plan. This makes it possible to provide more personalized training plans that take into account the individual emotional state of each crew member, thereby resolving issues of crew stress and decreased motivation. Furthermore, since the training plan is output in JSON format, it is easy to link with other systems, improving versatility.

[1132] "Crew" refers to individual members who are engaged in a specific task or project.

[1133] "Skills" refer to the knowledge and abilities that a crew member possesses in order to perform a specific job or task.

[1134] "Strengths" refers to the individual strengths and outstanding characteristics of each crew member.

[1135] "Weaknesses" refer to the individual weaknesses or areas for improvement of each crew member.

[1136] "Means of input" refers to methods of entering data into a system through user interfaces or forms.

[1137] A "cluster" refers to a group of crew members who share similar skills or characteristics.

[1138] "Clustering" refers to the process of classifying crews based on their similarity using data analysis techniques.

[1139] A "training plan" refers to specific goals and action plans aimed at improving the skills and characteristics of the crew.

[1140] An "emotion engine" refers to a combination of hardware and software used to recognize and analyze a user's emotional state.

[1141] "Dynamic adjustment methods" refer to methods of adapting the plan based on real-time sentiment data.

[1142] "JSON format" refers to JavaScript Object Notation, a lightweight data format for data exchange.

[1143] "Means of display" refers to methods that allow users to visually confirm their training plan.

[1144] System Overview

[1145] The present invention relates to a system for automatically generating crew training plans. This system includes means for inputting crew skills, strengths, and weaknesses, means for analyzing data, means for generating clusters, means for creating training plans, and means for displaying or outputting those plans, and further incorporates an emotion engine that recognizes user emotions.

[1146] Hardware and software to be used

[1147] Hardware: Server, terminal (with camera and microphone), network connection

[1148] Software: Database management system, clustering algorithms (e.g., K-Means), Python libraries (scikit-learn, OpenCV), sentiment analysis engine, display interface, JSON output function.

[1149] Specific operation of the system

[1150] 1. Enter crew information

[1151] Users input information such as crew IDs, strengths, weaknesses, and skills into the system. This information is collected through dedicated forms and user interfaces. For example, a user might use a web browser to enter information such as "Crew ID: C001," "Strengths: Honesty," "Weaknesses: Communication," and "Skills: Java, Spring framework," and then click the submit button.

[1152] 2. Saving input data

[1153] The server receives information sent by the user and stores it in the database. For example, it can use SQL queries to store data in the database.

[1154] 3. Data Analysis

[1155] The server retrieves crew information from the database and converts the crew's skills into text vectors. Specifically, it vectorizes the data using the TF-IDF method and performs clustering using the K-Means algorithm. For example, the scikit-learn library in Python is used to convert the skill data into numerical vectors and then perform clustering.

[1156] 4. Cluster generation

[1157] The server classifies crew members into groups (clusters) with similar skill sets based on the clustering results. The generated clustering results are stored in a database.

[1158] 5. Automatic generation of training plans

[1159] The server automatically generates a training plan tailored to each cluster based on the clustering results. For example, it might create a plan for cluster 0 that includes "improving Skill A" and "practicing project management." This plan is then stored in the database.

[1160] 6. Emotion recognition by an emotion engine

[1161] When reviewing the training plan, the device's camera and microphone are used to recognize the user's emotions in real time. The emotion engine analyzes the user's facial expressions and voice tone to generate emotion data. For example, OpenCV is used to analyze facial expressions and estimate emotional states.

[1162] 7. Collection of emotional data

[1163] The server receives emotion data from the emotion engine and stores it in a database. For example, if a user is feeling stressed, that emotion data is sent to the server.

[1164] 8. Dynamic adjustment of the training plan

[1165] The server dynamically adjusts the content of the training plan based on the collected emotional data. For example, if stress levels are high, the training plan's workload is reduced, and if motivation is high, challenging goals are added.

[1166] 9. Display of the training plan

[1167] The finalized training plan is sent to the device and displayed for the user to review. Since the training plan is output in JSON format, it can be easily integrated with other systems.

[1168] Example of a prompt

[1169] Please enter the information for Crew C001 (ID, Strengths: Honesty, Weaknesses: Communication, Skills: Java, Spring framework) and generate a training plan.

[1170] This system allows users to easily generate personalized training plans for each crew member, taking emotional data into consideration to support more effective training.

[1171] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1172] Step 1: The user enters the crew information.

[1173] The user accesses a dedicated form on the system. They enter information such as the crew member's ID, strengths, weaknesses, and skills into the form and click the submit button. This input information is sent to the server. Specifically, the user enters "Crew ID: C001", "Strengths: Honesty", "Weaknesses: Communication", and "Skills: Java, Spring framework" into the form using a web browser. Input data: Crew member ID / Strengths / Weaknesses / Skills. Output data: Once the user completes the input and clicks the submit button, the user's input information is sent to the server.

[1174] Step 2: The server saves the input data.

[1175] The server receives information sent by the user and stores it in the database. Based on the received data, the server uses SQL queries to save information to the database. Specifically, the following SQL query is executed: INSERT INTO crew_data (ID, strengths, weaknesses, skills) VALUES ('C001', 'Sincerity', 'Communication', 'Java, Spring framework'). Input data: Crew information sent by the user. Output data: Crew information saved by the server in the database.

[1176] Step 3: The server analyzes the data.

[1177] The server retrieves crew information from the database and converts the crew's skills into text vectors. The server performs vectorization using the TF-IDF method and then clustering. Specifically, it uses the Python scikit-learn library to convert the crew's skills into numerical vectors and performs clustering using the KMeans(n_clusters=3) algorithm. Input data: Crew information stored in the database. Output data: Vectorized skills and clustering results.

[1178] Step 4: The server generates the cluster.

[1179] The server classifies the crew into groups based on the clustering results. The classification results are then saved back to the database. Specifically, assuming crew C001 belongs to cluster 1, the SQL query INSERT INTO cluster_data (crew_id, cluster_id) VALUES ('C001', 1) is executed. Input data: Clustering results. Output data: Cluster information saved in the database.

[1180] Step 5: The server automatically generates a training plan.

[1181] The server automatically generates a training plan suitable for each cluster based on the clustering results. Specifically, tasks such as "Improving Skill A" and "Practicing Project Management" are assigned to Cluster 1. The generated training plans are saved in the database. Input data: Clustering results. Output data: Training plans.

[1182] Step 6: The device recognizes the user's emotions.

[1183] While the training plan is being reviewed, the device's built-in camera and microphone are used to recognize the user's emotions in real time. The device's emotion engine analyzes the user's facial expressions and voice tone to generate emotion data. Specifically, it uses OpenCV to analyze facial expressions and estimate the emotional state. Input data: User's facial expressions and voice captured by the camera and microphone. Output data: Generated emotion data.

[1184] Step 7: The server collects emotion data.

[1185] The server receives emotion data sent from the emotion engine and stores it in the database. Specifically, an SQL query, INSERT INTO emotion_data (user_id, stress_level) VALUES ('U001', 70), is executed based on the emotion data. Input data: Emotion data sent from the terminal. Output data: Emotion data stored in the database.

[1186] Step 8: The server dynamically adjusts the growth plan.

[1187] The server dynamically adjusts the training plan based on the collected emotional data. Specifically, if the stress level is high, the workload of the plan is reduced, and conversely, if motivation is high, challenging goals are added. For example, the following SQL query is executed: UPDATE training_plans SET adjustments = 'Load Reduction' WHERE user_id = 'U001' AND stress_level > 60. Input data: Emotional data stored in the database. Output data: Adjusted training plan.

[1188] Step 9: The device displays the training plan.

[1189] The terminal receives the training plan prepared by the server and displays it on the user interface. For example, for crew member C001, information such as the short-term goal "Improve Skill A", the long-term goal "Leadership Training", and the duration "6 months" is displayed. Input data: Prepared training plan. Output data: Training plan displayed to the user.

[1190] (Application Example 2)

[1191] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1192] In conventional systems, crew training plans are automatically generated based on skills and characteristics, but they fail to take into account specific situations and emotional aspects, resulting in insufficient personalization for each crew member. In particular, in the training of robot operators within factories, plans that ignore individual emotions and stress levels hinder efficient training. Therefore, dynamic adjustments that respond to the user's emotional state are necessary to enhance the effectiveness of training plans.

[1193] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1194] In this invention, the server includes means for inputting information on the skills, strengths, and weaknesses of crew members; means for dividing the crew into clusters based on the input information; means for automatically generating a training plan for each crew member based on the clusters; means for dynamically adjusting the plan using emotion recognition data when creating the plan; and means for displaying the training plan. This makes it possible to generate flexible training plans that take into account the emotional state of the user.

[1195] A "crew" refers to a group of people assembled to perform a specific task or job.

[1196] "Skills" refer to the specialized abilities and techniques necessary to perform a particular task or job.

[1197] "Strengths" refer to the outstanding characteristics or advantages that an individual or group possesses.

[1198] A "weakness" refers to a weakness or flaw that an individual or group possesses.

[1199] A "cluster" is a group of multiple data points that share similar characteristics or features.

[1200] A "training plan" outlines specific procedures and guidelines for improving individual skills and abilities in order to achieve particular goals.

[1201] "Emotion recognition data" refers to information that identifies a user's emotional state by analyzing their facial expressions, tone of voice, behavior, etc.

[1202] "Dynamic adjustment" refers to a process that automatically changes content and settings in response to changes in circumstances or data.

[1203] "Display means" refers to tools and interfaces that present generated information and data to users in an easily understandable way.

[1204] The present invention provides a system to support the training of robot operators in factories. This system provides an interface in which users can input the skills, strengths, and weaknesses of operators, and also has the function of creating a personalized training plan that reflects the user's emotional state in real time by combining it with an emotion recognition engine.

[1205] Means for entering crew information

[1206] Users input information such as the operator's ID, strengths, weaknesses, and skills into the system through a dedicated form or user interface. This information is transmitted to and stored on the server via the client terminal.

[1207] Data analysis methods

[1208] The server analyzes the data based on the input operator information. Specifically, it converts the operator's skills into numerical vectors and performs clustering based on these vectors. It calculates feature vectors using the TF-IDF (Term Frequency-Inverse Document Frequency) method and classifies operators into groups with similar skill sets using the K-Means algorithm.

[1209] Means for generating clusters

[1210] The server uses the above analysis data to perform clustering and classify operators into appropriate clusters. This categorizes operators with other operators who have similar skill sets.

[1211] Means for creating a training plan

[1212] The server automatically generates training plans based on the clustering results. Each cluster has specific training goals and action plans, and individual training plans are generated based on the cluster to which the operator belongs. For example, operators belonging to cluster 0 will have goals such as "improvement of specific skills" and "practical project management skills."

[1213] Adjustment methods based on emotion recognition

[1214] The system uses the camera and microphone built into the device to analyze the user's facial expressions and voice tone, recognizing their emotions. This emotion recognition engine operates in real time, generating emotion data. The server receives this emotion data and stores it in a database.

[1215] Dynamic adjustment method based on emotional data

[1216] The server dynamically adjusts the content and progress of the training plan based on emotion recognition data. For example, if a user is experiencing stress, the plan is adjusted to alleviate that stress; conversely, if the user is showing positive emotions, the plan is adjusted to increase the number of challenging tasks.

[1217] Means of displaying training plans

[1218] The terminal displays the training plan generated and adjusted by the server for user review. This allows users to check and implement the specific training plan for each operator. The training plan is output in JSON format, making it easy to integrate with other systems and tools.

[1219] Hardware and software to be used

[1220] Camera and microphone: Used to recognize the operator's emotions.

[1221] Tkinter: A library for creating GUIs.

[1222] Requests: Used for sending data.

[1223] Scikit-learn: Used for TF-IDF vectorization and KMeans clustering.

[1224] EmotionEngine: A custom emotion recognition engine.

[1225] Specific example

[1226] Skill set input: Enter specific skills such as "Java, Python".

[1227] Emotion Recognition: If a user is experiencing stress, the training plan automatically adjusts to a "reduced training plan."

[1228] Example of a prompt

[1229] "We are developing an application to support training plans for robot operators in factories. Please implement a function that analyzes the operator's emotional state in real time and dynamically adjusts the training plan based on their skill set."

[1230] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1231] Step 1:

[1232] Users input information such as operator ID, strengths, weaknesses, and skills through a dedicated form or user interface. This input data is sent from the terminal to the server and stored in a database. The input data consists of ID, strengths, weaknesses, skill set, etc. The stored data is used for subsequent analysis processing.

[1233] Step 2:

[1234] The server analyzes the data based on the input operator information. In particular, it converts the operator's skill set into a numerical vector. Feature vectors are calculated using the TF-IDF (Term Frequency-Inverse Document Frequency) method and used as data for clustering. In this step, skill data is used as input, and feature vectors are generated as output.

[1235] Step 3:

[1236] The server performs clustering using the K-Means algorithm based on feature vectors. This classifies operators into the same cluster as other operators with similar skill sets. Feature vectors are used as input, and the clustering results are obtained as output. The clustering results are data indicating which cluster each operator belongs to.

[1237] Step 4:

[1238] The server automatically generates a training plan based on the clustering results. Each cluster has specific training goals and action plans pre-configured, and an appropriate training plan is created based on the clustering results. The clustering results are used as input, and a training plan is generated as output. The specific training goals and action plans generated include both short-term and long-term goals.

[1239] Step 5:

[1240] The device uses its built-in camera and microphone to analyze the user's facial expressions and voice tone in real time and recognize their emotions. An emotion recognition engine processes this data to identify the user's emotional state. Facial expression data and voice data are used as input, and recognized emotion data is generated as output.

[1241] Step 6:

[1242] The server receives emotion recognition data and stores it in a database. This emotion data is also used for dynamic adjustments to the development plan. Emotion data is used as input, and the output stored in the database is data indicating each emotional state.

[1243] Step 7:

[1244] The server dynamically adjusts the content and progress of the training plan based on emotional data. For example, if a user is experiencing stress, the training plan is adjusted to alleviate that stress; if they are showing positive emotions, challenging tasks are added. The input is emotional data and the existing training plan, and the output is the adjusted training plan.

[1245] Step 8:

[1246] The adjusted training plan is displayed to the user via a terminal. This allows the user to review and implement a specific training plan for each operator. The adjusted training plan is used as input, and the output is the training plan displayed on the terminal. The training plan is also output in JSON format, allowing for easy integration with other systems and tools.

[1247] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1248] The data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of the data generation model 58 is ChatGPT (Internet Search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1249] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1250] [Fourth Embodiment]

[1251] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1252] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1253] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1254] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1255] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1256] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1257] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1258] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1259] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1260] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1261] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1262] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1263] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1264] The present invention provides a system for automatically generating crew development plans, comprising means for inputting crew skills, strengths, and weaknesses, means for analysis, means for generating clusters, means for creating development plans, and means for displaying or outputting those plans. The system is implemented as follows:

[1265] 1. Means of entering crew information

[1266] Subject: User

[1267] Users input information such as crew IDs, strengths, weaknesses, and skills into the system. This is done through a dedicated form or user interface. This input information is sent to and stored on the server.

[1268] 2. Data Analysis Methods

[1269] Subject: Server

[1270] The server analyzes the data based on the entered crew information. In particular, it vectorizes the crew's skills and uses this to perform clustering. This process classifies crew members into groups with similar skill sets.

[1271] For example, if Crew A's skills are "Python, Management" and Crew B's skills are "Java, Hadoop," these skills are converted into numerical vectors, and clustered using feature vectors calculated with the TF-IDF (Term Frequency-Inverse Document Frequency) method.

[1272] 3. Means for generating clusters

[1273] Subject: Server

[1274] The server performs clustering using the previously analyzed data. It uses a clustering algorithm (e.g., K-Means) to group crew members with similar skills.

[1275] 4. Means of creating a training plan

[1276] Subject: Server

[1277] The server automatically generates training plans based on the clustering results. Since each cluster corresponds to a specific training goal, individual training plans are generated based on the cluster to which each crew member belongs.

[1278] Specifically, crews belonging to Cluster 0 are set with goals such as "improving Skill A" and "practicing project management," while crews belonging to Cluster 1 are set with goals such as "technical training in Skill B" and "promoting collaborative work."

[1279] 5. Means of displaying the training plan

[1280] Subject: terminal

[1281] The terminal displays the training plan generated by the server. This allows users to check and implement the specific training plan for each crew member. The training plan is output in JSON format, making it easy to integrate with other systems and tools.

[1282] For example, the following training plan will be generated:

[1283] Crew ID: C001

[1284] Short-term goals: Improve Skill A, practice project management.

[1285] Long-term goal: Leadership training

[1286] Duration: 6 months

[1287] Users can verify this and support the development of each crew member.

[1288] As described above, the system of the present invention can automatically create an efficient and effective crew training plan, thereby improving crew skills and operational efficiency.

[1289] The following describes the processing flow.

[1290] Step 1:

[1291] Subject: User

[1292] Users input crew information. Specifically, they enter crew IDs, strengths, weaknesses, skills, etc., through a dedicated form or user interface, and send this data to the server.

[1293] Step 2:

[1294] Subject: Server

[1295] The server stores the received crew information in a database. This includes fields for crew ID, strengths, weaknesses, and skills.

[1296] Step 3:

[1297] Subject: Server

[1298] The server vectorizes the saved crew skill information. This vectorization uses TfidfVectorizer to convert each crew member's skill into a numerical vector.

[1299] Step 4:

[1300] Subject: Server

[1301] The server performs clustering based on vectorized skill information. A clustering algorithm (e.g., K-Means) is used to group crews with similar skill sets.

[1302] Step 5:

[1303] Subject: Server

[1304] Based on the clustering results, the server determines the cluster to which each crew member belongs and stores this information in the database. Each crew member is assigned information about which cluster they belong to.

[1305] Step 6:

[1306] Subject: Server

[1307] The server automatically generates a growth plan based on each cluster. Specifically, it applies pre-configured growth goals and action plans for each cluster.

[1308] Step 7:

[1309] Subject: Server

[1310] The server converts the generated training plan into JSON format and saves it. This JSON data will be provided to the user in a later step.

[1311] Step 8:

[1312] Subject: terminal

[1313] The terminal sends a training plan request to the server. This is the procedure for users to check the training plans for each crew member.

[1314] Step 9:

[1315] Subject: Server

[1316] The server receives the request and sends the saved breeding plan in JSON format to the terminal.

[1317] Step 10:

[1318] Subject: terminal

[1319] The terminal displays the training plan received from the server on the screen. The user reviews this and takes specific actions to support the training of each crew member.

[1320] (Example 1)

[1321] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1322] The creation of conventional crew training plans has been difficult to implement efficiently and effectively due to the significant time and effort required to consider individual skills and characteristics. Furthermore, manual input and analysis can lead to errors and biases, making it challenging to design appropriate training plans for each crew member. This invention solves these problems by providing a system that efficiently inputs and analyzes crew information and automatically generates training plans.

[1323] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1324] In this invention, the server includes means for inputting information on the crew's skills, strengths, and weaknesses; means for vectorizing the crew data and generating feature vectors using the TF-IDF method; and means for dividing the crew into clusters using the feature vectors. This makes it possible to efficiently and automatically generate training plans for each crew member and output and display the training plans in JSON format.

[1325] "Crew skills" refer to the abilities and knowledge necessary to perform specific tasks or roles.

[1326] "Strengths" refer to the strengths and outstanding characteristics that the crew possesses.

[1327] "Weaknesses" refer to characteristics or shortcomings that the crew possesses that could be improved.

[1328] "Input method" refers to an interface for recording and transmitting information about crew members' skills, strengths, and weaknesses to the system.

[1329] "Vectorization" refers to the process of converting crew skill information into mathematical vectors and treating it as numerical data.

[1330] The "TF-IDF method" is a technique for evaluating the importance of words in text data, and is an abbreviation for Term Frequency-Inverse Document Frequency.

[1331] A "feature vector" is a numerical vector generated based on the crew's skills and is used as input data for clustering.

[1332] "Clustering" refers to the process of dividing data into groups based on similarity.

[1333] The "K-Means algorithm" refers to a clustering method for dividing data into K clusters.

[1334] A "development plan" refers to a document that includes goals and action plans for improving the crew's abilities and skills.

[1335] "JSON format" is an abbreviation for JavaScript Object Notation, and refers to a lightweight data format for structuring, storing, and transferring data.

[1336] A "database" refers to a system for efficiently storing, managing, and retrieving structured data.

[1337] "Means of display" refers to an interface for receiving crew training plans from the server and displaying them in a format viewable by the user.

[1338] The system of the present invention is for automatically generating crew training plans and includes the following means.

[1339] 1. Means of entering crew information

[1340] Subject: User

[1341] Users enter information such as crew IDs, strengths, weaknesses, and skills through a dedicated form or user interface. This involves users using a browser to input information into the UI and clicking a submit button. This input information is sent to the server and stored in a database.

[1342] 2. A method for vectorizing data and generating feature vectors using the TF-IDF method.

[1343] Subject: Server

[1344] The server analyzes crew data retrieved from the database. Specifically, it converts crew skill information into numerical vectors and generates feature vectors using the TF-IDF (Term Frequency-Inverse Document Frequency) method. These feature vectors are then used as input data for clustering.

[1345] 3. Methods for dividing the crew into clusters

[1346] Subject: Server

[1347] The server applies a clustering algorithm (e.g., K-Means) based on the generated feature vectors to group crews with similar skill sets. This automatically classifies crews into specific clusters.

[1348] 4. Means for automatically generating training plans

[1349] Subject: Server

[1350] The server automatically generates training plans for each crew member based on the clustering results. Each cluster has specific training goals, and individual training plans are generated according to the cluster to which each crew member belongs. Specifically, each crew member is given a detailed training plan (short-term goals, long-term goals, and duration).

[1351] 5. A means of outputting and displaying the training plan in JSON format.

[1352] Subject: terminal

[1353] The terminal displays the training plan generated by the server to the user. The training plan is sent from the server to the terminal in JSON format and displayed on the web interface. The user can check the training plan using a browser.

[1354] Hardware and software to be used

[1355] Server: Database system (e.g., MySQL), machine learning library (e.g., Scikit-learn)

[1356] Device: Web browser

[1357] Specific example

[1358] The user opens a web browser, accesses the specified URL, enters "Crew ID: C001, Skills: Python, Management, Strengths: Team Leadership, Weaknesses: Lack of Presentation Skills" into the input form, and clicks the "Submit" button. This information is sent to the server, vectorized using the TF-IDF method, and clustered using the K-Means algorithm. Finally, an automatically generated training plan is output in JSON format and displayed on the user's terminal.

[1359] Example of a prompt

[1360] Please enter the crew information: [Crew ID, Strengths, Weaknesses, Skills]

[1361] This will generate a training plan. Please click "Generate" once you have finished entering the information.

[1362] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1363] Step 1:

[1364] Subject: User

[1365] Users input and submit crew information (crew ID, strengths, weaknesses, skills) into the system through a dedicated form or user interface.

[1366] Input: Crew ID, Strengths, Weaknesses, Skills

[1367] Output: Crew information in JSON format is sent to the server.

[1368] Specific actions: The user opens a web browser and accesses the specified URL. They enter "Crew ID: C001, Skills: Python, Management, Strengths: Team leadership, Weaknesses: Lack of presentation skills" into the displayed input form and click the "Submit" button.

[1369] Step 2:

[1370] Subject: Server

[1371] The server receives crew information in JSON format sent by the user and stores it in the database.

[1372] Input: Crew information submitted by the user (in JSON format)

[1373] Output: Crew information stored in the database

[1374] Specific operation: The server receives an HTTP request sent by the user and parses the JSON data contained in its body. Then, it inserts the parsed data into a database (e.g., MySQL).

[1375] Step 3:

[1376] Subject: Server

[1377] The server retrieves crew information from the database and vectorizes the skill information. Feature vectors are generated using the TF-IDF method.

[1378] Input: Crew skill information retrieved from the database

[1379] Output: Vectorized skill information (feature vectors)

[1380] Specific operation: The server executes a script (e.g., a Python program) to retrieve skill information from the database. This skill information is converted into a numerical vector, and TF-IDF calculations are performed.

[1381] Step 4:

[1382] Subject: Server

[1383] The server applies the K-Means algorithm based on the generated feature vectors to classify the crew into clusters.

[1384] Input: Feature vector

[1385] Output: Cluster ID for each crew

[1386] Specific operation: The server uses a machine learning library (e.g., Scikit-learn) to perform K-Means clustering on the feature vectors. The result of the calculation is the cluster ID for each crew.

[1387] Step 5:

[1388] Subject: Server

[1389] The server automatically generates training plans for each crew member based on the clustering results. Each cluster has set training goals, which are used to create individual plans.

[1390] Input: Cluster ID and growth objectives

[1391] Output: Training plan for each crew member (JSON format)

[1392] Specific operation: Based on the clustering results, the server retrieves training objectives corresponding to each cluster. Next, it generates specific training plans (short-term objectives, long-term objectives, and duration) for each crew member.

[1393] Step 6:

[1394] Subject: terminal

[1395] The terminal displays the training plan generated by the server to the user. The training plan is sent from the server in JSON format and displayed via a web interface.

[1396] Input: Training plan (JSON format) sent from the server

[1397] Output: Displayed training plan

[1398] Specific operation: The user reopens their web browser and accesses the training plan display page. The browser retrieves JSON data from the server and displays it in a user-friendly format (e.g., an HTML table).

[1399] Example: Crew ID: C001, Short-term goal: Improve Skill A, practice project management, Long-term goal: Leadership training, Duration: 6 months

[1400] (Application Example 1)

[1401] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1402] The problem lies in the lack of a system that effectively and efficiently manages the skill sets and training plans of crew members working in the factory, and provides optimal training plans for each crew member in real time. Traditional manual management and planning are time-consuming and labor-intensive, and it was difficult to create optimal training plans tailored to the individual skills of each crew member. In addition, there was a problem in that training plans became mere formalities and did not lead to actual work efficiency improvements or skill development.

[1403] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1404] In this invention, the server includes means for inputting information on the skills, strengths, and weaknesses of crew members, means for dividing the crew into clusters based on the input information, and means for automatically generating a training plan for each crew member based on the clusters. This makes it possible to effectively manage the skill sets of crew members and provide optimal training plans in real time. Furthermore, by providing the training plans in JSON format and supplying them to factory robots, and by providing means for allowing crew members to check their training plans via tablet terminals, it becomes possible to propose crew placement and training plans in real time.

[1405] "Crew" refers to the workers and staff who work in a factory.

[1406] "Skills" is a general term for the specialized knowledge and techniques that a crew member possesses.

[1407] "Strengths" refers to the distinctive strengths and advantages of each crew member.

[1408] "Weaknesses" refer to the crew's shortcomings or areas that need improvement.

[1409] "Means of inputting information" refers to interfaces or devices used to input data on crew members' skills, strengths, and weaknesses.

[1410] "Methods for dividing crews into clusters" refers to algorithms or processes that group similar crews together based on their skills and characteristics.

[1411] "Means for automatically generating training plans" refers to systems or programs that automatically create training goals and methods for crew members in each cluster.

[1412] "Means for displaying training plans" refers to devices or software that visually display automatically generated training plans.

[1413] "Method for outputting in JSON format" refers to the function that exports the training plan in JSON (JavaScript Object Notation) format.

[1414] "Factory robots" refer to robots that assist crew members or perform automated tasks within a factory.

[1415] "A means of proposing crew placement and training plans in real time" refers to a system or algorithm in which factory robots instantly propose the optimal placement and training plan based on the current status and skills of the crew.

[1416] A "tablet device" refers to a portable computer device used to review training plans.

[1417] The system for realizing this application generates, manages, and implements effective crew training plans through an application installed on factory robots. The system consists of multiple hardware and software components.

[1418] Hardware and software to be used:

[1419] Hardware: Factory robots, servers, tablet devices

[1420] Software: Python (for data analysis and clustering), Django (web interface), MySQL (database management)

[1421] System processing procedures and functions:

[1422] 1. Enter and save crew information:

[1423] Users enter information about crew members, including their ID, strengths, weaknesses, and skills, via a tablet device. This information is sent to the server through a dedicated form or user interface and stored in a MySQL database.

[1424] 2. Data Analysis and Clustering:

[1425] The server uses Python to analyze the input crew information. Specifically, it uses the TF-IDF (Term Frequency-Inverse Document Frequency) method to create numerical vectors of crew skills and then performs clustering using the K-Means clustering algorithm.

[1426] 3. Automatic generation of training plans:

[1427] The server automatically generates a growth plan for each cluster based on the clustering results. The growth plan includes short-term and long-term goals and is saved in JSON format.

[1428] 4. Display and implementation of the training plan:

[1429] The factory robot receives the generated training plan in real time and displays it via a tablet device. Users can review the displayed training plan and implement specific training plans for their crew. The robot also handles crew placement and provides real-time training plan suggestions.

[1430] Specific example:

[1431] For example, let's say there are crew members A, B, and C working on a factory assembly line.

[1432] Crew A's skills: "Mechanical Engineering, Basic Programming"

[1433] Crew B's skills: "Basic Electrical Engineering, Applied Programming"

[1434] Crew C's skills: "Mechanical maintenance, Electrical maintenance"

[1435] The system converts these skills into numerical vectors and performs clustering. As a result, for example, Crew A and Crew B are classified into the same cluster, and training plans are set for this cluster: "Improvement of programming skills" and "Basic training in machine maintenance." A different training plan is generated for Crew C.

[1436] Examples of prompt statements for generated AI models:

[1437] Based on the crew skill information provided below, generate an optimal training plan. Crew skills have been numerically vectorized using the TF-IDF method, and K-Means is used for clustering. Propose specific training goals for each cluster.

[1438] Crew information:

[1439] 1. Crew ID: C001, Skills: "Mechanical Engineering, Basic Programming"

[1440] 2. Crew ID: C002, Skills: "Basic Electrical Engineering, Applied Programming"

[1441] 3. Crew ID: C003, Skills: "Machine Maintenance, Electrical Maintenance"

[1442] "

[1443] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1444] Step 1:

[1445] Users input crew information using a tablet device. Specifically, they enter the crew's ID, strengths, weaknesses, and skills into a dedicated input form and send that data to the server. The entered data is stored in a MySQL database on the server.

[1446] Input: Crew ID, strengths, weaknesses, skills

[1447] Output: Crew information stored in the database on the server

[1448] Step 2:

[1449] The server performs data analysis based on the stored crew information. Specifically, it uses Python to numerically vectorize crew skills using the TF-IDF (Term Frequency-Inverse Document Frequency) method. This quantifies each crew member's skills as feature vectors.

[1450] Input: Crew skill information stored in the database

[1451] Output: Numerically vectorized crew skill data

[1452] Step 3:

[1453] The server performs clustering based on vectorized skill data. Using the K-Means clustering algorithm, crews with similar skills are divided into clusters. The clustering results are stored in the database as the cluster ID for each crew member.

[1454] Input: Numerically vectorized crew skill data

[1455] Output: Crew information with assigned cluster IDs

[1456] Step 4:

[1457] The server automatically generates training plans based on the clustering results. Specifically, it assigns predefined training objectives (short-term and long-term goals) to each cluster and creates training plans for each crew member. The generated training plans are stored in the database in JSON format.

[1458] Input: Crew information with a cluster ID assigned.

[1459] Output: Training plan in JSON format

[1460] Step 5:

[1461] Factory robots receive training plans transmitted from a server in real time and display the crew training plans via tablet devices. This allows users to review and implement training plans for each crew member. The robots also suggest crew assignments and training plans.

[1462] Input: Training plan in JSON format

[1463] Output: Training plan displayed on the tablet device

[1464] Through these steps, the system will be able to efficiently manage crew skill information and automatically generate and provide optimal training plans in real time.

[1465] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1466] The present invention provides a system for automatically generating crew training plans, which combines means for inputting crew skills, strengths, and weaknesses, means for analyzing them, means for generating clusters, means for creating training plans, and means for displaying or outputting those plans, with an emotion engine that recognizes user emotions. This system is implemented as follows:

[1467] 1. Means of entering crew information

[1468] Subject: User

[1469] Users input information such as crew IDs, strengths, weaknesses, and skills into the system. This is done through a dedicated form or user interface. This input information is sent to and stored on the server.

[1470] 2. Data Analysis Methods

[1471] Subject: Server

[1472] The server analyzes the data based on the entered crew information. In particular, it vectorizes the crew's skills and uses this to perform clustering. This process classifies crew members into groups with similar skill sets.

[1473] For example, if Crew A's skills are "Python, Management" and Crew B's skills are "Java, Hadoop," these skills are converted into numerical vectors, and clustered using feature vectors calculated with the TF-IDF (Term Frequency-Inverse Document Frequency) method.

[1474] 3. Means for generating clusters

[1475] Subject: Server

[1476] The server performs clustering using the previously analyzed data. It uses a clustering algorithm (e.g., K-Means) to group crew members with similar skills.

[1477] 4. Means of creating a training plan

[1478] Subject: Server

[1479] The server automatically generates training plans based on the clustering results. Each cluster has specific training goals and action plans set, and individual training plans are generated based on the cluster to which each crew member belongs.

[1480] For example, crews belonging to Cluster 0 are set with goals such as "improving Skill A" and "practicing project management," while crews belonging to Cluster 1 are set with goals such as "technical training in Skill B" and "promoting collaborative work."

[1481] 5. Emotion recognition means using an emotion engine

[1482] Subject: terminal

[1483] Using the camera and microphone built into the device, the system analyzes the user's facial expressions and voice tone in real time to recognize the user's emotions. The emotion engine processes this data and generates emotion data.

[1484] 6. Methods for collecting emotional data

[1485] Subject: Server

[1486] The server receives emotional data transmitted from the emotion engine and stores it in a database. This data is used to adapt the crew's training plan.

[1487] 7. Means of adjusting development plans to reflect emotions

[1488] Subject: Server

[1489] The server dynamically adjusts the content and progress of the training plan based on the collected user emotion data. For example, if a user is experiencing stress, the plan is adjusted to reduce stress; conversely, if the user is showing positive emotions, the plan is adjusted to increase the number of challenging tasks.

[1490] 8. Means of displaying the training plan

[1491] Subject: terminal

[1492] The terminal displays the training plan generated and adjusted by the server. This allows users to review and implement the specific training plan for each crew member. The training plan is output in JSON format, making it easy to integrate with other systems and tools.

[1493] For example, the following training plan will be generated:

[1494] Crew ID: C001

[1495] Short-term goals: Improve Skill A, practice project management.

[1496] Long-term goal: Leadership training

[1497] Duration: 6 months

[1498] Users can verify this and support the development of each crew member.

[1499] The system of the present invention can automatically create efficient and effective crew training plans and, by taking into account the user's feelings, can achieve more personalized training support.

[1500] The following describes the processing flow.

[1501] Step 1:

[1502] Subject: User

[1503] Users input crew information. Specifically, they enter information such as crew ID, strengths, weaknesses, and skills through a dedicated form or user interface, and send this data to the server.

[1504] Step 2:

[1505] Subject: Server

[1506] The server stores the received crew information in a database. The database contains fields for Crew ID, Strengths, Weaknesses, and Skills.

[1507] Step 3:

[1508] Subject: Server

[1509] The server retrieves crew skill information from the database and vectorizes it. It uses TfidfVectorizer to convert skills into numerical vectors.

[1510] Step 4:

[1511] Subject: Server

[1512] Clustering is performed based on vectorized skills. A clustering algorithm (e.g., K-Means) is used to group crews with similar skill sets.

[1513] Step 5:

[1514] Subject: Server

[1515] The clustering results are analyzed to determine the cluster to which each crew member belongs. The determined cluster information is added to the crew member's data and saved to the database.

[1516] Step 6:

[1517] Subject: Server

[1518] The system automatically generates training plans based on clusters. It applies specific training goals and action plans set for each cluster to generate individual training plans.

[1519] Step 7:

[1520] Subject: terminal

[1521] The device analyzes the user's facial expressions and tone of voice in real time to recognize the user's emotions. It then uses an emotion engine to generate emotion data.

[1522] Step 8:

[1523] Subject: Server

[1524] The server receives emotional data sent from the terminals and stores it in a database. This emotional data is then used to inform the crew's training plans.

[1525] Step 9:

[1526] Subject: Server

[1527] The server analyzes emotional data and dynamically adjusts the training plan based on the collected user emotions. For example, if it determines that the user is experiencing stress, it adjusts the training plan to reduce the workload.

[1528] Step 10:

[1529] Subject: Server

[1530] The adjusted training plan is output in JSON format and saved back to the database. This format facilitates data integration with other systems.

[1531] Step 11:

[1532] Subject: terminal

[1533] The terminal sends a request for a training plan to the server and receives a customized training plan from the server.

[1534] Step 12:

[1535] Subject: terminal

[1536] The terminal displays the received training plan on its screen. The user reviews the displayed training plan and takes specific actions to support the training of each crew member.

[1537] (Example 2)

[1538] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1539] Traditional crew training systems typically provided uniform training plans without considering the emotional state of each individual crew member. This led to problems such as crew members experiencing stress or, conversely, a lack of stimulation resulting in decreased motivation. Furthermore, the manual generation of training plans was inefficient and time-consuming. Additionally, data format issues arose when integrating the generated training plans with other systems, resulting in low versatility.

[1540] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1541] In this invention, the server includes means for inputting information on the skills, strengths, and weaknesses of crew members; means for dividing the crew into clusters based on the input information; means for automatically generating a training plan for each crew member based on the clusters; means for dynamically adjusting the training plan using an emotion engine that recognizes the user's emotions; and means for displaying the training plan. This makes it possible to provide more personalized training plans that take into account the individual emotional state of each crew member, thereby resolving issues of crew stress and decreased motivation. Furthermore, since the training plan is output in JSON format, it is easy to link with other systems, improving versatility.

[1542] "Crew" refers to individual members who are engaged in a specific task or project.

[1543] "Skills" refer to the knowledge and abilities that a crew member possesses in order to perform a specific job or task.

[1544] "Strengths" refers to the individual strengths and outstanding characteristics of each crew member.

[1545] "Weaknesses" refer to the individual weaknesses or areas for improvement of each crew member.

[1546] "Means of input" refers to methods of entering data into a system through user interfaces or forms.

[1547] A "cluster" refers to a group of crew members who share similar skills or characteristics.

[1548] "Clustering" refers to the process of classifying crews based on their similarity using data analysis techniques.

[1549] A "training plan" refers to specific goals and action plans aimed at improving the skills and characteristics of the crew.

[1550] An "emotion engine" refers to a combination of hardware and software used to recognize and analyze a user's emotional state.

[1551] "Dynamic adjustment methods" refer to methods of adapting the plan based on real-time sentiment data.

[1552] "JSON format" refers to JavaScript Object Notation, a lightweight data format for data exchange.

[1553] "Means of display" refers to methods that allow users to visually confirm their training plan.

[1554] System Overview

[1555] The present invention relates to a system for automatically generating crew training plans. This system includes means for inputting crew skills, strengths, and weaknesses, means for analyzing data, means for generating clusters, means for creating training plans, and means for displaying or outputting those plans, and further incorporates an emotion engine that recognizes user emotions.

[1556] Hardware and software to be used

[1557] Hardware: Server, terminal (with camera and microphone), network connection

[1558] Software: Database management system, clustering algorithms (e.g., K-Means), Python libraries (scikit-learn, OpenCV), sentiment analysis engine, display interface, JSON output function.

[1559] Specific operation of the system

[1560] 1. Enter crew information

[1561] Users input information such as crew IDs, strengths, weaknesses, and skills into the system. This information is collected through dedicated forms and user interfaces. For example, a user might use a web browser to enter information such as "Crew ID: C001," "Strengths: Honesty," "Weaknesses: Communication," and "Skills: Java, Spring framework," and then click the submit button.

[1562] 2. Saving input data

[1563] The server receives information sent by the user and stores it in the database. For example, it can use SQL queries to store data in the database.

[1564] 3. Data Analysis

[1565] The server retrieves crew information from the database and converts the crew's skills into text vectors. Specifically, it vectorizes the data using the TF-IDF method and performs clustering using the K-Means algorithm. For example, the scikit-learn library in Python is used to convert the skill data into numerical vectors and then perform clustering.

[1566] 4. Cluster generation

[1567] The server classifies crew members into groups (clusters) with similar skill sets based on the clustering results. The generated clustering results are stored in a database.

[1568] 5. Automatic generation of training plans

[1569] The server automatically generates a training plan tailored to each cluster based on the clustering results. For example, it might create a plan for cluster 0 that includes "improving Skill A" and "practicing project management." This plan is then stored in the database.

[1570] 6. Emotion recognition by an emotion engine

[1571] When reviewing the training plan, the device's camera and microphone are used to recognize the user's emotions in real time. The emotion engine analyzes the user's facial expressions and voice tone to generate emotion data. For example, OpenCV is used to analyze facial expressions and estimate emotional states.

[1572] 7. Collection of emotional data

[1573] The server receives emotion data from the emotion engine and stores it in a database. For example, if a user is feeling stressed, that emotion data is sent to the server.

[1574] 8. Dynamic adjustment of the training plan

[1575] The server dynamically adjusts the content of the training plan based on the collected emotional data. For example, if stress levels are high, the training plan's workload is reduced, and if motivation is high, challenging goals are added.

[1576] 9. Display of the training plan

[1577] The finalized training plan is sent to the device and displayed for the user to review. Since the training plan is output in JSON format, it can be easily integrated with other systems.

[1578] Example of a prompt

[1579] Please enter the information for Crew C001 (ID, Strengths: Honesty, Weaknesses: Communication, Skills: Java, Spring framework) and generate a training plan.

[1580] This system allows users to easily generate personalized training plans for each crew member, taking emotional data into consideration to support more effective training.

[1581] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1582] Step 1: The user enters the crew information.

[1583] The user accesses a dedicated form on the system. They enter information such as the crew member's ID, strengths, weaknesses, and skills into the form and click the submit button. This input information is sent to the server. Specifically, the user enters "Crew ID: C001", "Strengths: Honesty", "Weaknesses: Communication", and "Skills: Java, Spring framework" into the form using a web browser. Input data: Crew member ID / Strengths / Weaknesses / Skills. Output data: Once the user completes the input and clicks the submit button, the user's input information is sent to the server.

[1584] Step 2: The server saves the input data.

[1585] The server receives information sent by the user and stores it in the database. Based on the received data, the server uses SQL queries to save information to the database. Specifically, the following SQL query is executed: INSERT INTO crew_data (ID, strengths, weaknesses, skills) VALUES ('C001', 'Sincerity', 'Communication', 'Java, Spring framework'). Input data: Crew information sent by the user. Output data: Crew information saved by the server in the database.

[1586] Step 3: The server analyzes the data.

[1587] The server retrieves crew information from the database and converts the crew's skills into text vectors. The server performs vectorization using the TF-IDF method and then clustering. Specifically, it uses the Python scikit-learn library to convert the crew's skills into numerical vectors and performs clustering using the KMeans(n_clusters=3) algorithm. Input data: Crew information stored in the database. Output data: Vectorized skills and clustering results.

[1588] Step 4: The server generates the cluster.

[1589] The server classifies the crew into groups based on the clustering results. The classification results are then saved back to the database. Specifically, assuming crew C001 belongs to cluster 1, the SQL query INSERT INTO cluster_data (crew_id, cluster_id) VALUES ('C001', 1) is executed. Input data: Clustering results. Output data: Cluster information saved in the database.

[1590] Step 5: The server automatically generates a training plan.

[1591] The server automatically generates a training plan suitable for each cluster based on the clustering results. Specifically, tasks such as "Improving Skill A" and "Practicing Project Management" are assigned to Cluster 1. The generated training plans are saved in the database. Input data: Clustering results. Output data: Training plans.

[1592] Step 6: The device recognizes the user's emotions.

[1593] While the training plan is being reviewed, the device's built-in camera and microphone are used to recognize the user's emotions in real time. The device's emotion engine analyzes the user's facial expressions and voice tone to generate emotion data. Specifically, it uses OpenCV to analyze facial expressions and estimate the emotional state. Input data: User's facial expressions and voice captured by the camera and microphone. Output data: Generated emotion data.

[1594] Step 7: The server collects emotion data.

[1595] The server receives emotion data sent from the emotion engine and stores it in the database. Specifically, an SQL query, INSERT INTO emotion_data (user_id, stress_level) VALUES ('U001', 70), is executed based on the emotion data. Input data: Emotion data sent from the terminal. Output data: Emotion data stored in the database.

[1596] Step 8: The server dynamically adjusts the growth plan.

[1597] The server dynamically adjusts the training plan based on the collected emotional data. Specifically, if the stress level is high, the workload of the plan is reduced, and conversely, if motivation is high, challenging goals are added. For example, the following SQL query is executed: UPDATE training_plans SET adjustments = 'Load Reduction' WHERE user_id = 'U001' AND stress_level > 60. Input data: Emotional data stored in the database. Output data: Adjusted training plan.

[1598] Step 9: The device displays the training plan.

[1599] The terminal receives the training plan prepared by the server and displays it on the user interface. For example, for crew member C001, information such as the short-term goal "Improve Skill A", the long-term goal "Leadership Training", and the duration "6 months" is displayed. Input data: Prepared training plan. Output data: Training plan displayed to the user.

[1600] (Application Example 2)

[1601] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1602] In conventional systems, crew training plans are automatically generated based on skills and characteristics, but they fail to take into account specific situations and emotional aspects, resulting in insufficient personalization for each crew member. In particular, in the training of robot operators within factories, plans that ignore individual emotions and stress levels hinder efficient training. Therefore, dynamic adjustments that respond to the user's emotional state are necessary to enhance the effectiveness of training plans.

[1603] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1604] In this invention, the server includes means for inputting information on the skills, strengths, and weaknesses of crew members; means for dividing the crew into clusters based on the input information; means for automatically generating a training plan for each crew member based on the clusters; means for dynamically adjusting the plan using emotion recognition data when creating the plan; and means for displaying the training plan. This makes it possible to generate flexible training plans that take into account the emotional state of the user.

[1605] A "crew" refers to a group of people assembled to perform a specific task or job.

[1606] "Skills" refer to the specialized abilities and techniques necessary to perform a particular task or job.

[1607] "Strengths" refer to the outstanding characteristics or advantages that an individual or group possesses.

[1608] A "weakness" refers to a weakness or flaw that an individual or group possesses.

[1609] A "cluster" is a group of multiple data points that share similar characteristics or features.

[1610] A "training plan" outlines specific procedures and guidelines for improving individual skills and abilities in order to achieve particular goals.

[1611] "Emotion recognition data" refers to information that identifies a user's emotional state by analyzing their facial expressions, tone of voice, behavior, etc.

[1612] "Dynamic adjustment" refers to a process that automatically changes content and settings in response to changes in circumstances or data.

[1613] "Display means" refers to tools and interfaces that present generated information and data to users in an easily understandable way.

[1614] The present invention provides a system to support the training of robot operators in factories. This system provides an interface in which users can input the skills, strengths, and weaknesses of operators, and also has the function of creating a personalized training plan that reflects the user's emotional state in real time by combining it with an emotion recognition engine.

[1615] Means for entering crew information

[1616] Users input information such as the operator's ID, strengths, weaknesses, and skills into the system through a dedicated form or user interface. This information is transmitted to and stored on the server via the client terminal.

[1617] Data analysis methods

[1618] The server analyzes the data based on the input operator information. Specifically, it converts the operator's skills into numerical vectors and performs clustering based on these vectors. It calculates feature vectors using the TF-IDF (Term Frequency-Inverse Document Frequency) method and classifies operators into groups with similar skill sets using the K-Means algorithm.

[1619] Means for generating clusters

[1620] The server uses the above analysis data to perform clustering and classify operators into appropriate clusters. This categorizes operators with other operators who have similar skill sets.

[1621] Means for creating a training plan

[1622] The server automatically generates training plans based on the clustering results. Each cluster has specific training goals and action plans, and individual training plans are generated based on the cluster to which the operator belongs. For example, operators belonging to cluster 0 will have goals such as "improvement of specific skills" and "practical project management skills."

[1623] Adjustment methods based on emotion recognition

[1624] The system uses the camera and microphone built into the device to analyze the user's facial expressions and voice tone, recognizing their emotions. This emotion recognition engine operates in real time, generating emotion data. The server receives this emotion data and stores it in a database.

[1625] Dynamic adjustment method based on emotional data

[1626] The server dynamically adjusts the content and progress of the training plan based on emotion recognition data. For example, if a user is experiencing stress, the plan is adjusted to alleviate that stress; conversely, if the user is showing positive emotions, the plan is adjusted to increase the number of challenging tasks.

[1627] Means of displaying training plans

[1628] The terminal displays the training plan generated and adjusted by the server for user review. This allows users to check and implement the specific training plan for each operator. The training plan is output in JSON format, making it easy to integrate with other systems and tools.

[1629] Hardware and software to be used

[1630] Camera and microphone: Used to recognize the operator's emotions.

[1631] Tkinter: A library for creating GUIs.

[1632] Requests: Used for sending data.

[1633] Scikit-learn: Used for TF-IDF vectorization and KMeans clustering.

[1634] EmotionEngine: A custom emotion recognition engine.

[1635] Specific example

[1636] Skill set input: Enter specific skills such as "Java, Python".

[1637] Emotion Recognition: If a user is experiencing stress, the training plan automatically adjusts to a "reduced training plan."

[1638] Example of a prompt

[1639] "We are developing an application to support training plans for robot operators in factories. Please implement a function that analyzes the operator's emotional state in real time and dynamically adjusts the training plan based on their skill set."

[1640] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1641] Step 1:

[1642] Users input information such as operator ID, strengths, weaknesses, and skills through a dedicated form or user interface. This input data is sent from the terminal to the server and stored in a database. The input data consists of ID, strengths, weaknesses, skill set, etc. The stored data is used for subsequent analysis processing.

[1643] Step 2:

[1644] The server analyzes the data based on the input operator information. In particular, it converts the operator's skill set into a numerical vector. Feature vectors are calculated using the TF-IDF (Term Frequency-Inverse Document Frequency) method and used as data for clustering. In this step, skill data is used as input, and feature vectors are generated as output.

[1645] Step 3:

[1646] The server performs clustering using the K-Means algorithm based on feature vectors. This classifies operators into the same cluster as other operators with similar skill sets. Feature vectors are used as input, and the clustering results are obtained as output. The clustering results are data indicating which cluster each operator belongs to.

[1647] Step 4:

[1648] The server automatically generates a training plan based on the clustering results. Each cluster has specific training goals and action plans pre-configured, and an appropriate training plan is created based on the clustering results. The clustering results are used as input, and a training plan is generated as output. The specific training goals and action plans generated include both short-term and long-term goals.

[1649] Step 5:

[1650] The device uses its built-in camera and microphone to analyze the user's facial expressions and voice tone in real time and recognize their emotions. An emotion recognition engine processes this data to identify the user's emotional state. Facial expression data and voice data are used as input, and recognized emotion data is generated as output.

[1651] Step 6:

[1652] The server receives emotion recognition data and stores it in a database. This emotion data is also used for dynamic adjustments to the development plan. Emotion data is used as input, and the output stored in the database is data indicating each emotional state.

[1653] Step 7:

[1654] The server dynamically adjusts the content and progress of the training plan based on emotional data. For example, if a user is experiencing stress, the training plan is adjusted to alleviate that stress; if they are showing positive emotions, challenging tasks are added. The input is emotional data and the existing training plan, and the output is the adjusted training plan.

[1655] Step 8:

[1656] The adjusted training plan is displayed to the user via a terminal. This allows the user to review and implement a specific training plan for each operator. The adjusted training plan is used as input, and the output is the training plan displayed on the terminal. The training plan is also output in JSON format, allowing for easy integration with other systems and tools.

[1657] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1658] The data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of the data generation model 58 is ChatGPT (Internet Search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1659] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[1660] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1661] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[1662] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[1663] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[1664] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[1665] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[1666] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[1667] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[1668] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[1669] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[1670] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1671] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[1672] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[1673] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[1674] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[1675] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[1676] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[1677] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[1678] The following is further disclosed regarding the embodiments described above.

[1679] (Claim 1)

[1680] A means of inputting information about the crew's skills, strengths, and weaknesses,

[1681] A means of dividing the crew into clusters based on the input information,

[1682] A means for automatically generating a training plan for each crew member based on the aforementioned cluster,

[1683] The means for displaying the aforementioned training plan,

[1684] A system that includes this.

[1685] (Claim 2)

[1686] The system according to claim 1, comprising means for vectorizing and clustering the skills of the crew.

[1687] (Claim 3)

[1688] The system according to claim 1, further comprising means for outputting the training plan in JSON format.

[1689] "Example 1"

[1690] (Claim 1)

[1691] A means of inputting information about the crew's skills, strengths, and weaknesses,

[1692] A means for vectorizing crew data based on the input information and generating feature vectors using the TF-IDF method,

[1693] A means of dividing the crew into clusters using the aforementioned feature vector,

[1694] A means for automatically generating a training plan for each crew member based on the aforementioned cluster,

[1695] A means for outputting and displaying the aforementioned training plan in JSON format,

[1696] A system that includes this.

[1697] (Claim 2)

[1698] The system according to claim 1, further comprising means of using the K-Means algorithm for the clustering.

[1699] (Claim 3)

[1700] The system according to claim 1, further comprising means for storing the crew's input information in a database.

[1701] "Application Example 1"

[1702] (Claim 1)

[1703] A means of inputting information about the crew's skills, strengths, and weaknesses,

[1704] A means of dividing the crew into clusters based on the input information,

[1705] A means for automatically generating a training plan for each crew member based on the aforementioned cluster,

[1706] The means for displaying the aforementioned training plan,

[1707] A means for outputting the training plan in JSON format,

[1708] A means for providing the aforementioned training plan to factory robots and proposing crew assignment and training plans in real time,

[1709] A system that includes this.

[1710] (Claim 2)

[1711] The system according to claim 1, comprising means for vectorizing and clustering the skills of the crew.

[1712] (Claim 3)

[1713] The system according to claim 1, wherein the factory robot includes means for checking the crew training plan via a tablet device.

[1714] "Example 2 of combining an emotion engine"

[1715] (Claim 1)

[1716] A means of inputting information about the crew's skills, strengths, and weaknesses,

[1717] A means of dividing the crew into clusters based on the input information,

[1718] A means for automatically generating a training plan for each crew member based on the aforementioned cluster,

[1719] A means of dynamically adjusting the training plan using an emotion engine that recognizes the user's emotions,

[1720] The means for displaying the aforementioned training plan,

[1721] A system that includes this.

[1722] (Claim 2)

[1723] The system according to claim 1, comprising means for vectorizing and clustering the skills of the crew.

[1724] (Claim 3)

[1725] The system according to claim 1, further comprising means for outputting the training plan in JSON format.

[1726] "Application example 2 when combining with an emotional engine"

[1727] (Claim 1)

[1728] A means of inputting information about the crew's skills, strengths, and weaknesses,

[1729] A means of dividing the crew into clusters based on the input information,

[1730] A means for automatically generating a training plan for each crew member based on the aforementioned cluster,

[1731] A means for dynamically adjusting the plan using emotion recognition data when creating the aforementioned plan,

[1732] The means for displaying the aforementioned training plan,

[1733] A system that includes this.

[1734] (Claim 2)

[1735] The system according to claim 1, comprising means for vectorizing and clustering the skills of the crew.

[1736] (Claim 3)

[1737] The system according to claim 1, further comprising means for outputting the training plan in JSON format. [Explanation of symbols]

[1738] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of inputting information about the crew's skills, strengths, and weaknesses, A means of dividing the crew into clusters based on the input information, A means for automatically generating a training plan for each crew member based on the aforementioned cluster, The means for displaying the aforementioned training plan, A system that includes this.

2. The system according to claim 1, comprising means for vectorizing and clustering the skills of the crew.

3. The system according to claim 1, further comprising means for outputting the training plan in JSON format.

Images