System

Abstract

A system is provided.SOLUTION: A system comprising: means for accessing a work process database and managing each work step; wearable terminal means for outputting instructions related to a work process by visual display and voice; AI module means for monitoring an operation of a worker in real time and providing feedback via the wearable terminal means; and evaluation means for recording a completion time of each work step and comparing the completion time with a standard work time to evaluate a skill level of the worker.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Typically, when performing a new task or complex process for the first time, an experienced worker or manager must accompany the worker, which doubles the number of personnel required and significantly reduces productivity. Other issues include the occurrence of incidents due to incorrect work procedures and the lack of efficient means to support each worker in improving their skills. The objective of this invention is to provide a navigation system that allows workers to perform new tasks with confidence, thereby preventing incidents, promoting worker skill development, and improving overall productivity. [Means for solving the problem]

[0005] The present invention solves the problems by the following means.

[0006] The system includes a means for accessing a work process database and managing each work step, a wearable terminal means for visually and audibly outputting instructions related to the work process, an AI module means for monitoring the worker's operations in real time and providing feedback via the wearable terminal means, a control means for issuing visual and audio warnings and preventing the worker from proceeding to the next step if an erroneous operation is detected, and an evaluation means for recording the completion time of each work step and comparing it with the standard work time to evaluate the worker's skill level. It also includes a means for providing support proposals based on the worker's skill level via the wearable terminal to help improve the worker's skills, and a means for receiving the worker's authentication information and providing customized work procedures based on the worker's profile. This allows workers to perform new tasks with peace of mind, and is expected to improve productivity by preventing erroneous operations and improving skills.

[0007] The "work process database" is a digital database for storing and managing data such as detailed information on each work step, standard work time, and error conditions.

[0008] A "wearable terminal" is a terminal that has the function of visually displaying and audibly outputting work instructions when worn by a user, and in the present invention, audio glasses are an example of this.

[0009] The "AI module" is an artificial intelligence system that monitors and analyzes worker operations in real time, generating feedback and warnings.

[0010] The "control means" is the control part of the system that issues a warning when an incorrect operation is detected and prevents the process from proceeding to the next step.

[0011] The "evaluation means" is a system element that has the function of recording the completion time of each work step and comparing it with the standard work time to evaluate the skill level of the worker.

[0012] A "worker profile" is personal data that includes each worker's authentication information, skill level, and individual work history.

[0013] "Support proposals" are information and plans for providing advice and guidance tailored to individual skill levels with the aim of improving worker skills. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0015] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0018] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0019] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0020] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0022] [First embodiment]

[0023] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0024] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0027] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0030] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0034] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0035] MODE FOR CARRYING OUT THE INVENTION

[0036] The present invention is a system that utilizes an AI module and a wearable device to navigate workers in real time and improve productivity. Specific embodiments and program processing of the system are described below.

[0037] System Configuration

[0038] The system consists of the following main components:

[0039] 1. Server

[0040] Access to the work process database

[0041] Managing work steps

[0042] Analysis of work monitoring data

[0043] Generate feedback and warnings

[0044] Manage worker profiles

[0045] 2. Wearable devices

[0046] Visual display of work instructions (e.g. audio glasses)

[0047] Voice output of work instructions

[0048] User authentication function

[0049] Work monitoring using cameras and sensors

[0050] 3. Users

[0051] Check and implement work instructions

[0052] Receiving and responding to feedback

[0053] Implementing correction work

[0054] Program processing overview

[0055] Explain how the components of the system work together.

[0056] 1. Initial Server Setup

[0057] When the server starts up, it reads the work process database and worker profile data.

[0058] The work procedure data includes work steps, standard work times, and error conditions.

[0059] 2. User Authentication

[0060] When the device boots up, the user enters authentication information on a login screen.

[0061] The server verifies the credentials and retrieves the worker's profile.

[0062] 3. Display of work procedures

[0063] Once the authentication is complete, the server sends the work procedure information to the terminal.

[0064] The device visually displays the work procedure on the audio glasses and provides audio guidance.

[0065] 4. Work monitoring and feedback

[0066] Once the user begins work, the device's camera and sensors monitor the progress of the work.

[0067] The monitoring data is sent to the server in real time.

[0068] The server analyzes the monitoring data and checks for any incorrect operations.

[0069] 5. Warnings and Corrections

[0070] If an erroneous operation is detected, the server generates a warning message.

[0071] The device will display a visual warning on the audio glasses and provide audio instructions on how to fix the problem.

[0072] 6. Record and analyze work time

[0073] The completion time of each work step is transmitted from the terminal to the server.

[0074] The server evaluates the skill level of the worker by comparing with the standard work time.

[0075] Support proposals for skill development are generated and sent to the terminal along with the next work instructions.

[0076] Specific examples

[0077] The operation of the system will be explained below based on a specific warehouse work scenario.

[0078] 1. Initial Server Setup

[0079] The server reads the "picking work procedure data" and "worker A's profile data."

[0080] 2. User Authentication

[0081] Worker A enters login information on the terminal, and the server performs authentication.

[0082] 3. Display of work procedures

[0083] The server sends the instruction "First, pick product B from shelf number A1" to the terminal.

[0084] The terminal displays the location of shelf number A1 on the audio glasses display and provides voice guidance.

[0085] 4. Work monitoring and feedback

[0086] Worker A begins to take out product B from shelf A1.

[0087] The device sends the work status to the server via the camera and sensors.

[0088] 5. Warnings and Corrections

[0089] The server detects an incorrect operation (e.g., trying to remove product C from shelf A2).

[0090] The terminal will provide visual and audio guidance, saying, "Please take product B from shelf A1."

[0091] 6. Record and analyze work time

[0092] After worker A correctly picks up product B, the server records the task completion time.

[0093] The server compares the time with the standard work time and evaluates the skill level of worker A.

[0094] "Tips for improving work efficiency before proceeding to the next step" are displayed on the device and explained in audio.

[0095] This allows workers to perform new tasks with confidence, preventing operational errors and improving skills.

[0096] The processing flow will be explained below.

[0097] Program processing flow

[0098] Step 1: Initial Server Setup

[0099] server

[0100] The server starts up and loads the work process database and worker profile data.

[0101] The work procedure data includes work steps, standard work times, and error conditions.

[0102] The server logs that initialization is complete.

[0103] Step 2: Authenticate the user

[0104] Terminal

[0105] The device (Audio Glasses) will start up and the login screen will be displayed.

[0106] The user enters authentication information (user ID and password).

[0107] server

[0108] The server receives the authentication information and checks it against a database.

[0109] If the authentication is successful, the worker's profile information is acquired and sent to the terminal.

[0110] Step 3: View the work instructions

[0111] server

[0112] The server transmits current work process information to the terminal.

[0113] Terminal

[0114] The device displays the work procedure on the audio glasses display as a video and provides audio instructions.

[0115] For example, the instruction displayed might be, "Please pick product B from shelf number A1."

[0116] Step 4: Getting started

[0117] User

[0118] The user follows the instructions and starts the task. For example, the user goes to shelf A1 and takes out product B.

[0119] Terminal

[0120] The device's camera and sensors monitor the user's movements in real time and send the monitoring data to a server.

[0121] Step 5: Operational monitoring and analysis

[0122] server

[0123] The server receives the monitoring data and the AI ​​module analyzes the work progress.

[0124] Determine whether any malfunctions or abnormalities are detected.

[0125] Step 6: Detect and warn of misoperation

[0126] server

[0127] If the server detects an error, it generates a warning message.

[0128] Terminal

[0129] The device will display a visual warning message on the audio glasses and issue an audio warning.

[0130] For example, the system will guide you on how to correct the error by saying, "Please take product B from shelf A1."

[0131] User

[0132] The user checks the warning and corrects the error as instructed. For example, the user picks up the correct product from shelf A1.

[0133] Step 7: Notify and record completion of work

[0134] Terminal

[0135] After the user has correctly completed the task, the terminal sends a task completion notification to the server.

[0136] server

[0137] The server records the time the task is completed.

[0138] Evaluate user skill levels by comparing them with standard work hours.

[0139] Step 8: Assess skill levels and generate support suggestions

[0140] server

[0141] The server generates support proposals for skill improvement based on the user's skill level.

[0142] Send support suggestions to the device along with the next work instructions.

[0143] Terminal

[0144] The device displays the next steps on the audio glasses and provides audio instructions.

[0145] Provide users with visual and audio cues to improve their skills.

[0146] Step 9: Start the next process

[0147] User

[0148] The user starts the next work process and proceeds with the work while referring to the support proposal.

[0149] The above is the specific flow of each processing step of the program. This allows workers to perform new tasks with confidence, prevents operational errors, and improves their skills.

[0150] Example 1

[0151] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0152] Conventional work navigation systems lacked the ability to detect and correct worker errors in real time. They also lacked a mechanism for providing guidance and feedback tailored to each worker's skill level, making it difficult to improve productivity and work quality. Furthermore, it was difficult to provide customized work procedures based on user authentication and profiles, and they could only provide uniform guidance.

[0153] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0154] In this invention, the server includes means for accessing the work process database and managing each work step, wearable terminal means for outputting instructions related to the work process visually and audibly, AI module means for monitoring the worker's operations in real time and providing feedback via the wearable terminal means, control means for issuing a visual and audio warning and preventing the worker from proceeding to the next step if an incorrect operation is detected, evaluation means for recording the completion time of each work step and comparing it with a standard work time to evaluate the worker's skill level, means for receiving user authentication information and providing a customized work procedure based on the worker profile, and control means for analyzing monitoring data acquired in real time and providing appropriate feedback to the worker if an incorrect operation is detected. This makes it possible to detect and correct incorrect operations, provide customized guidance according to the worker's skill, and provide feedback in real time.

[0155] The "work process database" is a database for storing and managing information about work procedures and processes.

[0156] "Work steps" refers to the set of instructions or procedures required to accomplish a particular work process.

[0157] A "wearable terminal" is an information display and input device that can be worn by a worker and has the function of issuing visual and audio instructions.

[0158] The "AI module" is a system component that uses artificial intelligence technology to perform data analysis and pattern recognition, and provides appropriate feedback to workers.

[0159] A "controller" is a hardware or software function that governs the operation of the entire system and performs appropriate actions under specific conditions.

[0160] The "evaluation means" is a function for quantitatively evaluating the performance of a worker and evaluating the skill level by comparing it with the standard work time.

[0161] "User authentication information" refers to information used to identify a user, such as an ID or password entered when accessing a system.

[0162] A "worker profile" is data that includes detailed information about each individual worker, such as the worker's skill level and past work history.

[0163] A "customized work procedure" is a work procedure that is individually generated based on a worker profile and is adjusted according to the worker's skills and needs.

[0164] "Real-time monitoring data" refers to data that is collected and analyzed in real time to track the movements and working conditions of workers.

[0165] "Feedback" refers to instructions and suggested corrections that the system provides to the worker, including information that helps them understand the progress of their work and correct mistakes.

[0166] MODE FOR CARRYING OUT THE INVENTION

[0167] The present invention provides a system that utilizes an AI module and a wearable device to navigate workers in real time and improve productivity. Specific embodiments of the system are described below.

[0168] System Configuration

[0169] The system consists of the following main components:

[0170] 1. Server

[0171] Access to the work process database

[0172] Management of each work step

[0173] Analysis of work monitoring data

[0174] Generate feedback and warnings

[0175] Manage worker profiles

[0176] 2. Wearable devices

[0177] Visual display of work instructions (e.g. audio glasses)

[0178] Voice output of work instructions

[0179] User authentication function

[0180] Work monitoring using cameras and sensors

[0181] 3. Users

[0182] Check and implement work instructions

[0183] Receiving and responding to feedback

[0184] Implementing correction work

[0185] Program processing overview

[0186] Explain how the components of the system work together.

[0187] 1. Initial Server Setup

[0188] When the server starts up, it reads the work process database and worker profile data.

[0189] The work procedure data includes work steps, standard work times, and error conditions.

[0190] 2. User Authentication

[0191] When the device boots up, the user enters authentication information on a login screen.

[0192] The server verifies the credentials and retrieves the worker's profile.

[0193] 3. Display of work procedures

[0194] Once the authentication is complete, the server sends the work procedure information to the terminal.

[0195] The device visually displays the work procedure on the audio glasses and begins providing audio guidance.

[0196] 4. Work monitoring and feedback

[0197] Once the user begins work, the device's camera and sensors monitor the progress of the work.

[0198] The monitoring data is sent to a server in real time and analyzed.

[0199] The server analyzes the monitoring data and checks for any incorrect operations.

[0200] 5. Warnings and Corrections

[0201] If an erroneous operation is detected, the server generates a warning message.

[0202] The device will display a visual warning on the audio glasses and provide audio instructions on how to fix the problem.

[0203] 6. Record and analyze work time

[0204] The completion time of each work step is transmitted from the terminal to the server.

[0205] The server evaluates the skill level of the worker by comparing with the standard work time.

[0206] Support proposals for skill development are generated and sent to the terminal along with the next work instructions.

[0207] Specific examples

[0208] The operation of the system will be explained below based on a specific warehouse work scenario.

[0209] (example)

[0210] 1. Initial Server Setup

[0211] The server reads the "picking work procedure data" and obtains the "profile data" of worker A.

[0212] 2. User Authentication

[0213] Worker A enters login information on the terminal, and the server performs authentication.

[0214] 3. Display of work procedures

[0215] The server sends the instruction "First, pick product B from shelf number A1" to the terminal.

[0216] The terminal displays "Shelf Number A1" on the audio glasses and provides voice guidance.

[0217] 4. Work monitoring and feedback

[0218] Worker A goes to shelf A1 and begins the action of taking out product B.

[0219] The device's camera captures the movement and sends it to the server.

[0220] The server analyzes the monitoring data to determine the progress of the work.

[0221] 5. Warnings and Corrections

[0222] The server detects the incorrect operation and generates a "warning message."

[0223] The terminal will provide a visual warning saying "Please take product B from shelf A1" and also provide audio guidance.

[0224] 6. Record and analyze work time

[0225] The terminal sends the "task completion time" to the server.

[0226] The server compares the data with the "standard work time data" to evaluate the skill level of worker A.

[0227] It generates "hints for improving work efficiency along with the next step" and sends them to the terminal.

[0228] The system utilizes an AI module to monitor worker behavior in real time and improve work quality. It uses a generative AI model to generate appropriate prompts and provide timely instructions and feedback to workers.

[0229] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0230] Step 1:

[0231] Initial Server Configuration

[0232] Input: When the server starts, it reads the work process database and worker profile data.

[0233] Specific behavior:

[0234] The server retrieves "picking work procedure data" from the "work process database," which includes shelf numbers, product IDs, and operation procedures.

[0235] The server obtains worker A's skill level and past work history from the "worker profile data."

[0236] Output: The work procedure data and worker profile data are loaded, and the system completes its initial setup.

[0237] Step 2:

[0238] User Authentication

[0239] Input: The device boots up and the user enters their credentials at the login screen.

[0240] Specific behavior:

[0241] The terminal displays a login dialog and the user enters their "ID" and "password."

[0242] The server verifies the authentication information through the "User Authentication API."

[0243] If the authentication is successful, the server obtains the "worker profile data."

[0244] Output: User authentication is successful and worker profile data is retrieved.

[0245] Step 3:

[0246] Displaying work procedures

[0247] Input: After successful user authentication, the server sends the work procedure information to the terminal.

[0248] Specific behavior:

[0249] The server sends the "work procedure information" to the terminal.

[0250] The terminal displays on the audio glasses, "Please pick product B from shelf number A1."

[0251] The device will provide similar instructions via voice prompts.

[0252] Output: The work procedure is displayed on the terminal and the user is guided by voice.

[0253] Step 4:

[0254] Work monitoring and feedback

[0255] Input: The user starts a task, and the device's camera and sensors monitor the task's progress.

[0256] Specific behavior:

[0257] The user begins to move towards "shelf number A1" to take out product B.

[0258] The device's camera and sensors capture its movements and transmit "surveillance data" to a server in real time.

[0259] The server analyzes real-time monitoring data and compares it with prescribed work procedures.

[0260] Output: The monitoring data is sent to the server in real time and the analysis results are obtained.

[0261] Step 5:

[0262] Warnings and fixes

[0263] Input: Detects misoperation using monitoring data analyzed by the server.

[0264] Specific behavior:

[0265] If the server detects an "incorrect operation", it generates a "warning message".

[0266] For example, if a user attempts to remove product C from shelf A2, the server generates a "warning message."

[0267] The terminal will provide a visual warning and audio guidance saying, "Please take product B from shelf A1."

[0268] Output: Information is provided to help the user recognize and correct the error.

[0269] Step 6:

[0270] Record and analyze work time

[0271] Input: As each work step is completed, the completion time is sent from the terminal to the server.

[0272] Specific behavior:

[0273] The terminal sends the "task completion time" to the server.

[0274] The server compares the data with the "standard work time data" and evaluates the skill level of worker A.

[0275] As a result of the analysis, the server generates "hints for improving work efficiency along with the next step" and sends them to the terminal.

[0276] Output: The task completion time is recorded, the worker's skill level is evaluated, and hints for the next step are provided.

[0277] Through these steps, the system monitors the worker's actions in real time and provides appropriate feedback and instructions, thereby improving productivity and work quality.

[0278] (Application example 1)

[0279] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0280] Conventional work management systems have been plagued by problems such as operator errors and reduced work efficiency. Real-time monitoring and appropriate feedback were lacking, making it difficult to improve work accuracy and skills. Furthermore, customized guides for improving work efficiency were not provided. The present invention aims to solve these problems and improve worker productivity and accuracy.

[0281] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0282] In this invention, the server includes: means for accessing the work process database and managing each work step; wearable terminal means for outputting instructions related to the work process visually and audibly; AI module means for monitoring the worker's operations in real time and providing feedback via the wearable terminal means; control means for issuing visual and audio warnings and preventing the worker from proceeding to the next step if an incorrect operation is detected; evaluation means for recording the completion time of each work step and comparing it with a standard work time to evaluate the worker's skill level; means for providing visual and audio guides of the work instructions; means with a camera function for monitoring the user's actions in real time; and audio output means for generating audio guides. This makes it possible to monitor the worker's operations in real time and provide immediate feedback and guidance on the correct operation method if an incorrect operation occurs.

[0283] A "work process database" is a database for managing and storing information about the procedures and steps of a specific work.

[0284] A "visual display" is a means of presenting information in a form that can be seen by a user, and includes, for example, a display or a projector.

[0285] "Audio output" refers to a means of providing information to a user by voice using a speaker, headphones, or the like.

[0286] A "wearable device" is an electronic device that can be worn on the body and used, such as smart glasses and head-mounted displays.

[0287] "Real-time monitoring" is the process of checking the situation and collecting data at the exact moment that a task or operation is being performed.

[0288] An "AI module" is a software or hardware module that uses artificial intelligence algorithms to analyze data and make decisions.

[0289] "Feedback" is a means of indicating errors and areas for improvement by providing real-time evaluation and guidance on operations and actions.

[0290] A "warning" is a visual or audio notification that alerts the user to a risk or error and prompts caution.

[0291] A "control means" is a component equipped with the functions and algorithms necessary to monitor the entire system and provide appropriate operation and feedback.

[0292] The "evaluation means" is a function that evaluates the skill level of a worker by recording the completion time and work quality of each work step and comparing them with the standard work time.

[0293] The "camera function" is a means of collecting visual information and recording and analyzing it as images or videos.

[0294] "Voice guide" is a function that provides appropriate instructions and guidance to workers using voice.

[0295] The present invention is a system that utilizes an AI module and a wearable device to navigate workers in real time and improve productivity. Specific embodiments and program processing of the system are described below.

[0296] The system that realizes this application example is created based on the following program.

[0297] Program processing explanation

[0298] Initial Setup

[0299] The server accesses the work process database to manage each work step. The work process data includes work steps, standard work times, and error conditions. Worker profile data is also loaded.

[0300] User Authentication

[0301] The user enters login information on the terminal, the server verifies the authentication information, and if authentication is successful, retrieves the user's work profile.

[0302] View work orders

[0303] Once authenticated, the server sends customized work instructions to the device, which then displays the instructions visually and provides audio guidance via smart glasses or a head-mounted display. Google Text-to-Speech (gTTS) is used for voice output.

[0304] Work monitoring and feedback

[0305] When the user starts working, the device's built-in camera (using OpenCV) and sensors monitor the work status in real time. The monitoring data is sent to a server, where an AI module analyzes the data and checks for any operational errors.

[0306] Warnings and fixes

[0307] If an incorrect operation is detected, the server generates a warning message and provides visual and audio feedback to the operator via the terminal, allowing the operator to correct the incorrect operation.

[0308] Record and analyze work time

[0309] The completion time for each task is sent from the terminal to the server, which then compares it with the standard work time. This allows the worker's skill level to be evaluated, and the results are reflected in subsequent work instructions.

[0310] Specific examples

[0311] For example, consider a warehouse operation at a logistics center. When a worker is tasked with new picking work, the following scenario can be envisioned.

[0312] The server loads the "picking work procedure data" and "worker A's profile data." Worker A enters his / her login information on the terminal, and the server authenticates him / her. After authentication, the server sends the instruction "First, pick product B from shelf number A1" to the terminal. The terminal then displays the location of shelf number A1 on the smart glasses' display and provides audio guidance.

[0313] When worker A goes to shelf A1 and tries to take item B, the device's camera and sensors monitor the action and send the situation to the server in real time. If the server detects an incorrect operation (e.g., trying to take item C from shelf A2), the device will provide visual and audio guidance saying, "Please take item B from shelf A1." After correctly taking item B, the server records the task completion time and compares it with the standard work time to evaluate the skill level.

[0314] An example prompt is:

[0315] Example prompt

[0316] user_info = authenticate_user("new_worker", "securepass")

[0317] tasks = get_task_instructions(user_info['user_id'])

[0318] for task in tasks:

[0319] print(f"Next task: {task['instruction']}")

[0320] play_audio(task['instruction'])

[0321] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0322] Step 1:

[0323] The server accesses the work process database and reads each work step and worker profile data.

[0324] Input: Work process database, worker profile data

[0325] Data calculation: Retrieving information from a database.

[0326] Output: Loaded work procedure data, worker profile data

[0327] Step 2:

[0328] The user enters login information at the terminal and the server verifies the authentication information.

[0329] Input: User login information

[0330] Data calculation: Checks credentials and retrieves worker profile.

[0331] Output: Authenticated user's work profile

[0332] Step 3:

[0333] The server transmits customized work procedure information to the terminal for the authenticated user.

[0334] Input: Authenticated user's work profile

[0335] Data calculation: Generate customized work procedures based on work profiles

[0336] Output: Customized routing information

[0337] Step 4:

[0338] The device visually displays work procedures and provides audio guidance via smart glasses or a head-mounted display.

[0339] Input: Customized routing information

[0340] Data calculation: Visual display and speech translation of work instructions (using gTTS)

[0341] Output: Visually displayed work instructions, audio guide

[0342] Step 5:

[0343] The user starts working, and the camera (using OpenCV) and sensors on the device monitor the work status in real time.

[0344] Input: User's work actions

[0345] Data Calculation: Analysis of camera footage and sensor data

[0346] Output: Monitoring data (real-time)

[0347] Step 6:

[0348] The device sends the collected monitoring data to a server, which then uses an AI module to analyze the data and check for any operational errors.

[0349] Input: Monitoring data

[0350] Data calculation: Data analysis and error detection by AI module

[0351] Output: Misoperation detection result

[0352] Step 7:

[0353] If an incorrect operation is detected, the server generates a warning message and provides visual and audio feedback to the user via the terminal.

[0354] Input: Misoperation detection result

[0355] Data Calculation: Generate warning messages and provide visual and audio feedback

[0356] Output: Warning messages, visual and audio feedback

[0357] Step 8:

[0358] The server records the completion time of each work step, compares it with the standard work time to evaluate the skill level, and reflects the results in subsequent work instructions.

[0359] Input: Completion time for each work step

[0360] Data calculation: Comparison with standard work time and skill level evaluation

[0361] Output: Skill level evaluation results, data to be reflected in the next work instructions

[0362] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0363] MODE FOR CARRYING OUT THE INVENTION

[0364] The present invention is a system that utilizes an AI module, a wearable device, and an emotion engine to navigate workers in real time and provide feedback according to the worker's emotional state, thereby improving productivity. Specific embodiments and program processing will be described below.

[0365] System Configuration

[0366] The system consists of the following main components:

[0367] 1. Server

[0368] Access to the work process database

[0369] Managing work steps

[0370] Analysis of work monitoring data and emotion data

[0371] Generating feedback, warnings, and emotion-responsive messages

[0372] Manage worker profiles

[0373] 2. Wearable devices

[0374] Visual and audio display of work instructions (e.g. audio glasses)

[0375] User authentication function

[0376] Work monitoring using cameras and sensors

[0377] Emotion recognition sensors (e.g. facial recognition cameras, voice analysis microphones)

[0378] 3. Users

[0379] Check and implement work instructions

[0380] Receiving and responding to feedback and emotional responses

[0381] Implementing correction work

[0382] 4. Emotion Engine

[0383] Data analysis from emotion recognition sensors

[0384] Determining the user's emotional state

[0385] Message generation according to emotional state

[0386] Program processing overview

[0387] Explain how the components of the system work together.

[0388] 1. Initial Server Setup

[0389] When the server starts up, it loads the work process database, the worker profile data, and the emotional state data.

[0390] The work procedure data includes work steps, standard work times, and error conditions.

[0391] 2. User Authentication

[0392] When the device boots up, the user enters authentication information on a login screen.

[0393] The server verifies the credentials and retrieves the worker's profile.

[0394] 3. Display of work procedures

[0395] Once the authentication is complete, the server sends the work procedure information to the terminal.

[0396] The device visually displays the work procedure on the audio glasses display and provides audio guidance.

[0397] 4. Work monitoring and feedback

[0398] Once the user begins work, the device's camera and sensors monitor the progress of the work.

[0399] The monitoring data is sent to the server in real time.

[0400] The server analyzes the monitoring data and checks for any incorrect operations.

[0401] 5. Recognizing emotional states

[0402] The emotion engine receives and analyzes data from the device's emotion recognition sensor.

[0403] The server determines the user's emotional state based on their visual expressions, vocal tone, and physical movements.

[0404] 6. Warnings and Emotional Messages

[0405] If an erroneous operation is detected, the server generates a warning message.

[0406] The emotion engine generates relaxation instructions and motivational messages according to the user's emotional state.

[0407] The device will display a visual warning on the audio glasses and provide audio instructions on how to fix the problem.

[0408] 7. Record and analyze work time

[0409] The completion time of each work step is transmitted from the terminal to the server.

[0410] The server evaluates the skill level of the user by comparing it with standard work time.

[0411] 8. Skill level assessment and support proposal generation

[0412] The server generates support proposals for skill improvement based on the user's skill level.

[0413] Send support suggestions to the device along with the next work instructions.

[0414] 9. Start of the next process

[0415] The user starts the next work process and proceeds with the work while referring to the support suggestions and emotional response messages.

[0416] Specific examples

[0417] The operation of the system will be explained below based on a specific warehouse work scenario.

[0418] 1. Initial Server Setup

[0419] The server reads the "picking work procedure data" and "worker A's profile data."

[0420] 2. User Authentication

[0421] Worker A enters login information on the terminal, and the server performs authentication.

[0422] 3. Display of work procedures

[0423] The server sends the instruction "First, pick product B from shelf number A1" to the terminal.

[0424] The terminal displays the location of shelf number A1 on the audio glasses display and provides voice guidance.

[0425] 4. Work monitoring and feedback

[0426] Worker A begins to take out product B from shelf A1.

[0427] The device sends the work status to the server via the camera and sensors.

[0428] 5. Recognizing emotional states

[0429] The emotion engine analyzes worker A's facial expressions and tone of voice and determines that he is under stress.

[0430] 6. Warnings and Emotional Messages

[0431] If the server detects an incorrect operation, it generates a warning message saying, "Please remove product B from shelf A1."

[0432] The emotion engine generates an emotion-responsive message: "Relax and take a deep breath."

[0433] The device displays warnings and emotion-responsive messages on the audio glasses and provides voice guidance.

[0434] 7. Record and analyze work time

[0435] After worker A correctly picks up product B, the server records the task completion time.

[0436] The server compares the time with the standard work time and evaluates the skill level of worker A.

[0437] 8. Skill level assessment and support proposal generation

[0438] The server displays "tips for improving work efficiency before proceeding to the next step" on the terminal and provides audio explanations.

[0439] 9. Start of the next process

[0440] Worker A starts the next work process and proceeds with the work while referring to the support proposal and emotional response message.

[0441] This allows workers to perform new tasks with confidence, preventing operational errors and improving their skills. In addition, the emotion engine provides appropriate feedback according to the worker's emotional state, which is expected to further improve the work environment and work efficiency.

[0442] The processing flow will be explained below.

[0443] Program processing flow

[0444] Step 1: Initial Server Setup

[0445] server

[0446] The server starts up and loads the work process database, worker profile data, and emotional state data.

[0447] The work procedure data includes work steps, standard work times, and error conditions.

[0448] The server logs that initialization is complete.

[0449] Step 2: Authenticate the user

[0450] Terminal

[0451] The device (Audio Glasses) will start up and the login screen will be displayed.

[0452] The user enters authentication information (user ID and password).

[0453] server

[0454] The server receives the authentication information and checks it against a database.

[0455] If the authentication is successful, the worker's profile information is acquired and sent to the terminal.

[0456] Step 3: View the work instructions

[0457] server

[0458] The server transmits current work process information to the terminal.

[0459] Terminal

[0460] The device displays the work procedure on the audio glasses display as a video and provides audio instructions.

[0461] For example, the instruction displayed might be, "Please pick product B from shelf number A1."

[0462] Step 4: Getting started

[0463] User

[0464] The user follows the instructions and starts the task. For example, the user goes to shelf A1 and takes out product B.

[0465] Terminal

[0466] The device's camera and sensors monitor the user's movements in real time and send the monitoring data to a server.

[0467] Step 5: Operational monitoring and analysis

[0468] server

[0469] The server receives the monitoring data and the AI ​​module analyzes the work progress.

[0470] Determine whether any malfunctions or abnormalities are detected.

[0471] Step 6: Recognize your emotional state

[0472] Terminal

[0473] The device's emotion recognition sensors (e.g., facial expression recognition camera, voice analysis microphone) capture the user's visual expressions, voice tone, and physical movements.

[0474] The emotion data is sent to the server.

[0475] server

[0476] The server uses an emotion engine to analyze the user's emotional state.

[0477] Through analysis, it is determined that the user is feeling stressed.

[0478] Step 7: Misoperation detection and warning

[0479] server

[0480] If the server detects an incorrect operation, it generates a warning message saying, "Please remove product B from shelf A1."

[0481] Terminal

[0482] The device will visually display a warning message on the audio glasses and issue an audio warning.

[0483] User

[0484] The user checks the warning and corrects the error as instructed. For example, the user picks up the correct product from shelf A1.

[0485] Step 8: Generate and display emotion-aware messages

[0486] server

[0487] The emotion engine generates an emotion-responsive message such as "Relax and take a deep breath" depending on the user's emotional state.

[0488] The server sends an emotion-related message to the terminal.

[0489] Terminal

[0490] The device visually displays emotion-based messages on the audio glasses and provides audio guidance.

[0491] Step 9: Notify and record completion of work

[0492] Terminal

[0493] After the user has correctly completed the task, the terminal sends a task completion notification to the server.

[0494] server

[0495] The server records the time the task is completed.

[0496] Evaluate user skill levels by comparing them with standard work hours.

[0497] Step 10: Assess skill levels and generate support suggestions

[0498] server

[0499] The server generates support proposals for skill improvement based on the user's skill level.

[0500] Send support suggestions to the device along with the next work instructions.

[0501] Terminal

[0502] The device displays the next steps and tips for improving skills on the audio glasses and provides audio guidance.

[0503] Step 11: Start the next process

[0504] User

[0505] The user starts the next work process and proceeds with the work while referring to the support suggestions and emotional response messages.

[0506] The above is the specific flow of each processing step of the program that combines the emotion engine. This allows workers to work with peace of mind while receiving appropriate feedback according to their emotional state. This prevents operational errors, improves skills, and increases work efficiency.

[0507] Example 2

[0508] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0509] Conventional work support systems often focus on providing immediate feedback to workers regarding operational errors and improving work efficiency, but lack a mechanism to respond to the worker's emotional state. As a result, worker stress and a decline in motivation can have a negative impact on work efficiency and quality. Furthermore, it has been difficult to effectively navigate by combining real-time emotional feedback and work instructions.

[0510] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for accessing the work process database and managing each work step; wearable terminal means for outputting instructions related to the work process visually and audibly; AI module means for monitoring the worker's operations in real time and providing feedback via the wearable terminal means; control means for issuing visual and audio warnings and preventing the worker from proceeding to the next step when an erroneous operation is detected; emotion engine means for analyzing data from an emotion recognition sensor and determining the worker's emotional state; means for generating messages according to the emotional state and providing them via the wearable terminal means; and evaluation means for recording the completion time of each work step and comparing it with the standard work time to evaluate the worker's skill level. This makes it possible to comprehensively manage the worker's real-time work status and emotional state and provide appropriate feedback and emotion-responsive messages.

[0511] The "work process database" is a database for recording and managing information about each work step.

[0512] A "wearable terminal" is a device that can be worn by a user and provides work instructions and feedback.

[0513] An "AI module" is a device that includes an artificial intelligence program that monitors the operations of workers in real time and provides feedback based on the analysis results.

[0514] "Control means" refers to a device or function that detects operational errors, issues visual and audio warnings as a countermeasure, and controls the progress of work.

[0515] An "emotion recognition sensor" is a sensor that detects a worker's facial expression, tone of voice, physical movements, etc., and determines their emotional state.

[0516] An "emotion engine" is software or hardware that analyzes data from emotion recognition sensors and determines the user's emotional state.

[0517] The "evaluation means" is a device or function that records the time required to complete each work step and compares it with the standard work time to evaluate the skill level of the worker.

[0518] "Feedback" refers to information or instructions that provide an evaluation of the worker's operations and areas for improvement.

[0519] "Emotion-responsive messages" are messages that instruct users to relax or increase their motivation, which are generated in response to the user's emotional state.

[0520] "Real-time" means processing or responding immediately with little or no delay.

[0521] MODE FOR CARRYING OUT THE INVENTION

[0522] The present invention provides a system for providing feedback and navigation according to the emotional state of a worker. The system includes a server, a wearable device, an AI module, an emotion recognition sensor, and an emotion engine.

[0523] System Configuration

[0524] The system consists of the following main elements:

[0525] 1. Server

[0526] Access the work process database and manage each work step.

[0527] Stores worker profile data and retrieves necessary information.

[0528] Monitor the progress of work steps in real time and generate feedback.

[0529] 2. Wearable devices

[0530] This device outputs work instructions visually and audibly. An example of such a device is audio glasses.

[0531] A login screen is provided for user authentication.

[0532] Work is monitored in real time using cameras and various sensors.

[0533] It is equipped with an emotion recognition sensor that analyzes the user's facial expressions and tone of voice.

[0534] 3. AI Module

[0535] Analyzes work progress and provides feedback if an error is detected.

[0536] 4. Emotion Recognition Sensor

[0537] Sensors for detecting the user's emotional state in real time include a facial recognition camera and a voice analysis microphone.

[0538] 5. Emotion Engine

[0539] Data from emotion recognition sensors is analyzed to determine the user's emotional state.

[0540] Messages are generated according to the emotional state and provided via a wearable device.

[0541] Program processing overview

[0542] The following describes how each component of the system works together.

[0543] Initial Server Configuration

[0544] When the server starts up, it loads the work process database and the worker profile database. The work process data includes the work procedure for each step, the standard work time, the list of required materials, error conditions, etc.

[0545] User Authentication

[0546] When a user enters their authentication information on the login screen of their wearable device, the device sends the information to the server, which verifies the authentication information and retrieves the worker's profile.

[0547] Displaying work procedures

[0548] Once authentication is complete, the server sends the work procedure information to the device, which then displays visual instructions on the audio glasses and provides audio guidance.

[0549] Work monitoring and feedback

[0550] As the user performs a task, the device's camera and sensors monitor the progress and send the data in real time to a server, which analyzes the data and provides feedback via the wearable device if any errors are detected.

[0551] Recognition of emotional states

[0552] The emotion engine analyzes emotion recognition sensor data from the device and determines the user's emotional state. Based on the results, the server generates an appropriate emotion-responsive message and sends it to the device.

[0553] Warnings and Emotional Messages

[0554] If an incorrect operation is detected, the server generates a warning message, and the emotion engine simultaneously generates a message to instruct the user to relax or to increase motivation, which is then sent via the device.

[0555] Record and analyze work time

[0556] The completion time of each work step is sent from the terminal to a server, and the server compares the recorded data with standard work times to evaluate the skill level.

[0557] Skill level assessment and support suggestions

[0558] The server analyzes the user's skill level and generates support suggestions for skill improvement, which are then provided to the wearable device along with next work instructions.

[0559] Start of the next process

[0560] The user starts the next task, and the device displays support suggestions and emotional response messages to assist them.

[0561] Specific examples

[0562] The operation of the system will be explained below based on a specific warehouse work scenario.

[0563] 1. As the initial setting of the server, the picking work procedure data and worker profile data are read.

[0564] 2. The worker enters login information on the terminal, and the server performs authentication.

[0565] 3. Once authentication is complete, the instruction "First, please pick product B from shelf number A1" is sent to the terminal. The terminal displays the location on the audio glasses and gives instructions via voice.

[0566] 4. Once work begins, the device monitors progress using cameras and sensors and sends the data to the server.

[0567] 5. The emotion engine analyzes the worker's facial expressions and tone of voice and determines, for example, whether they are in a stressed state.

[0568] 6. If an incorrect operation is detected, the server generates a warning message and an emotion-based message such as "Relax and take a deep breath." This message is displayed on the device and guided by voice.

[0569] 7. The device records the time the task is completed, and the server analyzes the data.

[0570] 8. The server evaluates the skill level of the worker and provides tips to improve efficiency.

[0571] 9. The worker begins the next step and proceeds with the work while referring to the support suggestions and emotional response messages.

[0572] Example of input prompt for generative AI model

[0573] Below is an example of a prompt sentence to be input to the generative AI model.

[0574] "The goal of this system is to improve worker productivity. Please explain the following steps in natural language: The worker takes item B from shelf A1. If there is an error, detect the error and provide feedback for correction. Finally, recognize the worker's emotional state and generate an appropriate emotional response message."

[0575] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0576] Specific processing steps of the program

[0577] Step 1: Initial Server Setup

[0578] When the server starts up, it accesses the work process database and worker profile database and loads the necessary information into memory. The inputs are the work process database and worker profile database. The output is work procedure data (each work step, standard work time, error conditions) and worker profile data.

[0579] Specifically, the server acquires the work procedure data and the worker profile data from the database, analyzes them, and completes the initial settings.

[0580] Step 2: User authentication

[0581] The user enters their authentication information on the login screen of the wearable device. The device sends the information to the server, which checks the authentication information in the database and retrieves the user's profile if authentication is successful.

[0582] The input is user authentication information, and the output is profile data for a particular user.

[0583] Specifically, the user enters a username and password into the terminal, and the server returns the authentication result to the terminal.

[0584] Step 3: View the work instructions

[0585] For users who have been successfully authenticated, the server generates work instruction data and sends it to the terminal, which then displays the work procedure visually on the audio glasses and provides audio instructions.

[0586] The inputs are user profile data and work procedure data, and the output is a display of the work procedure and voice guidance.

[0587] Specifically, the server sends information about the next work step to the terminal, and the terminal displays the work procedure on the audio glasses and provides audio guidance.

[0588] Step 4: Work monitoring and feedback

[0589] When the user starts a task, the device's camera and various sensors monitor the progress of the task and send the data in real time to the server, which analyzes the data, checks for errors and smooth progress, and provides necessary feedback to the device.

[0590] The input is sensor data (work progress), and the output is feedback messages.

[0591] Specifically, the device's camera captures images of the user's hands, the location sensor tracks the progress of the task, and the acquired data is sent to a server, which analyzes the data and generates the necessary feedback.

[0592] Step 5: Recognizing your emotional state

[0593] The device's emotion recognition sensor captures the user's facial expressions and voice tone in real time, and the emotion engine analyzes the data. The server then determines the user's emotional state based on the results.

[0594] The input is emotion recognition sensor data (facial expressions, voice tone), and the output is the user's emotional state.

[0595] Specifically, the device's camera captures facial expressions and microphone collects voice tones, and then sends these data to the emotion engine, which analyzes the data and determines the user's emotional state.

[0596] Step 6: Warnings and Emotional Messages

[0597] If the server detects an incorrect operation, it generates a warning message, and the emotion engine generates a message according to the user's emotional state. These messages are sent to the terminal, which then displays and provides voice guidance.

[0598] The inputs are error data and emotional state data, and the output is a warning message and an emotional response message.

[0599] Specifically, when the server detects an incorrect operation, it generates a warning message saying, "An incorrect operation has occurred. Please return to the correct procedure." If the emotion engine determines that the user is in a stressful state, it outputs a message such as, "Relax and take a deep breath." The device displays these messages on the audio glasses and provides audio guidance.

[0600] Step 7: Record and analyze your work time

[0601] The device records the time it takes to complete each task and sends it to the server, which then compares it with the standard time to assess the user's skill level.

[0602] The input is task completion time data, and the output is a user skill level assessment.

[0603] Specifically, the device records the time when the user completes a task and sends the data to the server, which compares it with the standard work time and evaluates the skill level.

[0604] Step 8: Assess skill levels and generate support suggestions

[0605] The server generates support suggestions for the next step based on the user's skill level and sends them to the device, which then displays the suggestions and provides audio guidance.

[0606] The input is the skill level assessment result, and the output is a support proposal.

[0607] Specifically, the server evaluates the user's skill level, generates support suggestions such as "Points to help you work more efficiently in the next step," and sends them to the device. The device then displays the suggestions on the audio glasses and provides audio guidance.

[0608] Step 9: Start the next process

[0609] The user starts the next step of the work process, and the device proceeds with the work while referring to support suggestions and emotion-responsive messages.

[0610] The inputs are the next work instructions, support ideas, and emotion-responsive messages. The output is the progress of the next work process.

[0611] Specifically, the user reads the instructions for the next step, starts the work, and continues working efficiently while referring to support suggestions and emotional response messages from the device.

[0612] (Application example 2)

[0613] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0614] In the current work environment at logistics centers, work instructions are given without considering the emotional state of workers, which can lead to the accumulation of stress and fatigue, resulting in a decline in work efficiency and productivity. Another problem is that when an operational error occurs, a response is delayed, resulting in an increase in overall work time. The purpose of this invention is to solve these problems and improve work efficiency and productivity by providing appropriate feedback according to the emotional state of workers.

[0615] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for accessing a work process database and managing each work step; wearable terminal means for outputting instructions related to the work process visually and audibly; artificial intelligence module means for monitoring the worker's operations in real time and providing feedback via the wearable terminal means; control means for issuing a visual and audio warning and preventing the worker from proceeding to the next step if an erroneous operation is detected; emotion recognition means for analyzing the worker's facial expression and tone of voice to determine the worker's emotional state; emotion response feedback means for generating relaxation instructions or motivation-boosting messages according to the worker's emotional state and outputting them via the wearable terminal means; and evaluation means for recording the completion time of each work step and comparing it with the standard work time to evaluate the worker's skill level. This enables logistics centers to improve work efficiency and productivity while managing the emotions of workers.

[0616] The "work process database" is a database for managing information such as each work step and its procedure, standard work time, and conditions for operational errors.

[0617] "Wearable terminal means" refers to a device that can be worn by a worker and provides visual displays and audio instructions. For example, smart glasses and audio glasses are examples of such devices.

[0618] The "artificial intelligence module means" is a function that uses artificial intelligence to monitor the operator's operations in real time and provide feedback.

[0619] The "control means" is a function that issues a visual and audio warning when an erroneous operation is detected, and prevents the process from proceeding to the next step.

[0620] The "emotion recognition means" is a function for determining the emotional state of a worker by analyzing the worker's facial expression and tone of voice.

[0621] The "emotion-responsive feedback means" is a function for generating relaxation instructions and motivation-boosting messages according to the emotional state determined by the emotion recognition means, and outputting them on the wearable terminal means.

[0622] The "evaluation means" is a function for recording the completion time of each work step and comparing it with the standard work time to evaluate the skill level of the worker.

[0623] This invention is a system that combines artificial intelligence, wearable devices, and an emotion engine to improve the efficiency and productivity of workers at logistics centers. The system accesses a work process database, displays and outputs work instructions visually and audibly, monitors worker operations in real time, and issues a warning if an error is detected. It also analyzes the worker's facial expressions and tone of voice to determine their emotional state and provides feedback according to that state.

[0624] System Configuration

[0625] The system consists of the following main components:

[0626] 1. Server

[0627] Access to the work process database

[0628] Managing work steps

[0629] Analysis of work monitoring data and emotion data

[0630] Generating feedback, warnings, and emotion-responsive messages

[0631] Manage worker profiles

[0632] 2. Wearable devices

[0633] Visual and audio display of work instructions (e.g., smart glasses)

[0634] User authentication function

[0635] Work monitoring using cameras and sensors

[0636] Emotion recognition sensors (e.g. facial recognition cameras, voice analysis microphones)

[0637] 3. Users

[0638] Check and implement work instructions

[0639] Receiving and responding to feedback and emotional responses

[0640] Implementing correction work

[0641] 4. Emotion Engine

[0642] Data analysis from emotion recognition sensors

[0643] Determining the user's emotional state

[0644] Message generation according to emotional state

[0645] Program processing overview

[0646] Explain how the components of the system work together.

[0647] 1. Initial Server Setup

[0648] At startup, the server loads the work process database, worker profile data, and emotional state data.

[0649] The work procedure data includes work steps, standard work times, and error conditions.

[0650] 2. User Authentication

[0651] When the device boots up, the user enters authentication information on a login screen.

[0652] The server verifies the credentials and retrieves the worker's profile.

[0653] 3. Display of work procedures

[0654] Once the authentication is complete, the server sends the work procedure information to the terminal.

[0655] The device visually displays the work procedure on the smart glasses' display and provides audio guidance.

[0656] 4. Work monitoring and feedback

[0657] Once the user begins work, the device's camera and sensors monitor the progress of the work.

[0658] The monitoring data is sent to the server in real time.

[0659] The server analyzes the monitoring data and checks for any incorrect operations.

[0660] 5. Recognizing emotional states

[0661] The emotion engine receives and analyzes data from the device's emotion recognition sensor.

[0662] The server determines the user's emotional state based on their visual expressions, vocal tone, and physical movements.

[0663] 6. Warnings and Emotional Messages

[0664] If an erroneous operation is detected, the server generates a warning message.

[0665] The emotion engine generates relaxation instructions and motivational messages according to the user's emotional state.

[0666] The device will display a visual warning on the smart glasses and provide audio instructions on how to fix the problem.

[0667] 7. Record and analyze work time

[0668] The completion time of each work step is transmitted from the terminal to the server.

[0669] The server evaluates the skill level of the user by comparing it with standard work time.

[0670] 8. Skill level assessment and support proposal generation

[0671] The server generates support proposals for skill improvement based on the user's skill level.

[0672] Send support suggestions to the device along with the next work instructions.

[0673] 9. Start of the next process

[0674] The user starts the next work process and proceeds with the work while referring to the support suggestions and emotional response messages.

[0675] Specific examples

[0676] The operation of the system will be explained below based on a specific warehouse work scenario.

[0677] 1. Initial Server Setup

[0678] The server reads the "picking work procedure data" and "worker A's profile data."

[0679] 2. User Authentication

[0680] Worker A enters login information on the terminal, and the server performs authentication.

[0681] 3. Display of work procedures

[0682] The server sends the instruction "First, pick product B from shelf number A1" to the terminal.

[0683] The device displays the location of shelf number A1 on the smart glasses' display and provides voice guidance.

[0684] 4. Work monitoring and feedback

[0685] Worker A begins to take out product B from shelf A1.

[0686] The device sends the work status to the server via the camera and sensors.

[0687] 5. Recognizing emotional states

[0688] The emotion engine analyzes worker A's facial expressions and tone of voice and determines that he is under stress.

[0689] 6. Warnings and Emotional Messages

[0690] If the server detects an incorrect operation, it generates a warning message saying, "Please remove product B from shelf A1."

[0691] The emotion engine generates an emotion-responsive message: "Relax and take a deep breath."

[0692] The device displays warnings and emotion-responsive messages on the smart glasses and provides voice guidance.

[0693] 7. Record and analyze work time

[0694] After worker A correctly picks up product B, the server records the task completion time.

[0695] The server compares the time with the standard work time and evaluates the skill level of worker A.

[0696] 8. Skill level assessment and support proposal generation

[0697] The server displays "tips for improving work efficiency before proceeding to the next step" on the terminal and provides audio explanations.

[0698] 9. Start of the next process

[0699] Worker A starts the next work process and proceeds with the work while referring to the support proposal and emotional response message.

[0700] A specific example is, "The user's facial expression is captured by the camera. Analyze the emotion from the facial expression and provide appropriate feedback. Check the on-site work steps and whether there are any errors, and update the instructions in real time."

[0701] This allows workers to perform new tasks with confidence, preventing operational errors and improving their skills. In addition, the emotion engine provides appropriate feedback according to the worker's emotional state, which is expected to further improve the work environment and work efficiency.

[0702] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0703] Step 1:

[0704] Initial Setup

[0705] When the server starts up, it reads the work process database, worker profile data, and emotional state data. It uses the data obtained from each database as input. For data processing, the work procedure data includes each work step, standard work time, and error conditions. As output, this data is stored in the server memory.

[0706] Step 2:

[0707] User Authentication

[0708] When the terminal starts up, the user enters authentication information on the login screen. The user ID and password are used as input. The server verifies the authentication information and retrieves the worker's profile. The data calculation compares the authentication information with the information in the database. The output returns a status of authentication success or failure.

[0709] Step 3:

[0710] Displaying work procedures

[0711] Once authentication is complete, the server sends the work procedure information to the terminal. As input, the work procedure data obtained from the work process database is used. As data processing, the current work step is extracted and sent to the terminal. The terminal visually displays the work procedure on the smart glasses display and provides audio guidance. As output, the work procedure is generated and displayed to the user.

[0712] Step 4:

[0713] Work monitoring and feedback

[0714] When a user starts a task, the device's camera and sensors monitor the progress of the task. Camera footage and sensor data are used as input. The device sends this data to the server in real time. For data calculation, the server analyzes the monitoring data and checks for any errors. Feedback data is created as output.

[0715] Step 5:

[0716] Recognition of emotional states

[0717] The emotion engine receives and analyzes data from the device's emotion recognition sensor. Facial expression data and voice data are used as input. The server then processes the data and determines the user's emotional state based on their visual expressions, voice tone, and physical movements. The output is the user's emotional state.

[0718] Step 6:

[0719] Warnings and Emotional Messages

[0720] If an incorrect operation is detected, the server generates a warning message. The emotion engine generates messages to instruct users to relax or to increase their motivation according to their emotional state. Monitoring data and emotional state data are used as input. Appropriate messages are generated as data processing. As output, warnings and emotion-related messages are generated and displayed on the terminal.

[0721] Step 7:

[0722] Record and analyze work time

[0723] The completion time of each work step is sent from the terminal to the server. The timestamp of the work step completion is used as input. The data is calculated by comparing it with the standard work time to evaluate the user's skill level. Skill evaluation data is generated as output.

[0724] Step 8:

[0725] Skill level assessment and support suggestions

[0726] The server generates support proposals for skill improvement based on the user's skill level. Skill evaluation data is used as input. Data processing generates support proposals according to the user's skill level. As output, the support proposals are sent to the terminal along with instructions for the next work.

[0727] Step 9:

[0728] Start of the next process

[0729] The user starts the next work step and proceeds with the work, referring to the support suggestions and emotion-responsive messages. The new work steps and support suggestions are used as input. Data processing provides this information to the user in an appropriate format. The output is an update of the work progress.

[0730] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0731] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0732] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0733] [Second embodiment]

[0734] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0735] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0736] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0737] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0738] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0739] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0740] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0741] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0742] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0743] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0744] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0745] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0746] MODE FOR CARRYING OUT THE INVENTION

[0747] The present invention is a system that utilizes an AI module and a wearable device to navigate workers in real time and improve productivity. Specific embodiments and program processing of the system are described below.

[0748] System Configuration

[0749] The system consists of the following main components:

[0750] 1. Server

[0751] Access to the work process database

[0752] Managing work steps

[0753] Analysis of work monitoring data

[0754] Generate feedback and warnings

[0755] Manage worker profiles

[0756] 2. Wearable devices

[0757] Visual display of work instructions (e.g. audio glasses)

[0758] Voice output of work instructions

[0759] User authentication function

[0760] Work monitoring using cameras and sensors

[0761] 3. Users

[0762] Check and implement work instructions

[0763] Receiving and responding to feedback

[0764] Implementing correction work

[0765] Program processing overview

[0766] Explain how the components of the system work together.

[0767] 1. Initial Server Setup

[0768] When the server starts up, it reads the work process database and worker profile data.

[0769] The work procedure data includes work steps, standard work times, and error conditions.

[0770] 2. User Authentication

[0771] When the device boots up, the user enters authentication information on a login screen.

[0772] The server verifies the credentials and retrieves the worker's profile.

[0773] 3. Display of work procedures

[0774] Once the authentication is complete, the server sends the work procedure information to the terminal.

[0775] The device visually displays the work procedure on the audio glasses and provides audio guidance.

[0776] 4. Work monitoring and feedback

[0777] Once the user begins work, the device's camera and sensors monitor the progress of the work.

[0778] The monitoring data is sent to the server in real time.

[0779] The server analyzes the monitoring data and checks for any incorrect operations.

[0780] 5. Warnings and Corrections

[0781] If an erroneous operation is detected, the server generates a warning message.

[0782] The device will display a visual warning on the audio glasses and provide audio instructions on how to fix the problem.

[0783] 6. Record and analyze work time

[0784] The completion time of each work step is transmitted from the terminal to the server.

[0785] The server evaluates the skill level of the worker by comparing with the standard work time.

[0786] Support proposals for skill development are generated and sent to the terminal along with the next work instructions.

[0787] Specific examples

[0788] The operation of the system will be explained below based on a specific warehouse work scenario.

[0789] 1. Initial Server Setup

[0790] The server reads the "picking work procedure data" and "worker A's profile data."

[0791] 2. User Authentication

[0792] Worker A enters login information on the terminal, and the server performs authentication.

[0793] 3. Display of work procedures

[0794] The server sends the instruction "First, pick product B from shelf number A1" to the terminal.

[0795] The terminal displays the location of shelf number A1 on the audio glasses display and provides voice guidance.

[0796] 4. Work monitoring and feedback

[0797] Worker A begins to take out product B from shelf A1.

[0798] The device sends the work status to the server via the camera and sensors.

[0799] 5. Warnings and Corrections

[0800] The server detects an incorrect operation (e.g., trying to remove product C from shelf A2).

[0801] The terminal will provide visual and audio guidance, saying, "Please take product B from shelf A1."

[0802] 6. Record and analyze work time

[0803] After worker A correctly picks up product B, the server records the task completion time.

[0804] The server compares the time with the standard work time and evaluates the skill level of worker A.

[0805] "Tips for improving work efficiency before proceeding to the next step" are displayed on the device and explained in audio.

[0806] This allows workers to perform new tasks with confidence, preventing operational errors and improving skills.

[0807] The processing flow will be explained below.

[0808] Program processing flow

[0809] Step 1: Initial Server Setup

[0810] server

[0811] The server starts up and loads the work process database and worker profile data.

[0812] The work procedure data includes work steps, standard work times, and error conditions.

[0813] The server logs that initialization is complete.

[0814] Step 2: Authenticate the user

[0815] Terminal

[0816] The device (Audio Glasses) will start up and the login screen will be displayed.

[0817] The user enters authentication information (user ID and password).

[0818] server

[0819] The server receives the authentication information and checks it against a database.

[0820] If the authentication is successful, the worker's profile information is acquired and sent to the terminal.

[0821] Step 3: View the work instructions

[0822] server

[0823] The server transmits current work process information to the terminal.

[0824] Terminal

[0825] The device displays the work procedure on the audio glasses display as a video and provides audio instructions.

[0826] For example, the instruction displayed might be, "Please pick product B from shelf number A1."

[0827] Step 4: Getting started

[0828] User

[0829] The user follows the instructions and starts the task. For example, the user goes to shelf A1 and takes out product B.

[0830] Terminal

[0831] The device's camera and sensors monitor the user's movements in real time and send the monitoring data to a server.

[0832] Step 5: Operational monitoring and analysis

[0833] server

[0834] The server receives the monitoring data and the AI ​​module analyzes the work progress.

[0835] Determine whether any malfunctions or abnormalities are detected.

[0836] Step 6: Detect and warn of misoperation

[0837] server

[0838] If the server detects an error, it generates a warning message.

[0839] Terminal

[0840] The device will display a visual warning message on the audio glasses and issue an audio warning.

[0841] For example, the system will guide you on how to correct the error by saying, "Please take product B from shelf A1."

[0842] User

[0843] The user checks the warning and corrects the error as instructed. For example, the user picks up the correct product from shelf A1.

[0844] Step 7: Notify and record completion of work

[0845] Terminal

[0846] After the user has correctly completed the task, the terminal sends a task completion notification to the server.

[0847] server

[0848] The server records the time the task is completed.

[0849] Evaluate user skill levels by comparing them with standard work hours.

[0850] Step 8: Assess skill levels and generate support suggestions

[0851] server

[0852] The server generates support proposals for skill improvement based on the user's skill level.

[0853] Send support suggestions to the device along with the next work instructions.

[0854] Terminal

[0855] The device displays the next steps on the audio glasses and provides audio instructions.

[0856] Provide users with visual and audio cues to improve their skills.

[0857] Step 9: Start the next process

[0858] User

[0859] The user starts the next work process and proceeds with the work while referring to the support proposal.

[0860] The above is the specific flow of each processing step of the program. This allows workers to perform new tasks with confidence, prevents operational errors, and improves their skills.

[0861] Example 1

[0862] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0863] Conventional work navigation systems lacked the ability to detect and correct worker errors in real time. They also lacked a mechanism for providing guidance and feedback tailored to each worker's skill level, making it difficult to improve productivity and work quality. Furthermore, it was difficult to provide customized work procedures based on user authentication and profiles, and they could only provide uniform guidance.

[0864] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0865] In this invention, the server includes means for accessing the work process database and managing each work step, wearable terminal means for outputting instructions related to the work process visually and audibly, AI module means for monitoring the worker's operations in real time and providing feedback via the wearable terminal means, control means for issuing a visual and audio warning and preventing the worker from proceeding to the next step if an incorrect operation is detected, evaluation means for recording the completion time of each work step and comparing it with a standard work time to evaluate the worker's skill level, means for receiving user authentication information and providing a customized work procedure based on the worker profile, and control means for analyzing monitoring data acquired in real time and providing appropriate feedback to the worker if an incorrect operation is detected. This makes it possible to detect and correct incorrect operations, provide customized guidance according to the worker's skill, and provide feedback in real time.

[0866] The "work process database" is a database for storing and managing information about work procedures and processes.

[0867] "Work steps" refers to the set of instructions or procedures required to accomplish a particular work process.

[0868] A "wearable terminal" is an information display and input device that can be worn by a worker and has the function of issuing visual and audio instructions.

[0869] The "AI module" is a system component that uses artificial intelligence technology to perform data analysis and pattern recognition, and provides appropriate feedback to workers.

[0870] A "controller" is a hardware or software function that governs the operation of the entire system and performs appropriate actions under specific conditions.

[0871] The "evaluation means" is a function for quantitatively evaluating the performance of a worker and evaluating the skill level by comparing it with the standard work time.

[0872] "User authentication information" refers to information used to identify a user, such as an ID or password entered when accessing a system.

[0873] A "worker profile" is data that includes detailed information about each individual worker, such as the worker's skill level and past work history.

[0874] A "customized work procedure" is a work procedure that is individually generated based on a worker profile and is adjusted according to the worker's skills and needs.

[0875] "Real-time monitoring data" refers to data that is collected and analyzed in real time to track the movements and working conditions of workers.

[0876] "Feedback" refers to instructions and suggested corrections that the system provides to the worker, including information that helps them understand the progress of their work and correct mistakes.

[0877] MODE FOR CARRYING OUT THE INVENTION

[0878] The present invention provides a system that utilizes an AI module and a wearable device to navigate workers in real time and improve productivity. Specific embodiments of the system are described below.

[0879] System Configuration

[0880] The system consists of the following main components:

[0881] 1. Server

[0882] Access to the work process database

[0883] Management of each work step

[0884] Analysis of work monitoring data

[0885] Generate feedback and warnings

[0886] Manage worker profiles

[0887] 2. Wearable devices

[0888] Visual display of work instructions (e.g. audio glasses)

[0889] Voice output of work instructions

[0890] User authentication function

[0891] Work monitoring using cameras and sensors

[0892] 3. Users

[0893] Check and implement work instructions

[0894] Receiving and responding to feedback

[0895] Implementing correction work

[0896] Program processing overview

[0897] Explain how the components of the system work together.

[0898] 1. Initial Server Setup

[0899] When the server starts up, it reads the work process database and worker profile data.

[0900] The work procedure data includes work steps, standard work times, and error conditions.

[0901] 2. User Authentication

[0902] When the device boots up, the user enters authentication information on a login screen.

[0903] The server verifies the credentials and retrieves the worker's profile.

[0904] 3. Display of work procedures

[0905] Once the authentication is complete, the server sends the work procedure information to the terminal.

[0906] The device visually displays the work procedure on the audio glasses and begins providing audio guidance.

[0907] 4. Work monitoring and feedback

[0908] Once the user begins work, the device's camera and sensors monitor the progress of the work.

[0909] The monitoring data is sent to a server in real time and analyzed.

[0910] The server analyzes the monitoring data and checks for any incorrect operations.

[0911] 5. Warnings and Corrections

[0912] If an erroneous operation is detected, the server generates a warning message.

[0913] The device will display a visual warning on the audio glasses and provide audio instructions on how to fix the problem.

[0914] 6. Record and analyze work time

[0915] The completion time of each work step is transmitted from the terminal to the server.

[0916] The server evaluates the skill level of the worker by comparing with the standard work time.

[0917] Support proposals for skill development are generated and sent to the terminal along with the next work instructions.

[0918] Specific examples

[0919] The operation of the system will be explained below based on a specific warehouse work scenario.

[0920] (example)

[0921] 1. Initial Server Setup

[0922] The server reads the "picking work procedure data" and obtains the "profile data" of worker A.

[0923] 2. User Authentication

[0924] Worker A enters login information on the terminal, and the server performs authentication.

[0925] 3. Display of work procedures

[0926] The server sends the instruction "First, pick product B from shelf number A1" to the terminal.

[0927] The terminal displays "Shelf Number A1" on the audio glasses and provides voice guidance.

[0928] 4. Work monitoring and feedback

[0929] Worker A goes to shelf A1 and begins the action of taking out product B.

[0930] The device's camera captures the movement and sends it to the server.

[0931] The server analyzes the monitoring data to determine the progress of the work.

[0932] 5. Warnings and Corrections

[0933] The server detects the incorrect operation and generates a "warning message."

[0934] The terminal will provide a visual warning saying "Please take product B from shelf A1" and also provide audio guidance.

[0935] 6. Record and analyze work time

[0936] The terminal sends the "task completion time" to the server.

[0937] The server compares the data with the "standard work time data" to evaluate the skill level of worker A.

[0938] It generates "hints for improving work efficiency along with the next step" and sends them to the terminal.

[0939] The system utilizes an AI module to monitor worker behavior in real time and improve work quality. It uses a generative AI model to generate appropriate prompts and provide timely instructions and feedback to workers.

[0940] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0941] Step 1:

[0942] Initial Server Configuration

[0943] Input: When the server starts, it reads the work process database and worker profile data.

[0944] Specific behavior:

[0945] The server retrieves "picking work procedure data" from the "work process database," which includes shelf numbers, product IDs, and operation procedures.

[0946] The server obtains worker A's skill level and past work history from the "worker profile data."

[0947] Output: The work procedure data and worker profile data are loaded, and the system completes its initial setup.

[0948] Step 2:

[0949] User Authentication

[0950] Input: The device boots up and the user enters their credentials at the login screen.

[0951] Specific behavior:

[0952] The terminal displays a login dialog and the user enters their "ID" and "password."

[0953] The server verifies the authentication information through the "User Authentication API."

[0954] If the authentication is successful, the server obtains the "worker profile data."

[0955] Output: User authentication is successful and worker profile data is retrieved.

[0956] Step 3:

[0957] Displaying work procedures

[0958] Input: After successful user authentication, the server sends the work procedure information to the terminal.

[0959] Specific behavior:

[0960] The server sends the "work procedure information" to the terminal.

[0961] The terminal displays on the audio glasses, "Please pick product B from shelf number A1."

[0962] The device will provide similar instructions via voice prompts.

[0963] Output: The work procedure is displayed on the terminal and the user is guided by voice.

[0964] Step 4:

[0965] Work monitoring and feedback

[0966] Input: The user starts a task, and the device's camera and sensors monitor the task's progress.

[0967] Specific behavior:

[0968] The user begins to move towards "shelf number A1" to take out product B.

[0969] The device's camera and sensors capture its movements and transmit "surveillance data" to a server in real time.

[0970] The server analyzes real-time monitoring data and compares it with prescribed work procedures.

[0971] Output: The monitoring data is sent to the server in real time and the analysis results are obtained.

[0972] Step 5:

[0973] Warnings and fixes

[0974] Input: Detects misoperation using monitoring data analyzed by the server.

[0975] Specific behavior:

[0976] If the server detects an "incorrect operation", it generates a "warning message".

[0977] For example, if a user attempts to remove product C from shelf A2, the server generates a "warning message."

[0978] The terminal will provide a visual warning and audio guidance saying, "Please take product B from shelf A1."

[0979] Output: Information is provided to help the user recognize and correct the error.

[0980] Step 6:

[0981] Record and analyze work time

[0982] Input: As each work step is completed, the completion time is sent from the terminal to the server.

[0983] Specific behavior:

[0984] The terminal sends the "task completion time" to the server.

[0985] The server compares the data with the "standard work time data" and evaluates the skill level of worker A.

[0986] As a result of the analysis, the server generates "hints for improving work efficiency along with the next step" and sends them to the terminal.

[0987] Output: The task completion time is recorded, the worker's skill level is evaluated, and hints for the next step are provided.

[0988] Through these steps, the system monitors the worker's actions in real time and provides appropriate feedback and instructions, thereby improving productivity and work quality.

[0989] (Application example 1)

[0990] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0991] Conventional work management systems have been plagued by problems such as operator errors and reduced work efficiency. Real-time monitoring and appropriate feedback were lacking, making it difficult to improve work accuracy and skills. Furthermore, customized guides for improving work efficiency were not provided. The present invention aims to solve these problems and improve worker productivity and accuracy.

[0992] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0993] In this invention, the server includes: means for accessing the work process database and managing each work step; wearable terminal means for outputting instructions related to the work process visually and audibly; AI module means for monitoring the worker's operations in real time and providing feedback via the wearable terminal means; control means for issuing visual and audio warnings and preventing the worker from proceeding to the next step if an incorrect operation is detected; evaluation means for recording the completion time of each work step and comparing it with a standard work time to evaluate the worker's skill level; means for providing visual and audio guides of the work instructions; means with a camera function for monitoring the user's actions in real time; and audio output means for generating audio guides. This makes it possible to monitor the worker's operations in real time and provide immediate feedback and guidance on the correct operation method if an incorrect operation occurs.

[0994] A "work process database" is a database for managing and storing information about the procedures and steps of a specific work.

[0995] A "visual display" is a means of presenting information in a form that can be seen by a user, and includes, for example, a display or a projector.

[0996] "Audio output" refers to a means of providing information to a user by voice using a speaker, headphones, or the like.

[0997] A "wearable device" is an electronic device that can be worn on the body and used, such as smart glasses and head-mounted displays.

[0998] "Real-time monitoring" is the process of checking the situation and collecting data at the exact moment that a task or operation is being performed.

[0999] An "AI module" is a software or hardware module that uses artificial intelligence algorithms to analyze data and make decisions.

[1000] "Feedback" is a means of indicating errors and areas for improvement by providing real-time evaluation and guidance on operations and actions.

[1001] A "warning" is a visual or audio notification that alerts the user to a risk or error and prompts caution.

[1002] A "control means" is a component equipped with the functions and algorithms necessary to monitor the entire system and provide appropriate operation and feedback.

[1003] The "evaluation means" is a function that evaluates the skill level of a worker by recording the completion time and work quality of each work step and comparing them with the standard work time.

[1004] The "camera function" is a means of collecting visual information and recording and analyzing it as images or videos.

[1005] "Voice guide" is a function that provides appropriate instructions and guidance to workers using voice.

[1006] The present invention is a system that utilizes an AI module and a wearable device to navigate workers in real time and improve productivity. Specific embodiments and program processing of the system are described below.

[1007] The system that realizes this application example is created based on the following program.

[1008] Program processing explanation

[1009] Initial Setup

[1010] The server accesses the work process database to manage each work step. The work process data includes work steps, standard work times, and error conditions. Worker profile data is also loaded.

[1011] User Authentication

[1012] The user enters login information on the terminal, the server verifies the authentication information, and if authentication is successful, retrieves the user's work profile.

[1013] View work orders

[1014] Once authenticated, the server sends customized work instructions to the device, which then displays the instructions visually and provides audio guidance via smart glasses or a head-mounted display. Google Text-to-Speech (gTTS) is used for voice output.

[1015] Work monitoring and feedback

[1016] When the user starts working, the device's built-in camera (using OpenCV) and sensors monitor the work status in real time. The monitoring data is sent to a server, where an AI module analyzes the data and checks for any operational errors.

[1017] Warnings and fixes

[1018] If an incorrect operation is detected, the server generates a warning message and provides visual and audio feedback to the operator via the terminal, allowing the operator to correct the incorrect operation.

[1019] Record and analyze work time

[1020] The completion time for each task is sent from the terminal to the server, which then compares it with the standard work time. This allows the worker's skill level to be evaluated, and the results are reflected in subsequent work instructions.

[1021] Specific examples

[1022] For example, consider a warehouse operation at a logistics center. When a worker is tasked with new picking work, the following scenario can be envisioned.

[1023] The server loads the "picking work procedure data" and "worker A's profile data." Worker A enters his / her login information on the terminal, and the server authenticates him / her. After authentication, the server sends the instruction "First, pick product B from shelf number A1" to the terminal. The terminal then displays the location of shelf number A1 on the smart glasses' display and provides audio guidance.

[1024] When worker A goes to shelf A1 and tries to take item B, the device's camera and sensors monitor the action and send the situation to the server in real time. If the server detects an incorrect operation (e.g., trying to take item C from shelf A2), the device will provide visual and audio guidance saying, "Please take item B from shelf A1." After correctly taking item B, the server records the task completion time and compares it with the standard work time to evaluate the skill level.

[1025] An example prompt is:

[1026] Example prompt

[1027] user_info = authenticate_user("new_worker", "securepass")

[1028] tasks = get_task_instructions(user_info['user_id'])

[1029] for task in tasks:

[1030] print(f"Next task: {task['instruction']}")

[1031] play_audio(task['instruction'])

[1032] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1033] Step 1:

[1034] The server accesses the work process database and reads each work step and worker profile data.

[1035] Input: Work process database, worker profile data

[1036] Data calculation: Retrieving information from a database.

[1037] Output: Loaded work procedure data, worker profile data

[1038] Step 2:

[1039] The user enters login information at the terminal and the server verifies the authentication information.

[1040] Input: User login information

[1041] Data calculation: Checks credentials and retrieves worker profile.

[1042] Output: Authenticated user's work profile

[1043] Step 3:

[1044] The server transmits customized work procedure information to the terminal for the authenticated user.

[1045] Input: Authenticated user's work profile

[1046] Data calculation: Generate customized work procedures based on work profiles

[1047] Output: Customized routing information

[1048] Step 4:

[1049] The device visually displays work procedures and provides audio guidance via smart glasses or a head-mounted display.

[1050] Input: Customized routing information

[1051] Data calculation: Visual display and speech translation of work instructions (using gTTS)

[1052] Output: Visually displayed work instructions, audio guide

[1053] Step 5:

[1054] The user starts working, and the camera (using OpenCV) and sensors on the device monitor the work status in real time.

[1055] Input: User's work actions

[1056] Data Calculation: Analysis of camera footage and sensor data

[1057] Output: Monitoring data (real-time)

[1058] Step 6:

[1059] The device sends the collected monitoring data to a server, which then uses an AI module to analyze the data and check for any operational errors.

[1060] Input: Monitoring data

[1061] Data calculation: Data analysis and error detection by AI module

[1062] Output: Misoperation detection result

[1063] Step 7:

[1064] If an incorrect operation is detected, the server generates a warning message and provides visual and audio feedback to the user via the terminal.

[1065] Input: Misoperation detection result

[1066] Data Calculation: Generate warning messages and provide visual and audio feedback

[1067] Output: Warning messages, visual and audio feedback

[1068] Step 8:

[1069] The server records the completion time of each work step, compares it with the standard work time to evaluate the skill level, and reflects the results in subsequent work instructions.

[1070] Input: Completion time for each work step

[1071] Data calculation: Comparison with standard work time and skill level evaluation

[1072] Output: Skill level evaluation results, data to be reflected in the next work instructions

[1073] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1074] MODE FOR CARRYING OUT THE INVENTION

[1075] The present invention is a system that utilizes an AI module, a wearable device, and an emotion engine to navigate workers in real time and provide feedback according to the worker's emotional state, thereby improving productivity. Specific embodiments and program processing will be described below.

[1076] System Configuration

[1077] The system consists of the following main components:

[1078] 1. Server

[1079] Access to the work process database

[1080] Managing work steps

[1081] Analysis of work monitoring data and emotion data

[1082] Generating feedback, warnings, and emotion-responsive messages

[1083] Manage worker profiles

[1084] 2. Wearable devices

[1085] Visual and audio display of work instructions (e.g. audio glasses)

[1086] User authentication function

[1087] Work monitoring using cameras and sensors

[1088] Emotion recognition sensors (e.g. facial recognition cameras, voice analysis microphones)

[1089] 3. Users

[1090] Check and implement work instructions

[1091] Receiving and responding to feedback and emotional responses

[1092] Implementing correction work

[1093] 4. Emotion Engine

[1094] Data analysis from emotion recognition sensors

[1095] Determining the user's emotional state

[1096] Message generation according to emotional state

[1097] Program processing overview

[1098] Explain how the components of the system work together.

[1099] 1. Initial Server Setup

[1100] When the server starts up, it loads the work process database, the worker profile data, and the emotional state data.

[1101] The work procedure data includes work steps, standard work times, and error conditions.

[1102] 2. User Authentication

[1103] When the device boots up, the user enters authentication information on a login screen.

[1104] The server verifies the credentials and retrieves the worker's profile.

[1105] 3. Display of work procedures

[1106] Once the authentication is complete, the server sends the work procedure information to the terminal.

[1107] The device visually displays the work procedure on the audio glasses display and provides audio guidance.

[1108] 4. Work monitoring and feedback

[1109] Once the user begins work, the device's camera and sensors monitor the progress of the work.

[1110] The monitoring data is sent to the server in real time.

[1111] The server analyzes the monitoring data and checks for any incorrect operations.

[1112] 5. Recognizing emotional states

[1113] The emotion engine receives and analyzes data from the device's emotion recognition sensor.

[1114] The server determines the user's emotional state based on their visual expressions, vocal tone, and physical movements.

[1115] 6. Warnings and Emotional Messages

[1116] If an erroneous operation is detected, the server generates a warning message.

[1117] The emotion engine generates relaxation instructions and motivational messages according to the user's emotional state.

[1118] The device will display a visual warning on the audio glasses and provide audio instructions on how to fix the problem.

[1119] 7. Record and analyze work time

[1120] The completion time of each work step is transmitted from the terminal to the server.

[1121] The server evaluates the skill level of the user by comparing it with standard work time.

[1122] 8. Skill level assessment and support proposal generation

[1123] The server generates support proposals for skill improvement based on the user's skill level.

[1124] Send support suggestions to the device along with the next work instructions.

[1125] 9. Start of the next process

[1126] The user starts the next work process and proceeds with the work while referring to the support suggestions and emotional response messages.

[1127] Specific examples

[1128] The operation of the system will be explained below based on a specific warehouse work scenario.

[1129] 1. Initial Server Setup

[1130] The server reads the "picking work procedure data" and "worker A's profile data."

[1131] 2. User Authentication

[1132] Worker A enters login information on the terminal, and the server performs authentication.

[1133] 3. Display of work procedures

[1134] The server sends the instruction "First, pick product B from shelf number A1" to the terminal.

[1135] The terminal displays the location of shelf number A1 on the audio glasses display and provides voice guidance.

[1136] 4. Work monitoring and feedback

[1137] Worker A begins to take out product B from shelf A1.

[1138] The device sends the work status to the server via the camera and sensors.

[1139] 5. Recognizing emotional states

[1140] The emotion engine analyzes worker A's facial expressions and tone of voice and determines that he is under stress.

[1141] 6. Warnings and Emotional Messages

[1142] If the server detects an incorrect operation, it generates a warning message saying, "Please remove product B from shelf A1."

[1143] The emotion engine generates an emotion-responsive message: "Relax and take a deep breath."

[1144] The device displays warnings and emotion-responsive messages on the audio glasses and provides voice guidance.

[1145] 7. Record and analyze work time

[1146] After worker A correctly picks up product B, the server records the task completion time.

[1147] The server compares the time with the standard work time and evaluates the skill level of worker A.

[1148] 8. Skill level assessment and support proposal generation

[1149] The server displays "tips for improving work efficiency before proceeding to the next step" on the terminal and provides audio explanations.

[1150] 9. Start of the next process

[1151] Worker A starts the next work process and proceeds with the work while referring to the support proposal and emotional response message.

[1152] This allows workers to perform new tasks with confidence, preventing operational errors and improving their skills. In addition, the emotion engine provides appropriate feedback according to the worker's emotional state, which is expected to further improve the work environment and work efficiency.

[1153] The processing flow will be explained below.

[1154] Program processing flow

[1155] Step 1: Initial Server Setup

[1156] server

[1157] The server starts up and loads the work process database, worker profile data, and emotional state data.

[1158] The work procedure data includes work steps, standard work times, and error conditions.

[1159] The server logs that initialization is complete.

[1160] Step 2: Authenticate the user

[1161] Terminal

[1162] The device (Audio Glasses) will start up and the login screen will be displayed.

[1163] The user enters authentication information (user ID and password).

[1164] server

[1165] The server receives the authentication information and checks it against a database.

[1166] If the authentication is successful, the worker's profile information is acquired and sent to the terminal.

[1167] Step 3: View the work instructions

[1168] server

[1169] The server transmits current work process information to the terminal.

[1170] Terminal

[1171] The device displays the work procedure on the audio glasses display as a video and provides audio instructions.

[1172] For example, the instruction displayed might be, "Please pick product B from shelf number A1."

[1173] Step 4: Getting started

[1174] User

[1175] The user follows the instructions and starts the task. For example, the user goes to shelf A1 and takes out product B.

[1176] Terminal

[1177] The device's camera and sensors monitor the user's movements in real time and send the monitoring data to a server.

[1178] Step 5: Operational monitoring and analysis

[1179] server

[1180] The server receives the monitoring data and the AI ​​module analyzes the work progress.

[1181] Determine whether any malfunctions or abnormalities are detected.

[1182] Step 6: Recognize your emotional state

[1183] Terminal

[1184] The device's emotion recognition sensors (e.g., facial expression recognition camera, voice analysis microphone) capture the user's visual expressions, voice tone, and physical movements.

[1185] The emotion data is sent to the server.

[1186] server

[1187] The server uses an emotion engine to analyze the user's emotional state.

[1188] Through analysis, it is determined that the user is feeling stressed.

[1189] Step 7: Misoperation detection and warning

[1190] server

[1191] If the server detects an incorrect operation, it generates a warning message saying, "Please remove product B from shelf A1."

[1192] Terminal

[1193] The device will visually display a warning message on the audio glasses and issue an audio warning.

[1194] User

[1195] The user checks the warning and corrects the error as instructed. For example, the user picks up the correct product from shelf A1.

[1196] Step 8: Generate and display emotion-aware messages

[1197] server

[1198] The emotion engine generates an emotion-responsive message such as "Relax and take a deep breath" depending on the user's emotional state.

[1199] The server sends an emotion-related message to the terminal.

[1200] Terminal

[1201] The device visually displays emotion-based messages on the audio glasses and provides audio guidance.

[1202] Step 9: Notify and record completion of work

[1203] Terminal

[1204] After the user has correctly completed the task, the terminal sends a task completion notification to the server.

[1205] server

[1206] The server records the time the task is completed.

[1207] Evaluate user skill levels by comparing them with standard work hours.

[1208] Step 10: Assess skill levels and generate support suggestions

[1209] server

[1210] The server generates support proposals for skill improvement based on the user's skill level.

[1211] Send support suggestions to the device along with the next work instructions.

[1212] Terminal

[1213] The device displays the next steps and tips for improving skills on the audio glasses and provides audio guidance.

[1214] Step 11: Start the next process

[1215] User

[1216] The user starts the next work process and proceeds with the work while referring to the support suggestions and emotional response messages.

[1217] The above is the specific flow of each processing step of the program that combines the emotion engine. This allows workers to work with peace of mind while receiving appropriate feedback according to their emotional state. This prevents operational errors, improves skills, and increases work efficiency.

[1218] Example 2

[1219] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1220] Conventional work support systems often focus on providing immediate feedback to workers regarding operational errors and improving work efficiency, but lack a mechanism to respond to the worker's emotional state. As a result, worker stress and a decline in motivation can have a negative impact on work efficiency and quality. Furthermore, it has been difficult to effectively navigate by combining real-time emotional feedback and work instructions.

[1221] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for accessing the work process database and managing each work step; wearable terminal means for outputting instructions related to the work process visually and audibly; AI module means for monitoring the worker's operations in real time and providing feedback via the wearable terminal means; control means for issuing visual and audio warnings and preventing the worker from proceeding to the next step when an erroneous operation is detected; emotion engine means for analyzing data from an emotion recognition sensor and determining the worker's emotional state; means for generating messages according to the emotional state and providing them via the wearable terminal means; and evaluation means for recording the completion time of each work step and comparing it with the standard work time to evaluate the worker's skill level. This makes it possible to comprehensively manage the worker's real-time work status and emotional state and provide appropriate feedback and emotion-responsive messages.

[1222] The "work process database" is a database for recording and managing information about each work step.

[1223] A "wearable terminal" is a device that can be worn by a user and provides work instructions and feedback.

[1224] An "AI module" is a device that includes an artificial intelligence program that monitors the operations of workers in real time and provides feedback based on the analysis results.

[1225] "Control means" refers to a device or function that detects operational errors, issues visual and audio warnings as a countermeasure, and controls the progress of work.

[1226] An "emotion recognition sensor" is a sensor that detects a worker's facial expression, tone of voice, physical movements, etc., and determines their emotional state.

[1227] An "emotion engine" is software or hardware that analyzes data from emotion recognition sensors and determines the user's emotional state.

[1228] The "evaluation means" is a device or function that records the time required to complete each work step and compares it with the standard work time to evaluate the skill level of the worker.

[1229] "Feedback" refers to information or instructions that provide an evaluation of the worker's operations and areas for improvement.

[1230] "Emotion-responsive messages" are messages that instruct users to relax or increase their motivation, which are generated in response to the user's emotional state.

[1231] "Real-time" means processing or responding immediately with little or no delay.

[1232] MODE FOR CARRYING OUT THE INVENTION

[1233] The present invention provides a system for providing feedback and navigation according to the emotional state of a worker. The system includes a server, a wearable device, an AI module, an emotion recognition sensor, and an emotion engine.

[1234] System Configuration

[1235] The system consists of the following main elements:

[1236] 1. Server

[1237] Access the work process database and manage each work step.

[1238] Stores worker profile data and retrieves necessary information.

[1239] Monitor the progress of work steps in real time and generate feedback.

[1240] 2. Wearable devices

[1241] This device outputs work instructions visually and audibly. An example of such a device is audio glasses.

[1242] A login screen is provided for user authentication.

[1243] Work is monitored in real time using cameras and various sensors.

[1244] It is equipped with an emotion recognition sensor that analyzes the user's facial expressions and tone of voice.

[1245] 3. AI Module

[1246] Analyzes work progress and provides feedback if an error is detected.

[1247] 4. Emotion Recognition Sensor

[1248] Sensors for detecting the user's emotional state in real time include a facial recognition camera and a voice analysis microphone.

[1249] 5. Emotion Engine

[1250] Data from emotion recognition sensors is analyzed to determine the user's emotional state.

[1251] Messages are generated according to the emotional state and provided via a wearable device.

[1252] Program processing overview

[1253] The following describes how each component of the system works together.

[1254] Initial Server Configuration

[1255] When the server starts up, it loads the work process database and the worker profile database. The work process data includes the work procedure for each step, the standard work time, the list of required materials, error conditions, etc.

[1256] User Authentication

[1257] When a user enters their authentication information on the login screen of their wearable device, the device sends the information to the server, which verifies the authentication information and retrieves the worker's profile.

[1258] Displaying work procedures

[1259] Once authentication is complete, the server sends the work procedure information to the device, which then displays visual instructions on the audio glasses and provides audio guidance.

[1260] Work monitoring and feedback

[1261] As the user performs a task, the device's camera and sensors monitor the progress and send the data in real time to a server, which analyzes the data and provides feedback via the wearable device if any errors are detected.

[1262] Recognition of emotional states

[1263] The emotion engine analyzes emotion recognition sensor data from the device and determines the user's emotional state. Based on the results, the server generates an appropriate emotion-responsive message and sends it to the device.

[1264] Warnings and Emotional Messages

[1265] If an incorrect operation is detected, the server generates a warning message, and the emotion engine simultaneously generates a message to instruct the user to relax or to increase motivation, which is then sent via the device.

[1266] Record and analyze work time

[1267] The completion time of each work step is sent from the terminal to a server, and the server compares the recorded data with standard work times to evaluate the skill level.

[1268] Skill level assessment and support suggestions

[1269] The server analyzes the user's skill level and generates support suggestions for skill improvement, which are then provided to the wearable device along with next work instructions.

[1270] Start of the next process

[1271] The user starts the next task, and the device displays support suggestions and emotional response messages to assist them.

[1272] Specific examples

[1273] The operation of the system will be explained below based on a specific warehouse work scenario.

[1274] 1. As the initial setting of the server, the picking work procedure data and worker profile data are read.

[1275] 2. The worker enters login information on the terminal, and the server performs authentication.

[1276] 3. Once authentication is complete, the instruction "First, please pick product B from shelf number A1" is sent to the terminal. The terminal displays the location on the audio glasses and gives instructions via voice.

[1277] 4. Once work begins, the device monitors progress using cameras and sensors and sends the data to the server.

[1278] 5. The emotion engine analyzes the worker's facial expressions and tone of voice and determines, for example, whether they are in a stressed state.

[1279] 6. If an incorrect operation is detected, the server generates a warning message and an emotion-based message such as "Relax and take a deep breath." This message is displayed on the device and guided by voice.

[1280] 7. The device records the time the task is completed, and the server analyzes the data.

[1281] 8. The server evaluates the skill level of the worker and provides tips to improve efficiency.

[1282] 9. The worker begins the next step and proceeds with the work while referring to the support suggestions and emotional response messages.

[1283] Example of input prompt for generative AI model

[1284] Below is an example of a prompt sentence to be input to the generative AI model.

[1285] "The goal of this system is to improve worker productivity. Please explain the following steps in natural language: The worker takes item B from shelf A1. If there is an error, detect the error and provide feedback for correction. Finally, recognize the worker's emotional state and generate an appropriate emotional response message."

[1286] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1287] Specific processing steps of the program

[1288] Step 1: Initial Server Setup

[1289] When the server starts up, it accesses the work process database and worker profile database and loads the necessary information into memory. The inputs are the work process database and worker profile database. The output is work procedure data (each work step, standard work time, error conditions) and worker profile data.

[1290] Specifically, the server acquires the work procedure data and the worker profile data from the database, analyzes them, and completes the initial settings.

[1291] Step 2: User authentication

[1292] The user enters their authentication information on the login screen of the wearable device. The device sends the information to the server, which checks the authentication information in the database and retrieves the user's profile if authentication is successful.

[1293] The input is user authentication information, and the output is profile data for a particular user.

[1294] Specifically, the user enters a username and password into the terminal, and the server returns the authentication result to the terminal.

[1295] Step 3: View the work instructions

[1296] For users who have been successfully authenticated, the server generates work instruction data and sends it to the terminal, which then displays the work procedure visually on the audio glasses and provides audio instructions.

[1297] The inputs are user profile data and work procedure data, and the output is a display of the work procedure and voice guidance.

[1298] Specifically, the server sends information about the next work step to the terminal, and the terminal displays the work procedure on the audio glasses and provides audio guidance.

[1299] Step 4: Work monitoring and feedback

[1300] When the user starts a task, the device's camera and various sensors monitor the progress of the task and send the data in real time to the server, which analyzes the data, checks for errors and smooth progress, and provides necessary feedback to the device.

[1301] The input is sensor data (work progress), and the output is feedback messages.

[1302] Specifically, the device's camera captures images of the user's hands, the location sensor tracks the progress of the task, and the acquired data is sent to a server, which analyzes the data and generates the necessary feedback.

[1303] Step 5: Recognizing your emotional state

[1304] The device's emotion recognition sensor captures the user's facial expressions and voice tone in real time, and the emotion engine analyzes the data. The server then determines the user's emotional state based on the results.

[1305] The input is emotion recognition sensor data (facial expressions, voice tone), and the output is the user's emotional state.

[1306] Specifically, the device's camera captures facial expressions and microphone collects voice tones, and then sends these data to the emotion engine, which analyzes the data and determines the user's emotional state.

[1307] Step 6: Warnings and Emotional Messages

[1308] If the server detects an incorrect operation, it generates a warning message, and the emotion engine generates a message according to the user's emotional state. These messages are sent to the terminal, which then displays and provides voice guidance.

[1309] The inputs are error data and emotional state data, and the output is a warning message and an emotional response message.

[1310] Specifically, when the server detects an incorrect operation, it generates a warning message saying, "An incorrect operation has occurred. Please return to the correct procedure." If the emotion engine determines that the user is in a stressful state, it outputs a message such as, "Relax and take a deep breath." The device displays these messages on the audio glasses and provides audio guidance.

[1311] Step 7: Record and analyze your work time

[1312] The device records the time it takes to complete each task and sends it to the server, which then compares it with the standard time to assess the user's skill level.

[1313] The input is task completion time data, and the output is a user skill level assessment.

[1314] Specifically, the device records the time when the user completes a task and sends the data to the server, which compares it with the standard work time and evaluates the skill level.

[1315] Step 8: Assess skill levels and generate support suggestions

[1316] The server generates support suggestions for the next step based on the user's skill level and sends them to the device, which then displays the suggestions and provides audio guidance.

[1317] The input is the skill level assessment result, and the output is a support proposal.

[1318] Specifically, the server evaluates the user's skill level, generates support suggestions such as "Points to help you work more efficiently in the next step," and sends them to the device. The device then displays the suggestions on the audio glasses and provides audio guidance.

[1319] Step 9: Start the next process

[1320] The user starts the next step of the work process, and the device proceeds with the work while referring to support suggestions and emotion-responsive messages.

[1321] The inputs are the next work instructions, support ideas, and emotion-responsive messages. The output is the progress of the next work process.

[1322] Specifically, the user reads the instructions for the next step, starts the work, and continues working efficiently while referring to support suggestions and emotional response messages from the device.

[1323] (Application example 2)

[1324] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1325] In the current work environment at logistics centers, work instructions are given without considering the emotional state of workers, which can lead to the accumulation of stress and fatigue, resulting in a decline in work efficiency and productivity. Another problem is that when an operational error occurs, a response is delayed, resulting in an increase in overall work time. The purpose of this invention is to solve these problems and improve work efficiency and productivity by providing appropriate feedback according to the emotional state of workers.

[1326] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for accessing a work process database and managing each work step; wearable terminal means for outputting instructions related to the work process visually and audibly; artificial intelligence module means for monitoring the worker's operations in real time and providing feedback via the wearable terminal means; control means for issuing a visual and audio warning and preventing the worker from proceeding to the next step if an erroneous operation is detected; emotion recognition means for analyzing the worker's facial expression and tone of voice to determine the worker's emotional state; emotion response feedback means for generating relaxation instructions or motivation-boosting messages according to the worker's emotional state and outputting them via the wearable terminal means; and evaluation means for recording the completion time of each work step and comparing it with the standard work time to evaluate the worker's skill level. This enables logistics centers to improve work efficiency and productivity while managing the emotions of workers.

[1327] The "work process database" is a database for managing information such as each work step and its procedure, standard work time, and conditions for operational errors.

[1328] "Wearable terminal means" refers to a device that can be worn by a worker and provides visual displays and audio instructions. For example, smart glasses and audio glasses are examples of such devices.

[1329] The "artificial intelligence module means" is a function that uses artificial intelligence to monitor the operator's operations in real time and provide feedback.

[1330] The "control means" is a function that issues a visual and audio warning when an erroneous operation is detected, and prevents the process from proceeding to the next step.

[1331] The "emotion recognition means" is a function for determining the emotional state of a worker by analyzing the worker's facial expression and tone of voice.

[1332] The "emotion-responsive feedback means" is a function for generating relaxation instructions and motivation-boosting messages according to the emotional state determined by the emotion recognition means, and outputting them on the wearable terminal means.

[1333] The "evaluation means" is a function for recording the completion time of each work step and comparing it with the standard work time to evaluate the skill level of the worker.

[1334] This invention is a system that combines artificial intelligence, wearable devices, and an emotion engine to improve the efficiency and productivity of workers at logistics centers. The system accesses a work process database, displays and outputs work instructions visually and audibly, monitors worker operations in real time, and issues a warning if an error is detected. It also analyzes the worker's facial expressions and tone of voice to determine their emotional state and provides feedback according to that state.

[1335] System Configuration

[1336] The system consists of the following main components:

[1337] 1. Server

[1338] Access to the work process database

[1339] Managing work steps

[1340] Analysis of work monitoring data and emotion data

[1341] Generating feedback, warnings, and emotion-responsive messages

[1342] Manage worker profiles

[1343] 2. Wearable devices

[1344] Visual and audio display of work instructions (e.g., smart glasses)

[1345] User authentication function

[1346] Work monitoring using cameras and sensors

[1347] Emotion recognition sensors (e.g. facial recognition cameras, voice analysis microphones)

[1348] 3. Users

[1349] Check and implement work instructions

[1350] Receiving and responding to feedback and emotional responses

[1351] Implementing correction work

[1352] 4. Emotion Engine

[1353] Data analysis from emotion recognition sensors

[1354] Determining the user's emotional state

[1355] Message generation according to emotional state

[1356] Program processing overview

[1357] Explain how the components of the system work together.

[1358] 1. Initial Server Setup

[1359] At startup, the server loads the work process database, worker profile data, and emotional state data.

[1360] The work procedure data includes work steps, standard work times, and error conditions.

[1361] 2. User Authentication

[1362] When the device boots up, the user enters authentication information on a login screen.

[1363] The server verifies the credentials and retrieves the worker's profile.

[1364] 3. Display of work procedures

[1365] Once the authentication is complete, the server sends the work procedure information to the terminal.

[1366] The device visually displays the work procedure on the smart glasses' display and provides audio guidance.

[1367] 4. Work monitoring and feedback

[1368] Once the user begins work, the device's camera and sensors monitor the progress of the work.

[1369] The monitoring data is sent to the server in real time.

[1370] The server analyzes the monitoring data and checks for any incorrect operations.

[1371] 5. Recognizing emotional states

[1372] The emotion engine receives and analyzes data from the device's emotion recognition sensor.

[1373] The server determines the user's emotional state based on their visual expressions, vocal tone, and physical movements.

[1374] 6. Warnings and Emotional Messages

[1375] If an erroneous operation is detected, the server generates a warning message.

[1376] The emotion engine generates relaxation instructions and motivational messages according to the user's emotional state.

[1377] The device will display a visual warning on the smart glasses and provide audio instructions on how to fix the problem.

[1378] 7. Record and analyze work time

[1379] The completion time of each work step is transmitted from the terminal to the server.

[1380] The server evaluates the skill level of the user by comparing it with standard work time.

[1381] 8. Skill level assessment and support proposal generation

[1382] The server generates support proposals for skill improvement based on the user's skill level.

[1383] Send support suggestions to the device along with the next work instructions.

[1384] 9. Start of the next process

[1385] The user starts the next work process and proceeds with the work while referring to the support suggestions and emotional response messages.

[1386] Specific examples

[1387] The operation of the system will be explained below based on a specific warehouse work scenario.

[1388] 1. Initial Server Setup

[1389] The server reads the "picking work procedure data" and "worker A's profile data."

[1390] 2. User Authentication

[1391] Worker A enters login information on the terminal, and the server performs authentication.

[1392] 3. Display of work procedures

[1393] The server sends the instruction "First, pick product B from shelf number A1" to the terminal.

[1394] The device displays the location of shelf number A1 on the smart glasses' display and provides voice guidance.

[1395] 4. Work monitoring and feedback

[1396] Worker A begins to take out product B from shelf A1.

[1397] The device sends the work status to the server via the camera and sensors.

[1398] 5. Recognizing emotional states

[1399] The emotion engine analyzes worker A's facial expressions and tone of voice and determines that he is under stress.

[1400] 6. Warnings and Emotional Messages

[1401] If the server detects an incorrect operation, it generates a warning message saying, "Please remove product B from shelf A1."

[1402] The emotion engine generates an emotion-responsive message: "Relax and take a deep breath."

[1403] The device displays warnings and emotion-responsive messages on the smart glasses and provides voice guidance.

[1404] 7. Record and analyze work time

[1405] After worker A correctly picks up product B, the server records the task completion time.

[1406] The server compares the time with the standard work time and evaluates the skill level of worker A.

[1407] 8. Skill level assessment and support proposal generation

[1408] The server displays "tips for improving work efficiency before proceeding to the next step" on the terminal and provides audio explanations.

[1409] 9. Start of the next process

[1410] Worker A starts the next work process and proceeds with the work while referring to the support proposal and emotional response message.

[1411] A specific example is, "The user's facial expression is captured by the camera. Analyze the emotion from the facial expression and provide appropriate feedback. Check the on-site work steps and whether there are any errors, and update the instructions in real time."

[1412] This allows workers to perform new tasks with confidence, preventing operational errors and improving their skills. In addition, the emotion engine provides appropriate feedback according to the worker's emotional state, which is expected to further improve the work environment and work efficiency.

[1413] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1414] Step 1:

[1415] Initial Setup

[1416] When the server starts up, it reads the work process database, worker profile data, and emotional state data. It uses the data obtained from each database as input. For data processing, the work procedure data includes each work step, standard work time, and error conditions. As output, this data is stored in the server memory.

[1417] Step 2:

[1418] User Authentication

[1419] When the terminal starts up, the user enters authentication information on the login screen. The user ID and password are used as input. The server verifies the authentication information and retrieves the worker's profile. The data calculation compares the authentication information with the information in the database. The output returns a status of authentication success or failure.

[1420] Step 3:

[1421] Displaying work procedures

[1422] Once authentication is complete, the server sends the work procedure information to the terminal. As input, the work procedure data obtained from the work process database is used. As data processing, the current work step is extracted and sent to the terminal. The terminal visually displays the work procedure on the smart glasses display and provides audio guidance. As output, the work procedure is generated and displayed to the user.

[1423] Step 4:

[1424] Work monitoring and feedback

[1425] When a user starts a task, the device's camera and sensors monitor the progress of the task. Camera footage and sensor data are used as input. The device sends this data to the server in real time. For data calculation, the server analyzes the monitoring data and checks for any errors. Feedback data is created as output.

[1426] Step 5:

[1427] Recognition of emotional states

[1428] The emotion engine receives and analyzes data from the device's emotion recognition sensor. Facial expression data and voice data are used as input. The server then processes the data and determines the user's emotional state based on their visual expressions, voice tone, and physical movements. The output is the user's emotional state.

[1429] Step 6:

[1430] Warnings and Emotional Messages

[1431] If an incorrect operation is detected, the server generates a warning message. The emotion engine generates messages to instruct users to relax or to increase their motivation according to their emotional state. Monitoring data and emotional state data are used as input. Appropriate messages are generated as data processing. As output, warnings and emotion-related messages are generated and displayed on the terminal.

[1432] Step 7:

[1433] Record and analyze work time

[1434] The completion time of each work step is sent from the terminal to the server. The timestamp of the work step completion is used as input. The data is calculated by comparing it with the standard work time to evaluate the user's skill level. Skill evaluation data is generated as output.

[1435] Step 8:

[1436] Skill level assessment and support suggestions

[1437] The server generates support proposals for skill improvement based on the user's skill level. Skill evaluation data is used as input. Data processing generates support proposals according to the user's skill level. As output, the support proposals are sent to the terminal along with instructions for the next work.

[1438] Step 9:

[1439] Start of the next process

[1440] The user starts the next work step and proceeds with the work, referring to the support suggestions and emotion-responsive messages. The new work steps and support suggestions are used as input. Data processing provides this information to the user in an appropriate format. The output is an update of the work progress.

[1441] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1442] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1443] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1444] [Third embodiment]

[1445] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1446] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1447] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1448] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1449] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1450] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1451] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1452] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1453] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1454] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1455] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1456] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1457] MODE FOR CARRYING OUT THE INVENTION

[1458] The present invention is a system that utilizes an AI module and a wearable device to navigate workers in real time and improve productivity. Specific embodiments and program processing of the system are described below.

[1459] System Configuration

[1460] The system consists of the following main components:

[1461] 1. Server

[1462] Access to the work process database

[1463] Managing work steps

[1464] Analysis of work monitoring data

[1465] Generate feedback and warnings

[1466] Manage worker profiles

[1467] 2. Wearable devices

[1468] Visual display of work instructions (e.g. audio glasses)

[1469] Voice output of work instructions

[1470] User authentication function

[1471] Work monitoring using cameras and sensors

[1472] 3. Users

[1473] Check and implement work instructions

[1474] Receiving and responding to feedback

[1475] Implementing correction work

[1476] Program processing overview

[1477] Explain how the components of the system work together.

[1478] 1. Initial Server Setup

[1479] When the server starts up, it reads the work process database and worker profile data.

[1480] The work procedure data includes work steps, standard work times, and error conditions.

[1481] 2. User Authentication

[1482] When the device boots up, the user enters authentication information on a login screen.

[1483] The server verifies the credentials and retrieves the worker's profile.

[1484] 3. Display of work procedures

[1485] Once the authentication is complete, the server sends the work procedure information to the terminal.

[1486] The device visually displays the work procedure on the audio glasses and provides audio guidance.

[1487] 4. Work monitoring and feedback

[1488] Once the user begins work, the device's camera and sensors monitor the progress of the work.

[1489] The monitoring data is sent to the server in real time.

[1490] The server analyzes the monitoring data and checks for any incorrect operations.

[1491] 5. Warnings and Corrections

[1492] If an erroneous operation is detected, the server generates a warning message.

[1493] The device will display a visual warning on the audio glasses and provide audio instructions on how to fix the problem.

[1494] 6. Record and analyze work time

[1495] The completion time of each work step is transmitted from the terminal to the server.

[1496] The server evaluates the skill level of the worker by comparing with the standard work time.

[1497] Support proposals for skill development are generated and sent to the terminal along with the next work instructions.

[1498] Specific examples

[1499] The operation of the system will be explained below based on a specific warehouse work scenario.

[1500] 1. Initial Server Setup

[1501] The server reads the "picking work procedure data" and "worker A's profile data."

[1502] 2. User Authentication

[1503] Worker A enters login information on the terminal, and the server performs authentication.

[1504] 3. Display of work procedures

[1505] The server sends the instruction "First, pick product B from shelf number A1" to the terminal.

[1506] The terminal displays the location of shelf number A1 on the audio glasses display and provides voice guidance.

[1507] 4. Work monitoring and feedback

[1508] Worker A begins to take out product B from shelf A1.

[1509] The device sends the work status to the server via the camera and sensors.

[1510] 5. Warnings and Corrections

[1511] The server detects an incorrect operation (e.g., trying to remove product C from shelf A2).

[1512] The terminal will provide visual and audio guidance, saying, "Please take product B from shelf A1."

[1513] 6. Record and analyze work time

[1514] After worker A correctly picks up product B, the server records the task completion time.

[1515] The server compares the time with the standard work time and evaluates the skill level of worker A.

[1516] "Tips for improving work efficiency before proceeding to the next step" are displayed on the device and explained in audio.

[1517] This allows workers to perform new tasks with confidence, preventing operational errors and improving skills.

[1518] The processing flow will be explained below.

[1519] Program processing flow

[1520] Step 1: Initial Server Setup

[1521] server

[1522] The server starts up and loads the work process database and worker profile data.

[1523] The work procedure data includes work steps, standard work times, and error conditions.

[1524] The server logs that initialization is complete.

[1525] Step 2: Authenticate the user

[1526] Terminal

[1527] The device (Audio Glasses) will start up and the login screen will be displayed.

[1528] The user enters authentication information (user ID and password).

[1529] server

[1530] The server receives the authentication information and checks it against a database.

[1531] If the authentication is successful, the worker's profile information is acquired and sent to the terminal.

[1532] Step 3: View the work instructions

[1533] server

[1534] The server transmits current work process information to the terminal.

[1535] Terminal

[1536] The device displays the work procedure on the audio glasses display as a video and provides audio instructions.

[1537] For example, the instruction displayed might be, "Please pick product B from shelf number A1."

[1538] Step 4: Getting started

[1539] User

[1540] The user follows the instructions and starts the task. For example, the user goes to shelf A1 and takes out product B.

[1541] Terminal

[1542] The device's camera and sensors monitor the user's movements in real time and send the monitoring data to a server.

[1543] Step 5: Operational monitoring and analysis

[1544] server

[1545] The server receives the monitoring data and the AI ​​module analyzes the work progress.

[1546] Determine whether any malfunctions or abnormalities are detected.

[1547] Step 6: Detect and warn of misoperation

[1548] server

[1549] If the server detects an error, it generates a warning message.

[1550] Terminal

[1551] The device will display a visual warning message on the audio glasses and issue an audio warning.

[1552] For example, the system will guide you on how to correct the error by saying, "Please take product B from shelf A1."

[1553] User

[1554] The user checks the warning and corrects the error as instructed. For example, the user picks up the correct product from shelf A1.

[1555] Step 7: Notify and record completion of work

[1556] Terminal

[1557] After the user has correctly completed the task, the terminal sends a task completion notification to the server.

[1558] server

[1559] The server records the time the task is completed.

[1560] Evaluate user skill levels by comparing them with standard work hours.

[1561] Step 8: Assess skill levels and generate support suggestions

[1562] server

[1563] The server generates support proposals for skill improvement based on the user's skill level.

[1564] Send support suggestions to the device along with the next work instructions.

[1565] Terminal

[1566] The device displays the next steps on the audio glasses and provides audio instructions.

[1567] Provide users with visual and audio cues to improve their skills.

[1568] Step 9: Start the next process

[1569] User

[1570] The user starts the next work process and proceeds with the work while referring to the support proposal.

[1571] The above is the specific flow of each processing step of the program. This allows workers to perform new tasks with confidence, prevents operational errors, and improves their skills.

[1572] Example 1

[1573] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1574] Conventional work navigation systems lacked the ability to detect and correct worker errors in real time. They also lacked a mechanism for providing guidance and feedback tailored to each worker's skill level, making it difficult to improve productivity and work quality. Furthermore, it was difficult to provide customized work procedures based on user authentication and profiles, and they could only provide uniform guidance.

[1575] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1576] In this invention, the server includes means for accessing the work process database and managing each work step, wearable terminal means for outputting instructions related to the work process visually and audibly, AI module means for monitoring the worker's operations in real time and providing feedback via the wearable terminal means, control means for issuing a visual and audio warning and preventing the worker from proceeding to the next step if an incorrect operation is detected, evaluation means for recording the completion time of each work step and comparing it with a standard work time to evaluate the worker's skill level, means for receiving user authentication information and providing a customized work procedure based on the worker profile, and control means for analyzing monitoring data acquired in real time and providing appropriate feedback to the worker if an incorrect operation is detected. This makes it possible to detect and correct incorrect operations, provide customized guidance according to the worker's skill, and provide feedback in real time.

[1577] The "work process database" is a database for storing and managing information about work procedures and processes.

[1578] "Work steps" refers to the set of instructions or procedures required to accomplish a particular work process.

[1579] A "wearable terminal" is an information display and input device that can be worn by a worker and has the function of issuing visual and audio instructions.

[1580] The "AI module" is a system component that uses artificial intelligence technology to perform data analysis and pattern recognition, and provides appropriate feedback to workers.

[1581] A "controller" is a hardware or software function that governs the operation of the entire system and performs appropriate actions under specific conditions.

[1582] The "evaluation means" is a function for quantitatively evaluating the performance of a worker and evaluating the skill level by comparing it with the standard work time.

[1583] "User authentication information" refers to information used to identify a user, such as an ID or password entered when accessing a system.

[1584] A "worker profile" is data that includes detailed information about each individual worker, such as the worker's skill level and past work history.

[1585] A "customized work procedure" is a work procedure that is individually generated based on a worker profile and is adjusted according to the worker's skills and needs.

[1586] "Real-time monitoring data" refers to data that is collected and analyzed in real time to track the movements and working conditions of workers.

[1587] "Feedback" refers to instructions and suggested corrections that the system provides to the worker, including information that helps them understand the progress of their work and correct mistakes.

[1588] MODE FOR CARRYING OUT THE INVENTION

[1589] The present invention provides a system that utilizes an AI module and a wearable device to navigate workers in real time and improve productivity. Specific embodiments of the system are described below.

[1590] System Configuration

[1591] The system consists of the following main components:

[1592] 1. Server

[1593] Access to the work process database

[1594] Management of each work step

[1595] Analysis of work monitoring data

[1596] Generate feedback and warnings

[1597] Manage worker profiles

[1598] 2. Wearable devices

[1599] Visual display of work instructions (e.g. audio glasses)

[1600] Voice output of work instructions

[1601] User authentication function

[1602] Work monitoring using cameras and sensors

[1603] 3. Users

[1604] Check and implement work instructions

[1605] Receiving and responding to feedback

[1606] Implementing correction work

[1607] Program processing overview

[1608] Explain how the components of the system work together.

[1609] 1. Initial Server Setup

[1610] When the server starts up, it reads the work process database and worker profile data.

[1611] The work procedure data includes work steps, standard work times, and error conditions.

[1612] 2. User Authentication

[1613] When the device boots up, the user enters authentication information on a login screen.

[1614] The server verifies the credentials and retrieves the worker's profile.

[1615] 3. Display of work procedures

[1616] Once the authentication is complete, the server sends the work procedure information to the terminal.

[1617] The device visually displays the work procedure on the audio glasses and begins providing audio guidance.

[1618] 4. Work monitoring and feedback

[1619] Once the user begins work, the device's camera and sensors monitor the progress of the work.

[1620] The monitoring data is sent to a server in real time and analyzed.

[1621] The server analyzes the monitoring data and checks for any incorrect operations.

[1622] 5. Warnings and Corrections

[1623] If an erroneous operation is detected, the server generates a warning message.

[1624] The device will display a visual warning on the audio glasses and provide audio instructions on how to fix the problem.

[1625] 6. Record and analyze work time

[1626] The completion time of each work step is transmitted from the terminal to the server.

[1627] The server evaluates the skill level of the worker by comparing with the standard work time.

[1628] Support proposals for skill development are generated and sent to the terminal along with the next work instructions.

[1629] Specific examples

[1630] The operation of the system will be explained below based on a specific warehouse work scenario.

[1631] (example)

[1632] 1. Initial Server Setup

[1633] The server reads the "picking work procedure data" and obtains the "profile data" of worker A.

[1634] 2. User Authentication

[1635] Worker A enters login information on the terminal, and the server performs authentication.

[1636] 3. Display of work procedures

[1637] The server sends the instruction "First, pick product B from shelf number A1" to the terminal.

[1638] The terminal displays "Shelf Number A1" on the audio glasses and provides voice guidance.

[1639] 4. Work monitoring and feedback

[1640] Worker A goes to shelf A1 and begins the action of taking out product B.

[1641] The device's camera captures the movement and sends it to the server.

[1642] The server analyzes the monitoring data to determine the progress of the work.

[1643] 5. Warnings and Corrections

[1644] The server detects the incorrect operation and generates a "warning message."

[1645] The terminal will provide a visual warning saying "Please take product B from shelf A1" and also provide audio guidance.

[1646] 6. Record and analyze work time

[1647] The terminal sends the "task completion time" to the server.

[1648] The server compares the data with the "standard work time data" to evaluate the skill level of worker A.

[1649] It generates "hints for improving work efficiency along with the next step" and sends them to the terminal.

[1650] The system utilizes an AI module to monitor worker behavior in real time and improve work quality. It uses a generative AI model to generate appropriate prompts and provide timely instructions and feedback to workers.

[1651] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1652] Step 1:

[1653] Initial Server Configuration

[1654] Input: When the server starts, it reads the work process database and worker profile data.

[1655] Specific behavior:

[1656] The server retrieves "picking work procedure data" from the "work process database," which includes shelf numbers, product IDs, and operation procedures.

[1657] The server obtains worker A's skill level and past work history from the "worker profile data."

[1658] Output: The work procedure data and worker profile data are loaded, and the system completes its initial setup.

[1659] Step 2:

[1660] User Authentication

[1661] Input: The device boots up and the user enters their credentials at the login screen.

[1662] Specific behavior:

[1663] The terminal displays a login dialog and the user enters their "ID" and "password."

[1664] The server verifies the authentication information through the "User Authentication API."

[1665] If the authentication is successful, the server obtains the "worker profile data."

[1666] Output: User authentication is successful and worker profile data is retrieved.

[1667] Step 3:

[1668] Displaying work procedures

[1669] Input: After successful user authentication, the server sends the work procedure information to the terminal.

[1670] Specific behavior:

[1671] The server sends the "work procedure information" to the terminal.

[1672] The terminal displays on the audio glasses, "Please pick product B from shelf number A1."

[1673] The device will provide similar instructions via voice prompts.

[1674] Output: The work procedure is displayed on the terminal and the user is guided by voice.

[1675] Step 4:

[1676] Work monitoring and feedback

[1677] Input: The user starts a task, and the device's camera and sensors monitor the task's progress.

[1678] Specific behavior:

[1679] The user begins to move towards "shelf number A1" to take out product B.

[1680] The device's camera and sensors capture its movements and transmit "surveillance data" to a server in real time.

[1681] The server analyzes real-time monitoring data and compares it with prescribed work procedures.

[1682] Output: The monitoring data is sent to the server in real time and the analysis results are obtained.

[1683] Step 5:

[1684] Warnings and fixes

[1685] Input: Detects misoperation using monitoring data analyzed by the server.

[1686] Specific behavior:

[1687] If the server detects an "incorrect operation", it generates a "warning message".

[1688] For example, if a user attempts to remove product C from shelf A2, the server generates a "warning message."

[1689] The terminal will provide a visual warning and audio guidance saying, "Please take product B from shelf A1."

[1690] Output: Information is provided to help the user recognize and correct the error.

[1691] Step 6:

[1692] Record and analyze work time

[1693] Input: As each work step is completed, the completion time is sent from the terminal to the server.

[1694] Specific behavior:

[1695] The terminal sends the "task completion time" to the server.

[1696] The server compares the data with the "standard work time data" and evaluates the skill level of worker A.

[1697] As a result of the analysis, the server generates "hints for improving work efficiency along with the next step" and sends them to the terminal.

[1698] Output: The task completion time is recorded, the worker's skill level is evaluated, and hints for the next step are provided.

[1699] Through these steps, the system monitors the worker's actions in real time and provides appropriate feedback and instructions, thereby improving productivity and work quality.

[1700] (Application example 1)

[1701] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1702] Conventional work management systems have been plagued by problems such as operator errors and reduced work efficiency. Real-time monitoring and appropriate feedback were lacking, making it difficult to improve work accuracy and skills. Furthermore, customized guides for improving work efficiency were not provided. The present invention aims to solve these problems and improve worker productivity and accuracy.

[1703] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1704] In this invention, the server includes: means for accessing the work process database and managing each work step; wearable terminal means for outputting instructions related to the work process visually and audibly; AI module means for monitoring the worker's operations in real time and providing feedback via the wearable terminal means; control means for issuing visual and audio warnings and preventing the worker from proceeding to the next step if an incorrect operation is detected; evaluation means for recording the completion time of each work step and comparing it with a standard work time to evaluate the worker's skill level; means for providing visual and audio guides of the work instructions; means with a camera function for monitoring the user's actions in real time; and audio output means for generating audio guides. This makes it possible to monitor the worker's operations in real time and provide immediate feedback and guidance on the correct operation method if an incorrect operation occurs.

[1705] A "work process database" is a database for managing and storing information about the procedures and steps of a specific work.

[1706] A "visual display" is a means of presenting information in a form that can be seen by a user, and includes, for example, a display or a projector.

[1707] "Audio output" refers to a means of providing information to a user by voice using a speaker, headphones, or the like.

[1708] A "wearable device" is an electronic device that can be worn on the body and used, such as smart glasses and head-mounted displays.

[1709] "Real-time monitoring" is the process of checking the situation and collecting data at the exact moment that a task or operation is being performed.

[1710] An "AI module" is a software or hardware module that uses artificial intelligence algorithms to analyze data and make decisions.

[1711] "Feedback" is a means of indicating errors and areas for improvement by providing real-time evaluation and guidance on operations and actions.

[1712] A "warning" is a visual or audio notification that alerts the user to a risk or error and prompts caution.

[1713] A "control means" is a component equipped with the functions and algorithms necessary to monitor the entire system and provide appropriate operation and feedback.

[1714] The "evaluation means" is a function that evaluates the skill level of a worker by recording the completion time and work quality of each work step and comparing them with the standard work time.

[1715] The "camera function" is a means of collecting visual information and recording and analyzing it as images or videos.

[1716] "Voice guide" is a function that provides appropriate instructions and guidance to workers using voice.

[1717] The present invention is a system that utilizes an AI module and a wearable device to navigate workers in real time and improve productivity. Specific embodiments and program processing of the system are described below.

[1718] The system that realizes this application example is created based on the following program.

[1719] Program processing explanation

[1720] Initial Setup

[1721] The server accesses the work process database to manage each work step. The work process data includes work steps, standard work times, and error conditions. Worker profile data is also loaded.

[1722] User Authentication

[1723] The user enters login information on the terminal, the server verifies the authentication information, and if authentication is successful, retrieves the user's work profile.

[1724] View work orders

[1725] Once authenticated, the server sends customized work instructions to the device, which then displays the instructions visually and provides audio guidance via smart glasses or a head-mounted display. Google Text-to-Speech (gTTS) is used for voice output.

[1726] Work monitoring and feedback

[1727] When the user starts working, the device's built-in camera (using OpenCV) and sensors monitor the work status in real time. The monitoring data is sent to a server, where an AI module analyzes the data and checks for any operational errors.

[1728] Warnings and fixes

[1729] If an incorrect operation is detected, the server generates a warning message and provides visual and audio feedback to the operator via the terminal, allowing the operator to correct the incorrect operation.

[1730] Record and analyze work time

[1731] The completion time for each task is sent from the terminal to the server, which then compares it with the standard work time. This allows the worker's skill level to be evaluated, and the results are reflected in subsequent work instructions.

[1732] Specific examples

[1733] For example, consider a warehouse operation at a logistics center. When a worker is tasked with new picking work, the following scenario can be envisioned.

[1734] The server loads the "picking work procedure data" and "worker A's profile data." Worker A enters his / her login information on the terminal, and the server authenticates him / her. After authentication, the server sends the instruction "First, pick product B from shelf number A1" to the terminal. The terminal then displays the location of shelf number A1 on the smart glasses' display and provides audio guidance.

[1735] When worker A goes to shelf A1 and tries to take item B, the device's camera and sensors monitor the action and send the situation to the server in real time. If the server detects an incorrect operation (e.g., trying to take item C from shelf A2), the device will provide visual and audio guidance saying, "Please take item B from shelf A1." After correctly taking item B, the server records the task completion time and compares it with the standard work time to evaluate the skill level.

[1736] An example prompt is:

[1737] Example prompt

[1738] user_info = authenticate_user("new_worker", "securepass")

[1739] tasks = get_task_instructions(user_info['user_id'])

[1740] for task in tasks:

[1741] print(f"Next task: {task['instruction']}")

[1742] play_audio(task['instruction'])

[1743] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1744] Step 1:

[1745] The server accesses the work process database and reads each work step and worker profile data.

[1746] Input: Work process database, worker profile data

[1747] Data calculation: Retrieving information from a database.

[1748] Output: Loaded work procedure data, worker profile data

[1749] Step 2:

[1750] The user enters login information at the terminal and the server verifies the authentication information.

[1751] Input: User login information

[1752] Data calculation: Checks credentials and retrieves worker profile.

[1753] Output: Authenticated user's work profile

[1754] Step 3:

[1755] The server transmits customized work procedure information to the terminal for the authenticated user.

[1756] Input: Authenticated user's work profile

[1757] Data calculation: Generate customized work procedures based on work profiles

[1758] Output: Customized routing information

[1759] Step 4:

[1760] The device visually displays work procedures and provides audio guidance via smart glasses or a head-mounted display.

[1761] Input: Customized routing information

[1762] Data calculation: Visual display and speech translation of work instructions (using gTTS)

[1763] Output: Visually displayed work instructions, audio guide

[1764] Step 5:

[1765] The user starts working, and the camera (using OpenCV) and sensors on the device monitor the work status in real time.

[1766] Input: User's work actions

[1767] Data Calculation: Analysis of camera footage and sensor data

[1768] Output: Monitoring data (real-time)

[1769] Step 6:

[1770] The device sends the collected monitoring data to a server, which then uses an AI module to analyze the data and check for any operational errors.

[1771] Input: Monitoring data

[1772] Data calculation: Data analysis and error detection by AI module

[1773] Output: Misoperation detection result

[1774] Step 7:

[1775] If an incorrect operation is detected, the server generates a warning message and provides visual and audio feedback to the user via the terminal.

[1776] Input: Misoperation detection result

[1777] Data Calculation: Generate warning messages and provide visual and audio feedback

[1778] Output: Warning messages, visual and audio feedback

[1779] Step 8:

[1780] The server records the completion time of each work step, compares it with the standard work time to evaluate the skill level, and reflects the results in subsequent work instructions.

[1781] Input: Completion time for each work step

[1782] Data calculation: Comparison with standard work time and skill level evaluation

[1783] Output: Skill level evaluation results, data to be reflected in the next work instructions

[1784] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1785] MODE FOR CARRYING OUT THE INVENTION

[1786] The present invention is a system that utilizes an AI module, a wearable device, and an emotion engine to navigate workers in real time and provide feedback according to the worker's emotional state, thereby improving productivity. Specific embodiments and program processing will be described below.

[1787] System Configuration

[1788] The system consists of the following main components:

[1789] 1. Server

[1790] Access to the work process database

[1791] Managing work steps

[1792] Analysis of work monitoring data and emotion data

[1793] Generating feedback, warnings, and emotion-responsive messages

[1794] Manage worker profiles

[1795] 2. Wearable devices

[1796] Visual and audio display of work instructions (e.g. audio glasses)

[1797] User authentication function

[1798] Work monitoring using cameras and sensors

[1799] Emotion recognition sensors (e.g. facial recognition cameras, voice analysis microphones)

[1800] 3. Users

[1801] Check and implement work instructions

[1802] Receiving and responding to feedback and emotional responses

[1803] Implementing correction work

[1804] 4. Emotion Engine

[1805] Data analysis from emotion recognition sensors

[1806] Determining the user's emotional state

[1807] Message generation according to emotional state

[1808] Program processing overview

[1809] Explain how the components of the system work together.

[1810] 1. Initial Server Setup

[1811] When the server starts up, it loads the work process database, the worker profile data, and the emotional state data.

[1812] The work procedure data includes work steps, standard work times, and error conditions.

[1813] 2. User Authentication

[1814] When the device boots up, the user enters authentication information on a login screen.

[1815] The server verifies the credentials and retrieves the worker's profile.

[1816] 3. Display of work procedures

[1817] Once the authentication is complete, the server sends the work procedure information to the terminal.

[1818] The device visually displays the work procedure on the audio glasses display and provides audio guidance.

[1819] 4. Work monitoring and feedback

[1820] Once the user begins work, the device's camera and sensors monitor the progress of the work.

[1821] The monitoring data is sent to the server in real time.

[1822] The server analyzes the monitoring data and checks for any incorrect operations.

[1823] 5. Recognizing emotional states

[1824] The emotion engine receives and analyzes data from the device's emotion recognition sensor.

[1825] The server determines the user's emotional state based on their visual expressions, vocal tone, and physical movements.

[1826] 6. Warnings and Emotional Messages

[1827] If an erroneous operation is detected, the server generates a warning message.

[1828] The emotion engine generates relaxation instructions and motivational messages according to the user's emotional state.

[1829] The device will display a visual warning on the audio glasses and provide audio instructions on how to fix the problem.

[1830] 7. Record and analyze work time

[1831] The completion time of each work step is transmitted from the terminal to the server.

[1832] The server evaluates the skill level of the user by comparing it with standard work time.

[1833] 8. Skill level assessment and support proposal generation

[1834] The server generates support proposals for skill improvement based on the user's skill level.

[1835] Send support suggestions to the device along with the next work instructions.

[1836] 9. Start of the next process

[1837] The user starts the next work process and proceeds with the work while referring to the support suggestions and emotional response messages.

[1838] Specific examples

[1839] The operation of the system will be explained below based on a specific warehouse work scenario.

[1840] 1. Initial Server Setup

[1841] The server reads the "picking work procedure data" and "worker A's profile data."

[1842] 2. User Authentication

[1843] Worker A enters login information on the terminal, and the server performs authentication.

[1844] 3. Display of work procedures

[1845] The server sends the instruction "First, pick product B from shelf number A1" to the terminal.

[1846] The terminal displays the location of shelf number A1 on the audio glasses display and provides voice guidance.

[1847] 4. Work monitoring and feedback

[1848] Worker A begins to take out product B from shelf A1.

[1849] The device sends the work status to the server via the camera and sensors.

[1850] 5. Recognizing emotional states

[1851] The emotion engine analyzes worker A's facial expressions and tone of voice and determines that he is under stress.

[1852] 6. Warnings and Emotional Messages

[1853] If the server detects an incorrect operation, it generates a warning message saying, "Please remove product B from shelf A1."

[1854] The emotion engine generates an emotion-responsive message: "Relax and take a deep breath."

[1855] The device displays warnings and emotion-responsive messages on the audio glasses and provides voice guidance.

[1856] 7. Record and analyze work time

[1857] After worker A correctly picks up product B, the server records the task completion time.

[1858] The server compares the time with the standard work time and evaluates the skill level of worker A.

[1859] 8. Skill level assessment and support proposal generation

[1860] The server displays "tips for improving work efficiency before proceeding to the next step" on the terminal and provides audio explanations.

[1861] 9. Start of the next process

[1862] Worker A starts the next work process and proceeds with the work while referring to the support proposal and emotional response message.

[1863] This allows workers to perform new tasks with confidence, preventing operational errors and improving their skills. In addition, the emotion engine provides appropriate feedback according to the worker's emotional state, which is expected to further improve the work environment and work efficiency.

[1864] The processing flow will be explained below.

[1865] Program processing flow

[1866] Step 1: Initial Server Setup

[1867] server

[1868] The server starts up and loads the work process database, worker profile data, and emotional state data.

[1869] The work procedure data includes work steps, standard work times, and error conditions.

[1870] The server logs that initialization is complete.

[1871] Step 2: Authenticate the user

[1872] Terminal

[1873] The device (Audio Glasses) will start up and the login screen will be displayed.

[1874] The user enters authentication information (user ID and password).

[1875] server

[1876] The server receives the authentication information and checks it against a database.

[1877] If the authentication is successful, the worker's profile information is acquired and sent to the terminal.

[1878] Step 3: View the work instructions

[1879] server

[1880] The server transmits current work process information to the terminal.

[1881] Terminal

[1882] The device displays the work procedure on the audio glasses display as a video and provides audio instructions.

[1883] For example, the instruction displayed might be, "Please pick product B from shelf number A1."

[1884] Step 4: Getting started

[1885] User

[1886] The user follows the instructions and starts the task. For example, the user goes to shelf A1 and takes out product B.

[1887] Terminal

[1888] The device's camera and sensors monitor the user's movements in real time and send the monitoring data to a server.

[1889] Step 5: Operational monitoring and analysis

[1890] server

[1891] The server receives the monitoring data and the AI ​​module analyzes the work progress.

[1892] Determine whether any malfunctions or abnormalities are detected.

[1893] Step 6: Recognize your emotional state

[1894] Terminal

[1895] The device's emotion recognition sensors (e.g., facial expression recognition camera, voice analysis microphone) capture the user's visual expressions, voice tone, and physical movements.

[1896] The emotion data is sent to the server.

[1897] server

[1898] The server uses an emotion engine to analyze the user's emotional state.

[1899] Through analysis, it is determined that the user is feeling stressed.

[1900] Step 7: Misoperation detection and warning

[1901] server

[1902] If the server detects an incorrect operation, it generates a warning message saying, "Please remove product B from shelf A1."

[1903] Terminal

[1904] The device will visually display a warning message on the audio glasses and issue an audio warning.

[1905] User

[1906] The user checks the warning and corrects the error as instructed. For example, the user picks up the correct product from shelf A1.

[1907] Step 8: Generate and display emotion-aware messages

[1908] server

[1909] The emotion engine generates an emotion-responsive message such as "Relax and take a deep breath" depending on the user's emotional state.

[1910] The server sends an emotion-related message to the terminal.

[1911] Terminal

[1912] The device visually displays emotion-based messages on the audio glasses and provides audio guidance.

[1913] Step 9: Notify and record completion of work

[1914] Terminal

[1915] After the user has correctly completed the task, the terminal sends a task completion notification to the server.

[1916] server

[1917] The server records the time the task is completed.

[1918] Evaluate user skill levels by comparing them with standard work hours.

[1919] Step 10: Assess skill levels and generate support suggestions

[1920] server

[1921] The server generates support proposals for skill improvement based on the user's skill level.

[1922] Send support suggestions to the device along with the next work instructions.

[1923] Terminal

[1924] The device displays the next steps and tips for improving skills on the audio glasses and provides audio guidance.

[1925] Step 11: Start the next process

[1926] User

[1927] The user starts the next work process and proceeds with the work while referring to the support suggestions and emotional response messages.

[1928] The above is the specific flow of each processing step of the program that combines the emotion engine. This allows workers to work with peace of mind while receiving appropriate feedback according to their emotional state. This prevents operational errors, improves skills, and increases work efficiency.

[1929] Example 2

[1930] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1931] Conventional work support systems often focus on providing immediate feedback to workers regarding operational errors and improving work efficiency, but lack a mechanism to respond to the worker's emotional state. As a result, worker stress and a decline in motivation can have a negative impact on work efficiency and quality. Furthermore, it has been difficult to effectively navigate by combining real-time emotional feedback and work instructions.

[1932] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for accessing the work process database and managing each work step; wearable terminal means for outputting instructions related to the work process visually and audibly; AI module means for monitoring the worker's operations in real time and providing feedback via the wearable terminal means; control means for issuing visual and audio warnings and preventing the worker from proceeding to the next step when an erroneous operation is detected; emotion engine means for analyzing data from an emotion recognition sensor and determining the worker's emotional state; means for generating messages according to the emotional state and providing them via the wearable terminal means; and evaluation means for recording the completion time of each work step and comparing it with the standard work time to evaluate the worker's skill level. This makes it possible to comprehensively manage the worker's real-time work status and emotional state and provide appropriate feedback and emotion-responsive messages.

[1933] The "work process database" is a database for recording and managing information about each work step.

[1934] A "wearable terminal" is a device that can be worn by a user and provides work instructions and feedback.

[1935] An "AI module" is a device that includes an artificial intelligence program that monitors the operations of workers in real time and provides feedback based on the analysis results.

[1936] "Control means" refers to a device or function that detects operational errors, issues visual and audio warnings as a countermeasure, and controls the progress of work.

[1937] An "emotion recognition sensor" is a sensor that detects a worker's facial expression, tone of voice, physical movements, etc., and determines their emotional state.

[1938] An "emotion engine" is software or hardware that analyzes data from emotion recognition sensors and determines the user's emotional state.

[1939] The "evaluation means" is a device or function that records the time required to complete each work step and compares it with the standard work time to evaluate the skill level of the worker.

[1940] "Feedback" refers to information or instructions that provide an evaluation of the worker's operations and areas for improvement.

[1941] "Emotion-responsive messages" are messages that instruct users to relax or increase their motivation, which are generated in response to the user's emotional state.

[1942] "Real-time" means processing or responding immediately with little or no delay.

[1943] MODE FOR CARRYING OUT THE INVENTION

[1944] The present invention provides a system for providing feedback and navigation according to the emotional state of a worker. The system includes a server, a wearable device, an AI module, an emotion recognition sensor, and an emotion engine.

[1945] System Configuration

[1946] The system consists of the following main elements:

[1947] 1. Server

[1948] Access the work process database and manage each work step.

[1949] Stores worker profile data and retrieves necessary information.

[1950] Monitor the progress of work steps in real time and generate feedback.

[1951] 2. Wearable devices

[1952] This device outputs work instructions visually and audibly. An example of such a device is audio glasses.

[1953] A login screen is provided for user authentication.

[1954] Work is monitored in real time using cameras and various sensors.

[1955] It is equipped with an emotion recognition sensor that analyzes the user's facial expressions and tone of voice.

[1956] 3. AI Module

[1957] Analyzes work progress and provides feedback if an error is detected.

[1958] 4. Emotion Recognition Sensor

[1959] Sensors for detecting the user's emotional state in real time include a facial recognition camera and a voice analysis microphone.

[1960] 5. Emotion Engine

[1961] Data from emotion recognition sensors is analyzed to determine the user's emotional state.

[1962] Messages are generated according to the emotional state and provided via a wearable device.

[1963] Program processing overview

[1964] The following describes how each component of the system works together.

[1965] Initial Server Configuration

[1966] When the server starts up, it loads the work process database and the worker profile database. The work process data includes the work procedure for each step, the standard work time, the list of required materials, error conditions, etc.

[1967] User Authentication

[1968] When a user enters their authentication information on the login screen of their wearable device, the device sends the information to the server, which verifies the authentication information and retrieves the worker's profile.

[1969] Displaying work procedures

[1970] Once authentication is complete, the server sends the work procedure information to the device, which then displays visual instructions on the audio glasses and provides audio guidance.

[1971] Work monitoring and feedback

[1972] As the user performs a task, the device's camera and sensors monitor the progress and send the data in real time to a server, which analyzes the data and provides feedback via the wearable device if any errors are detected.

[1973] Recognition of emotional states

[1974] The emotion engine analyzes emotion recognition sensor data from the device and determines the user's emotional state. Based on the results, the server generates an appropriate emotion-responsive message and sends it to the device.

[1975] Warnings and Emotional Messages

[1976] If an incorrect operation is detected, the server generates a warning message, and the emotion engine simultaneously generates a message to instruct the user to relax or to increase motivation, which is then sent via the device.

[1977] Record and analyze work time

[1978] The completion time of each work step is sent from the terminal to a server, and the server compares the recorded data with standard work times to evaluate the skill level.

[1979] Skill level assessment and support suggestions

[1980] The server analyzes the user's skill level and generates support suggestions for skill improvement, which are then provided to the wearable device along with next work instructions.

[1981] Start of the next process

[1982] The user starts the next task, and the device displays support suggestions and emotional response messages to assist them.

[1983] Specific examples

[1984] The operation of the system will be explained below based on a specific warehouse work scenario.

[1985] 1. As the initial setting of the server, the picking work procedure data and worker profile data are read.

[1986] 2. The worker enters login information on the terminal, and the server performs authentication.

[1987] 3. Once authentication is complete, the instruction "First, please pick product B from shelf number A1" is sent to the terminal. The terminal displays the location on the audio glasses and gives instructions via voice.

[1988] 4. Once work begins, the device monitors progress using cameras and sensors and sends the data to the server.

[1989] 5. The emotion engine analyzes the worker's facial expressions and tone of voice and determines, for example, whether they are in a stressed state.

[1990] 6. If an incorrect operation is detected, the server generates a warning message and an emotion-based message such as "Relax and take a deep breath." This message is displayed on the device and guided by voice.

[1991] 7. The device records the time the task is completed, and the server analyzes the data.

[1992] 8. The server evaluates the skill level of the worker and provides tips to improve efficiency.

[1993] 9. The worker begins the next step and proceeds with the work while referring to the support suggestions and emotional response messages.

[1994] Example of input prompt for generative AI model

[1995] Below is an example of a prompt sentence to be input to the generative AI model.

[1996] "The goal of this system is to improve worker productivity. Please explain the following steps in natural language: The worker takes item B from shelf A1. If there is an error, detect the error and provide feedback for correction. Finally, recognize the worker's emotional state and generate an appropriate emotional response message."

[1997] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1998] Specific processing steps of the program

[1999] Step 1: Initial Server Setup

[2000] When the server starts up, it accesses the work process database and worker profile database and loads the necessary information into memory. The inputs are the work process database and worker profile database. The output is work procedure data (each work step, standard work time, error conditions) and worker profile data.

[2001] Specifically, the server acquires the work procedure data and the worker profile data from the database, analyzes them, and completes the initial settings.

[2002] Step 2: User authentication

[2003] The user enters their authentication information on the login screen of the wearable device. The device sends the information to the server, which checks the authentication information in the database and retrieves the user's profile if authentication is successful.

[2004] The input is user authentication information, and the output is profile data for a particular user.

[2005] Specifically, the user enters a username and password into the terminal, and the server returns the authentication result to the terminal.

[2006] Step 3: View the work instructions

[2007] For users who have been successfully authenticated, the server generates work instruction data and sends it to the terminal, which then displays the work procedure visually on the audio glasses and provides audio instructions.

[2008] The inputs are user profile data and work procedure data, and the output is a display of the work procedure and voice guidance.

[2009] Specifically, the server sends information about the next work step to the terminal, and the terminal displays the work procedure on the audio glasses and provides audio guidance.

[2010] Step 4: Work monitoring and feedback

[2011] When the user starts a task, the device's camera and various sensors monitor the progress of the task and send the data in real time to the server, which analyzes the data, checks for errors and smooth progress, and provides necessary feedback to the device.

[2012] The input is sensor data (work progress), and the output is feedback messages.

[2013] Specifically, the device's camera captures images of the user's hands, the location sensor tracks the progress of the task, and the acquired data is sent to a server, which analyzes the data and generates the necessary feedback.

[2014] Step 5: Recognizing your emotional state

[2015] The device's emotion recognition sensor captures the user's facial expressions and voice tone in real time, and the emotion engine analyzes the data. The server then determines the user's emotional state based on the results.

[2016] The input is emotion recognition sensor data (facial expressions, voice tone), and the output is the user's emotional state.

[2017] Specifically, the device's camera captures facial expressions and microphone collects voice tones, and then sends these data to the emotion engine, which analyzes the data and determines the user's emotional state.

[2018] Step 6: Warnings and Emotional Messages

[2019] If the server detects an incorrect operation, it generates a warning message, and the emotion engine generates a message according to the user's emotional state. These messages are sent to the terminal, which then displays and provides voice guidance.

[2020] The inputs are error data and emotional state data, and the output is a warning message and an emotional response message.

[2021] Specifically, when the server detects an incorrect operation, it generates a warning message saying, "An incorrect operation has occurred. Please return to the correct procedure." If the emotion engine determines that the user is in a stressful state, it outputs a message such as, "Relax and take a deep breath." The device displays these messages on the audio glasses and provides audio guidance.

[2022] Step 7: Record and analyze your work time

[2023] The device records the time it takes to complete each task and sends it to the server, which then compares it with the standard time to assess the user's skill level.

[2024] The input is task completion time data, and the output is a user skill level assessment.

[2025] Specifically, the device records the time when the user completes a task and sends the data to the server, which compares it with the standard work time and evaluates the skill level.

[2026] Step 8: Assess skill levels and generate support suggestions

[2027] The server generates support suggestions for the next step based on the user's skill level and sends them to the device, which then displays the suggestions and provides audio guidance.

[2028] The input is the skill level assessment result, and the output is a support proposal.

[2029] Specifically, the server evaluates the user's skill level, generates support suggestions such as "Points to help you work more efficiently in the next step," and sends them to the device. The device then displays the suggestions on the audio glasses and provides audio guidance.

[2030] Step 9: Start the next process

[2031] The user starts the next step of the work process, and the device proceeds with the work while referring to support suggestions and emotion-responsive messages.

[2032] The inputs are the next work instructions, support ideas, and emotion-responsive messages. The output is the progress of the next work process.

[2033] Specifically, the user reads the instructions for the next step, starts the work, and continues working efficiently while referring to support suggestions and emotional response messages from the device.

[2034] (Application example 2)

[2035] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2036] In the current work environment at logistics centers, work instructions are given without considering the emotional state of workers, which can lead to the accumulation of stress and fatigue, resulting in a decline in work efficiency and productivity. Another problem is that when an operational error occurs, a response is delayed, resulting in an increase in overall work time. The purpose of this invention is to solve these problems and improve work efficiency and productivity by providing appropriate feedback according to the emotional state of workers.

[2037] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for accessing a work process database and managing each work step; wearable terminal means for outputting instructions related to the work process visually and audibly; artificial intelligence module means for monitoring the worker's operations in real time and providing feedback via the wearable terminal means; control means for issuing a visual and audio warning and preventing the worker from proceeding to the next step if an erroneous operation is detected; emotion recognition means for analyzing the worker's facial expression and tone of voice to determine the worker's emotional state; emotion response feedback means for generating relaxation instructions or motivation-boosting messages according to the worker's emotional state and outputting them via the wearable terminal means; and evaluation means for recording the completion time of each work step and comparing it with the standard work time to evaluate the worker's skill level. This enables logistics centers to improve work efficiency and productivity while managing the emotions of workers.

[2038] The "work process database" is a database for managing information such as each work step and its procedure, standard work time, and conditions for operational errors.

[2039] "Wearable terminal means" refers to a device that can be worn by a worker and provides visual displays and audio instructions. For example, smart glasses and audio glasses are examples of such devices.

[2040] The "artificial intelligence module means" is a function that uses artificial intelligence to monitor the operator's operations in real time and provide feedback.

[2041] The "control means" is a function that issues a visual and audio warning when an erroneous operation is detected, and prevents the process from proceeding to the next step.

[2042] The "emotion recognition means" is a function for determining the emotional state of a worker by analyzing the worker's facial expression and tone of voice.

[2043] The "emotion-responsive feedback means" is a function for generating relaxation instructions and motivation-boosting messages according to the emotional state determined by the emotion recognition means, and outputting them on the wearable terminal means.

[2044] The "evaluation means" is a function for recording the completion time of each work step and comparing it with the standard work time to evaluate the skill level of the worker.

[2045] This invention is a system that combines artificial intelligence, wearable devices, and an emotion engine to improve the efficiency and productivity of workers at logistics centers. The system accesses a work process database, displays and outputs work instructions visually and audibly, monitors worker operations in real time, and issues a warning if an error is detected. It also analyzes the worker's facial expressions and tone of voice to determine their emotional state and provides feedback according to that state.

[2046] System Configuration

[2047] The system consists of the following main components:

[2048] 1. Server

[2049] Access to the work process database

[2050] Managing work steps

[2051] Analysis of work monitoring data and emotion data

[2052] Generating feedback, warnings, and emotion-responsive messages

[2053] Manage worker profiles

[2054] 2. Wearable devices

[2055] Visual and audio display of work instructions (e.g., smart glasses)

[2056] User authentication function

[2057] Work monitoring using cameras and sensors

[2058] Emotion recognition sensors (e.g. facial recognition cameras, voice analysis microphones)

[2059] 3. Users

[2060] Check and implement work instructions

[2061] Receiving and responding to feedback and emotional responses

[2062] Implementing correction work

[2063] 4. Emotion Engine

[2064] Data analysis from emotion recognition sensors

[2065] Determining the user's emotional state

[2066] Message generation according to emotional state

[2067] Program processing overview

[2068] Explain how the components of the system work together.

[2069] 1. Initial Server Setup

[2070] At startup, the server loads the work process database, worker profile data, and emotional state data.

[2071] The work procedure data includes work steps, standard work times, and error conditions.

[2072] 2. User Authentication

[2073] When the device boots up, the user enters authentication information on a login screen.

[2074] The server verifies the credentials and retrieves the worker's profile.

[2075] 3. Display of work procedures

[2076] Once the authentication is complete, the server sends the work procedure information to the terminal.

[2077] The device visually displays the work procedure on the smart glasses' display and provides audio guidance.

[2078] 4. Work monitoring and feedback

[2079] Once the user begins work, the device's camera and sensors monitor the progress of the work.

[2080] The monitoring data is sent to the server in real time.

[2081] The server analyzes the monitoring data and checks for any incorrect operations.

[2082] 5. Recognizing emotional states

[2083] The emotion engine receives and analyzes data from the device's emotion recognition sensor.

[2084] The server determines the user's emotional state based on their visual expressions, vocal tone, and physical movements.

[2085] 6. Warnings and Emotional Messages

[2086] If an erroneous operation is detected, the server generates a warning message.

[2087] The emotion engine generates relaxation instructions and motivational messages according to the user's emotional state.

[2088] The device will display a visual warning on the smart glasses and provide audio instructions on how to fix the problem.

[2089] 7. Record and analyze work time

[2090] The completion time of each work step is transmitted from the terminal to the server.

[2091] The server evaluates the skill level of the user by comparing it with standard work time.

[2092] 8. Skill level assessment and support proposal generation

[2093] The server generates support proposals for skill improvement based on the user's skill level.

[2094] Send support suggestions to the device along with the next work instructions.

[2095] 9. Start of the next process

[2096] The user starts the next work process and proceeds with the work while referring to the support suggestions and emotional response messages.

[2097] Specific examples

[2098] The operation of the system will be explained below based on a specific warehouse work scenario.

[2099] 1. Initial Server Setup

[2100] The server reads the "picking work procedure data" and "worker A's profile data."

[2101] 2. User Authentication

[2102] Worker A enters login information on the terminal, and the server performs authentication.

[2103] 3. Display of work procedures

[2104] The server sends the instruction "First, pick product B from shelf number A1" to the terminal.

[2105] The device displays the location of shelf number A1 on the smart glasses' display and provides voice guidance.

[2106] 4. Work monitoring and feedback

[2107] Worker A begins to take out product B from shelf A1.

[2108] The device sends the work status to the server via the camera and sensors.

[2109] 5. Recognizing emotional states

[2110] The emotion engine analyzes worker A's facial expressions and tone of voice and determines that he is under stress.

[2111] 6. Warnings and Emotional Messages

[2112] If the server detects an incorrect operation, it generates a warning message saying, "Please remove product B from shelf A1."

[2113] The emotion engine generates an emotion-responsive message: "Relax and take a deep breath."

[2114] The device displays warnings and emotion-responsive messages on the smart glasses and provides voice guidance.

[2115] 7. Record and analyze work time

[2116] After worker A correctly picks up product B, the server records the task completion time.

[2117] The server compares the time with the standard work time and evaluates the skill level of worker A.

[2118] 8. Skill level assessment and support proposal generation

[2119] The server displays "tips for improving work efficiency before proceeding to the next step" on the terminal and provides audio explanations.

[2120] 9. Start of the next process

[2121] Worker A starts the next work process and proceeds with the work while referring to the support proposal and emotional response message.

[2122] A specific example is, "The user's facial expression is captured by the camera. Analyze the emotion from the facial expression and provide appropriate feedback. Check the on-site work steps and whether there are any errors, and update the instructions in real time."

[2123] This allows workers to perform new tasks with confidence, preventing operational errors and improving their skills. In addition, the emotion engine provides appropriate feedback according to the worker's emotional state, which is expected to further improve the work environment and work efficiency.

[2124] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2125] Step 1:

[2126] Initial Setup

[2127] When the server starts up, it reads the work process database, worker profile data, and emotional state data. It uses the data obtained from each database as input. For data processing, the work procedure data includes each work step, standard work time, and error conditions. As output, this data is stored in the server memory.

[2128] Step 2:

[2129] User Authentication

[2130] When the terminal starts up, the user enters authentication information on the login screen. The user ID and password are used as input. The server verifies the authentication information and retrieves the worker's profile. The data calculation compares the authentication information with the information in the database. The output returns a status of authentication success or failure.

[2131] Step 3:

[2132] Displaying work procedures

[2133] Once authentication is complete, the server sends the work procedure information to the terminal. As input, the work procedure data obtained from the work process database is used. As data processing, the current work step is extracted and sent to the terminal. The terminal visually displays the work procedure on the smart glasses display and provides audio guidance. As output, the work procedure is generated and displayed to the user.

[2134] Step 4:

[2135] Work monitoring and feedback

[2136] When a user starts a task, the device's camera and sensors monitor the progress of the task. Camera footage and sensor data are used as input. The device sends this data to the server in real time. For data calculation, the server analyzes the monitoring data and checks for any errors. Feedback data is created as output.

[2137] Step 5:

[2138] Recognition of emotional states

[2139] The emotion engine receives and analyzes data from the device's emotion recognition sensor. Facial expression data and voice data are used as input. The server then processes the data and determines the user's emotional state based on their visual expressions, voice tone, and physical movements. The output is the user's emotional state.

[2140] Step 6:

[2141] Warnings and Emotional Messages

[2142] If an incorrect operation is detected, the server generates a warning message. The emotion engine generates messages to instruct users to relax or to increase their motivation according to their emotional state. Monitoring data and emotional state data are used as input. Appropriate messages are generated as data processing. As output, warnings and emotion-related messages are generated and displayed on the terminal.

[2143] Step 7:

[2144] Record and analyze work time

[2145] The completion time of each work step is sent from the terminal to the server. The timestamp of the work step completion is used as input. The data is calculated by comparing it with the standard work time to evaluate the user's skill level. Skill evaluation data is generated as output.

[2146] Step 8:

[2147] Skill level assessment and support suggestions

[2148] The server generates support proposals for skill improvement based on the user's skill level. Skill evaluation data is used as input. Data processing generates support proposals according to the user's skill level. As output, the support proposals are sent to the terminal along with instructions for the next work.

[2149] Step 9:

[2150] Start of the next process

[2151] The user starts the next work step and proceeds with the work, referring to the support suggestions and emotion-responsive messages. The new work steps and support suggestions are used as input. Data processing provides this information to the user in an appropriate format. The output is an update of the work progress.

[2152] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[2153] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2154] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[2155] [Fourth embodiment]

[2156] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[2157] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[2158] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[2159] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[2160] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[2161] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[2162] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[2163] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[2164] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[2165] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[2166] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[2167] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[2168] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2169] MODE FOR CARRYING OUT THE INVENTION

[2170] The present invention is a system that utilizes an AI module and a wearable device to navigate workers in real time and improve productivity. Specific embodiments and program processing of the system are described below.

[2171] System Configuration

[2172] The system consists of the following main components:

[2173] 1. Server

[2174] Access to the work process database

[2175] Managing work steps

[2176] Analysis of work monitoring data

[2177] Generate feedback and warnings

[2178] Manage worker profiles

[2179] 2. Wearable devices

[2180] Visual display of work instructions (e.g. audio glasses)

[2181] Voice output of work instructions

[2182] User authentication function

[2183] Work monitoring using cameras and sensors

[2184] 3. Users

[2185] Check and implement work instructions

[2186] Receiving and responding to feedback

[2187] Implementing correction work

[2188] Program processing overview

[2189] Explain how the components of the system work together.

[2190] 1. Initial Server Setup

[2191] When the server starts up, it reads the work process database and worker profile data.

[2192] The work procedure data includes work steps, standard work times, and error conditions.

[2193] 2. User Authentication

[2194] When the device boots up, the user enters authentication information on a login screen.

[2195] The server verifies the credentials and retrieves the worker's profile.

[2196] 3. Display of work procedures

[2197] Once the authentication is complete, the server sends the work procedure information to the terminal.

[2198] The device visually displays the work procedure on the audio glasses and provides audio guidance.

[2199] 4. Work monitoring and feedback

[2200] Once the user begins work, the device's camera and sensors monitor the progress of the work.

[2201] The monitoring data is sent to the server in real time.

[2202] The server analyzes the monitoring data and checks for any incorrect operations.

[2203] 5. Warnings and Corrections

[2204] If an erroneous operation is detected, the server generates a warning message.

[2205] The device will display a visual warning on the audio glasses and provide audio instructions on how to fix the problem.

[2206] 6. Record and analyze work time

[2207] The completion time of each work step is transmitted from the terminal to the server.

[2208] The server evaluates the skill level of the worker by comparing with the standard work time.

[2209] Support proposals for skill development are generated and sent to the terminal along with the next work instructions.

[2210] Specific examples

[2211] The operation of the system will be explained below based on a specific warehouse work scenario.

[2212] 1. Initial Server Setup

[2213] The server reads the "picking work procedure data" and "worker A's profile data."

[2214] 2. User Authentication

[2215] Worker A enters login information on the terminal, and the server performs authentication.

[2216] 3. Display of work procedures

[2217] The server sends the instruction "First, pick product B from shelf number A1" to the terminal.

[2218] The terminal displays the location of shelf number A1 on the audio glasses display and provides voice guidance.

[2219] 4. Work monitoring and feedback

[2220] Worker A begins to take out product B from shelf A1.

[2221] The device sends the work status to the server via the camera and sensors.

[2222] 5. Warnings and Corrections

[2223] The server detects an incorrect operation (e.g., trying to remove product C from shelf A2).

[2224] The terminal will provide visual and audio guidance, saying, "Please take product B from shelf A1."

[2225] 6. Record and analyze work time

[2226] After worker A correctly picks up product B, the server records the task completion time.

[2227] The server compares the time with the standard work time and evaluates the skill level of worker A.

[2228] "Tips for improving work efficiency before proceeding to the next step" are displayed on the device and explained in audio.

[2229] This allows workers to perform new tasks with confidence, preventing operational errors and improving skills.

[2230] The processing flow will be explained below.

[2231] Program processing flow

[2232] Step 1: Initial Server Setup

[2233] server

[2234] The server starts up and loads the work process database and worker profile data.

[2235] The work procedure data includes work steps, standard work times, and error conditions.

[2236] The server logs that initialization is complete.

[2237] Step 2: Authenticate the user

[2238] Terminal

[2239] The device (Audio Glasses) will start up and the login screen will be displayed.

[2240] The user enters authentication information (user ID and password).

[2241] server

[2242] The server receives the authentication information and checks it against a database.

[2243] If the authentication is successful, the worker's profile information is acquired and sent to the terminal.

[2244] Step 3: View the work instructions

[2245] server

[2246] The server transmits current work process information to the terminal.

[2247] Terminal

[2248] The device displays the work procedure on the audio glasses display as a video and provides audio instructions.

[2249] For example, the instruction displayed might be, "Please pick product B from shelf number A1."

[2250] Step 4: Getting started

[2251] User

[2252] The user follows the instructions and starts the task. For example, the user goes to shelf A1 and takes out product B.

[2253] Terminal

[2254] The device's camera and sensors monitor the user's movements in real time and send the monitoring data to a server.

[2255] Step 5: Operational monitoring and analysis

[2256] server

[2257] The server receives the monitoring data and the AI ​​module analyzes the work progress.

[2258] Determine whether any malfunctions or abnormalities are detected.

[2259] Step 6: Detect and warn of misoperation

[2260] server

[2261] If the server detects an error, it generates a warning message.

[2262] Terminal

[2263] The device will display a visual warning message on the audio glasses and issue an audio warning.

[2264] For example, the system will guide you on how to correct the error by saying, "Please take product B from shelf A1."

[2265] User

[2266] The user checks the warning and corrects the error as instructed. For example, the user picks up the correct product from shelf A1.

[2267] Step 7: Notify and record completion of work

[2268] Terminal

[2269] After the user has correctly completed the task, the terminal sends a task completion notification to the server.

[2270] server

[2271] The server records the time the task is completed.

[2272] Evaluate user skill levels by comparing them with standard work hours.

[2273] Step 8: Assess skill levels and generate support suggestions

[2274] server

[2275] The server generates support proposals for skill improvement based on the user's skill level.

[2276] Send support suggestions to the device along with the next work instructions.

[2277] Terminal

[2278] The device displays the next steps on the audio glasses and provides audio instructions.

[2279] Provide users with visual and audio cues to improve their skills.

[2280] Step 9: Start the next process

[2281] User

[2282] The user starts the next work process and proceeds with the work while referring to the support proposal.

[2283] The above is the specific flow of each processing step of the program. This allows workers to perform new tasks with confidence, prevents operational errors, and improves their skills.

[2284] Example 1

[2285] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2286] Conventional work navigation systems lacked the ability to detect and correct worker errors in real time. They also lacked a mechanism for providing guidance and feedback tailored to each worker's skill level, making it difficult to improve productivity and work quality. Furthermore, it was difficult to provide customized work procedures based on user authentication and profiles, and they could only provide uniform guidance.

[2287] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[2288] In this invention, the server includes means for accessing the work process database and managing each work step, wearable terminal means for outputting instructions related to the work process visually and audibly, AI module means for monitoring the worker's operations in real time and providing feedback via the wearable terminal means, control means for issuing a visual and audio warning and preventing the worker from proceeding to the next step if an incorrect operation is detected, evaluation means for recording the completion time of each work step and comparing it with a standard work time to evaluate the worker's skill level, means for receiving user authentication information and providing a customized work procedure based on the worker profile, and control means for analyzing monitoring data acquired in real time and providing appropriate feedback to the worker if an incorrect operation is detected. This makes it possible to detect and correct incorrect operations, provide customized guidance according to the worker's skill, and provide feedback in real time.

[2289] The "work process database" is a database for storing and managing information about work procedures and processes.

[2290] "Work steps" refers to the set of instructions or procedures required to accomplish a particular work process.

[2291] A "wearable terminal" is an information display and input device that can be worn by a worker and has the function of issuing visual and audio instructions.

[2292] The "AI module" is a system component that uses artificial intelligence technology to perform data analysis and pattern recognition, and provides appropriate feedback to workers.

[2293] A "controller" is a hardware or software function that governs the operation of the entire system and performs appropriate actions under specific conditions.

[2294] The "evaluation means" is a function for quantitatively evaluating the performance of a worker and evaluating the skill level by comparing it with the standard work time.

[2295] "User authentication information" refers to information used to identify a user, such as an ID or password entered when accessing a system.

[2296] A "worker profile" is data that includes detailed information about each individual worker, such as the worker's skill level and past work history.

[2297] A "customized work procedure" is a work procedure that is individually generated based on a worker profile and is adjusted according to the worker's skills and needs.

[2298] "Real-time monitoring data" refers to data that is collected and analyzed in real time to track the movements and working conditions of workers.

[2299] "Feedback" refers to instructions and suggested corrections that the system provides to the worker, including information that helps them understand the progress of their work and correct mistakes.

[2300] MODE FOR CARRYING OUT THE INVENTION

[2301] The present invention provides a system that utilizes an AI module and a wearable device to navigate workers in real time and improve productivity. Specific embodiments of the system are described below.

[2302] System Configuration

[2303] The system consists of the following main components:

[2304] 1. Server

[2305] Access to the work process database

[2306] Management of each work step

[2307] Analysis of work monitoring data

[2308] Generate feedback and warnings

[2309] Manage worker profiles

[2310] 2. Wearable devices

[2311] Visual display of work instructions (e.g. audio glasses)

[2312] Voice output of work instructions

[2313] User authentication function

[2314] Work monitoring using cameras and sensors

[2315] 3. Users

[2316] Check and implement work instructions

[2317] Receiving and responding to feedback

[2318] Implementing correction work

[2319] Program processing overview

[2320] Explain how the components of the system work together.

[2321] 1. Initial Server Setup

[2322] When the server starts up, it reads the work process database and worker profile data.

[2323] The work procedure data includes work steps, standard work times, and error conditions.

[2324] 2. User Authentication

[2325] When the device boots up, the user enters authentication information on a login screen.

[2326] The server verifies the credentials and retrieves the worker's profile.

[2327] 3. Display of work procedures

[2328] Once the authentication is complete, the server sends the work procedure information to the terminal.

[2329] The device visually displays the work procedure on the audio glasses and begins providing audio guidance.

[2330] 4. Work monitoring and feedback

[2331] Once the user begins work, the device's camera and sensors monitor the progress of the work.

[2332] The monitoring data is sent to a server in real time and analyzed.

[2333] The server analyzes the monitoring data and checks for any incorrect operations.

[2334] 5. Warnings and Corrections

[2335] If an erroneous operation is detected, the server generates a warning message.

[2336] The device will display a visual warning on the audio glasses and provide audio instructions on how to fix the problem.

[2337] 6. Record and analyze work time

[2338] The completion time of each work step is transmitted from the terminal to the server.

[2339] The server evaluates the skill level of the worker by comparing with the standard work time.

[2340] Support proposals for skill development are generated and sent to the terminal along with the next work instructions.

[2341] Specific examples

[2342] The operation of the system will be explained below based on a specific warehouse work scenario.

[2343] (example)

[2344] 1. Initial Server Setup

[2345] The server reads the "picking work procedure data" and obtains the "profile data" of worker A.

[2346] 2. User Authentication

[2347] Worker A enters login information on the terminal, and the server performs authentication.

[2348] 3. Display of work procedures

[2349] The server sends the instruction "First, pick product B from shelf number A1" to the terminal.

[2350] The terminal displays "Shelf Number A1" on the audio glasses and provides voice guidance.

[2351] 4. Work monitoring and feedback

[2352] Worker A goes to shelf A1 and begins the action of taking out product B.

[2353] The device's camera captures the movement and sends it to the server.

[2354] The server analyzes the monitoring data to determine the progress of the work.

[2355] 5. Warnings and Corrections

[2356] The server detects the incorrect operation and generates a "warning message."

[2357] The terminal will provide a visual warning saying "Please take product B from shelf A1" and also provide audio guidance.

[2358] 6. Record and analyze work time

[2359] The terminal sends the "task completion time" to the server.

[2360] The server compares the data with the "standard work time data" to evaluate the skill level of worker A.

[2361] It generates "hints for improving work efficiency along with the next step" and sends them to the terminal.

[2362] The system utilizes an AI module to monitor worker behavior in real time and improve work quality. It uses a generative AI model to generate appropriate prompts and provide timely instructions and feedback to workers.

[2363] The flow of the identification process in the first embodiment will be described with reference to FIG.

[2364] Step 1:

[2365] Initial Server Configuration

[2366] Input: When the server starts, it reads the work process database and worker profile data.

[2367] Specific behavior:

[2368] The server retrieves "picking work procedure data" from the "work process database," which includes shelf numbers, product IDs, and operation procedures.

[2369] The server obtains worker A's skill level and past work history from the "worker profile data."

[2370] Output: The work procedure data and worker profile data are loaded, and the system completes its initial setup.

[2371] Step 2:

[2372] User Authentication

[2373] Input: The device boots up and the user enters their credentials at the login screen.

[2374] Specific behavior:

[2375] The terminal displays a login dialog and the user enters their "ID" and "password."

[2376] The server verifies the authentication information through the "User Authentication API."

[2377] If the authentication is successful, the server obtains the "worker profile data."

[2378] Output: User authentication is successful and worker profile data is retrieved.

[2379] Step 3:

[2380] Displaying work procedures

[2381] Input: After successful user authentication, the server sends the work procedure information to the terminal.

[2382] Specific behavior:

[2383] The server sends the "work procedure information" to the terminal.

[2384] The terminal displays on the audio glasses, "Please pick product B from shelf number A1."

[2385] The device will provide similar instructions via voice prompts.

[2386] Output: The work procedure is displayed on the terminal and the user is guided by voice.

[2387] Step 4:

[2388] Work monitoring and feedback

[2389] Input: The user starts a task, and the device's camera and sensors monitor the task's progress.

[2390] Specific behavior:

[2391] The user begins to move towards "shelf number A1" to take out product B.

[2392] The device's camera and sensors capture its movements and transmit "surveillance data" to a server in real time.

[2393] The server analyzes real-time monitoring data and compares it with prescribed work procedures.

[2394] Output: The monitoring data is sent to the server in real time and the analysis results are obtained.

[2395] Step 5:

[2396] Warnings and fixes

[2397] Input: Detects misoperation using monitoring data analyzed by the server.

[2398] Specific behavior:

[2399] If the server detects an "incorrect operation", it generates a "warning message".

[2400] For example, if a user attempts to remove product C from shelf A2, the server generates a "warning message."

[2401] The terminal will provide a visual warning and audio guidance saying, "Please take product B from shelf A1."

[2402] Output: Information is provided to help the user recognize and correct the error.

[2403] Step 6:

[2404] Record and analyze work time

[2405] Input: As each work step is completed, the completion time is sent from the terminal to the server.

[2406] Specific behavior:

[2407] The terminal sends the "task completion time" to the server.

[2408] The server compares the data with the "standard work time data" and evaluates the skill level of worker A.

[2409] As a result of the analysis, the server generates "hints for improving work efficiency along with the next step" and sends them to the terminal.

[2410] Output: The task completion time is recorded, the worker's skill level is evaluated, and hints for the next step are provided.

[2411] Through these steps, the system monitors the worker's actions in real time and provides appropriate feedback and instructions, thereby improving productivity and work quality.

[2412] (Application example 1)

[2413] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2414] Conventional work management systems have been plagued by problems such as operator errors and reduced work efficiency. Real-time monitoring and appropriate feedback were lacking, making it difficult to improve work accuracy and skills. Furthermore, customized guides for improving work efficiency were not provided. The present invention aims to solve these problems and improve worker productivity and accuracy.

[2415] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2416] In this invention, the server includes: means for accessing the work process database and managing each work step; wearable terminal means for outputting instructions related to the work process visually and audibly; AI module means for monitoring the worker's operations in real time and providing feedback via the wearable terminal means; control means for issuing visual and audio warnings and preventing the worker from proceeding to the next step if an incorrect operation is detected; evaluation means for recording the completion time of each work step and comparing it with a standard work time to evaluate the worker's skill level; means for providing visual and audio guides of the work instructions; means with a camera function for monitoring the user's actions in real time; and audio output means for generating audio guides. This makes it possible to monitor the worker's operations in real time and provide immediate feedback and guidance on the correct operation method if an incorrect operation occurs.

[2417] A "work process database" is a database for managing and storing information about the procedures and steps of a specific work.

[2418] A "visual display" is a means of presenting information in a form that can be seen by a user, and includes, for example, a display or a projector.

[2419] "Audio output" refers to a means of providing information to a user by voice using a speaker, headphones, or the like.

[2420] A "wearable device" is an electronic device that can be worn on the body and used, such as smart glasses and head-mounted displays.

[2421] "Real-time monitoring" is the process of checking the situation and collecting data at the exact moment that a task or operation is being performed.

[2422] An "AI module" is a software or hardware module that uses artificial intelligence algorithms to analyze data and make decisions.

[2423] "Feedback" is a means of indicating errors and areas for improvement by providing real-time evaluation and guidance on operations and actions.

[2424] A "warning" is a visual or audio notification that alerts the user to a risk or error and prompts caution.

[2425] A "control means" is a component equipped with the functions and algorithms necessary to monitor the entire system and provide appropriate operation and feedback.

[2426] The "evaluation means" is a function that evaluates the skill level of a worker by recording the completion time and work quality of each work step and comparing them with the standard work time.

[2427] The "camera function" is a means of collecting visual information and recording and analyzing it as images or videos.

[2428] "Voice guide" is a function that provides appropriate instructions and guidance to workers using voice.

[2429] The present invention is a system that utilizes an AI module and a wearable device to navigate workers in real time and improve productivity. Specific embodiments and program processing of the system are described below.

[2430] The system that realizes this application example is created based on the following program.

[2431] Program processing explanation

[2432] Initial Setup

[2433] The server accesses the work process database to manage each work step. The work process data includes work steps, standard work times, and error conditions. Worker profile data is also loaded.

[2434] User Authentication

[2435] The user enters login information on the terminal, the server verifies the authentication information, and if authentication is successful, retrieves the user's work profile.

[2436] View work orders

[2437] Once authenticated, the server sends customized work instructions to the device, which then displays the instructions visually and provides audio guidance via smart glasses or a head-mounted display. Google Text-to-Speech (gTTS) is used for voice output.

[2438] Work monitoring and feedback

[2439] When the user starts working, the device's built-in camera (using OpenCV) and sensors monitor the work status in real time. The monitoring data is sent to a server, where an AI module analyzes the data and checks for any operational errors.

[2440] Warnings and fixes

[2441] If an incorrect operation is detected, the server generates a warning message and provides visual and audio feedback to the operator via the terminal, allowing the operator to correct the incorrect operation.

[2442] Record and analyze work time

[2443] The completion time for each task is sent from the terminal to the server, which then compares it with the standard work time. This allows the worker's skill level to be evaluated, and the results are reflected in subsequent work instructions.

[2444] Specific examples

[2445] For example, consider a warehouse operation at a logistics center. When a worker is tasked with new picking work, the following scenario can be envisioned.

[2446] The server loads the "picking work procedure data" and "worker A's profile data." Worker A enters his / her login information on the terminal, and the server authenticates him / her. After authentication, the server sends the instruction "First, pick product B from shelf number A1" to the terminal. The terminal then displays the location of shelf number A1 on the smart glasses' display and provides audio guidance.

[2447] When worker A goes to shelf A1 and tries to take item B, the device's camera and sensors monitor the action and send the situation to the server in real time. If the server detects an incorrect operation (e.g., trying to take item C from shelf A2), the device will provide visual and audio guidance saying, "Please take item B from shelf A1." After correctly taking item B, the server records the task completion time and compares it with the standard work time to evaluate the skill level.

[2448] An example prompt is:

[2449] Example prompt

[2450] user_info = authenticate_user("new_worker", "securepass")

[2451] tasks = get_task_instructions(user_info['user_id'])

[2452] for task in tasks:

[2453] print(f"Next task: {task['instruction']}")

[2454] play_audio(task['instruction'])

[2455] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2456] Step 1:

[2457] The server accesses the work process database and reads each work step and worker profile data.

[2458] Input: Work process database, worker profile data

[2459] Data calculation: Retrieving information from a database.

[2460] Output: Loaded work procedure data, worker profile data

[2461] Step 2:

[2462] The user enters login information at the terminal and the server verifies the authentication information.

[2463] Input: User login information

[2464] Data calculation: Checks credentials and retrieves worker profile.

[2465] Output: Authenticated user's work profile

[2466] Step 3:

[2467] The server transmits customized work procedure information to the terminal for the authenticated user.

[2468] Input: Authenticated user's work profile

[2469] Data calculation: Generate customized work procedures based on work profiles

[2470] Output: Customized routing information

[2471] Step 4:

[2472] The device visually displays work procedures and provides audio guidance via smart glasses or a head-mounted display.

[2473] Input: Customized routing information

[2474] Data calculation: Visual display and speech translation of work instructions (using gTTS)

[2475] Output: Visually displayed work instructions, audio guide

[2476] Step 5:

[2477] The user starts working, and the camera (using OpenCV) and sensors on the device monitor the work status in real time.

[2478] Input: User's work actions

[2479] Data Calculation: Analysis of camera footage and sensor data

[2480] Output: Monitoring data (real-time)

[2481] Step 6:

[2482] The device sends the collected monitoring data to a server, which then uses an AI module to analyze the data and check for any operational errors.

[2483] Input: Monitoring data

[2484] Data calculation: Data analysis and error detection by AI module

[2485] Output: Misoperation detection result

[2486] Step 7:

[2487] If an incorrect operation is detected, the server generates a warning message and provides visual and audio feedback to the user via the terminal.

[2488] Input: Misoperation detection result

[2489] Data Calculation: Generate warning messages and provide visual and audio feedback

[2490] Output: Warning messages, visual and audio feedback

[2491] Step 8:

[2492] The server records the completion time of each work step, compares it with the standard work time to evaluate the skill level, and reflects the results in subsequent work instructions.

[2493] Input: Completion time for each work step

[2494] Data calculation: Comparison with standard work time and skill level evaluation

[2495] Output: Skill level evaluation results, data to be reflected in the next work instructions

[2496] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2497] MODE FOR CARRYING OUT THE INVENTION

[2498] The present invention is a system that utilizes an AI module, a wearable device, and an emotion engine to navigate workers in real time and provide feedback according to the worker's emotional state, thereby improving productivity. Specific embodiments and program processing will be described below.

[2499] System Configuration

[2500] The system consists of the following main components:

[2501] 1. Server

[2502] Access to the work process database

[2503] Managing work steps

[2504] Analysis of work monitoring data and emotion data

[2505] Generating feedback, warnings, and emotion-responsive messages

[2506] Manage worker profiles

[2507] 2. Wearable devices

[2508] Visual and audio display of work instructions (e.g. audio glasses)

[2509] User authentication function

[2510] Work monitoring using cameras and sensors

[2511] Emotion recognition sensors (e.g. facial recognition cameras, voice analysis microphones)

[2512] 3. Users

[2513] Check and implement work instructions

[2514] Receiving and responding to feedback and emotional responses

[2515] Implementing correction work

[2516] 4. Emotion Engine

[2517] Data analysis from emotion recognition sensors

[2518] Determining the user's emotional state

[2519] Message generation according to emotional state

[2520] Program processing overview

[2521] Explain how the components of the system work together.

[2522] 1. Initial Server Setup

[2523] When the server starts up, it loads the work process database, the worker profile data, and the emotional state data.

[2524] The work procedure data includes work steps, standard work times, and error conditions.

[2525] 2. User Authentication

[2526] When the device boots up, the user enters authentication information on a login screen.

[2527] The server verifies the credentials and retrieves the worker's profile.

[2528] 3. Display of work procedures

[2529] Once the authentication is complete, the server sends the work procedure information to the terminal.

[2530] The device visually displays the work procedure on the audio glasses display and provides audio guidance.

[2531] 4. Work monitoring and feedback

[2532] Once the user begins work, the device's camera and sensors monitor the progress of the work.

[2533] The monitoring data is sent to the server in real time.

[2534] The server analyzes the monitoring data and checks for any incorrect operations.

[2535] 5. Recognizing emotional states

[2536] The emotion engine receives and analyzes data from the device's emotion recognition sensor.

[2537] The server determines the user's emotional state based on their visual expressions, vocal tone, and physical movements.

[2538] 6. Warnings and Emotional Messages

[2539] If an erroneous operation is detected, the server generates a warning message.

[2540] The emotion engine generates relaxation instructions and motivational messages according to the user's emotional state.

[2541] The device will display a visual warning on the audio glasses and provide audio instructions on how to fix the problem.

[2542] 7. Record and analyze work time

[2543] The completion time of each work step is transmitted from the terminal to the server.

[2544] The server evaluates the skill level of the user by comparing it with standard work time.

[2545] 8. Skill level assessment and support proposal generation

[2546] The server generates support proposals for skill improvement based on the user's skill level.

[2547] Send support suggestions to the device along with the next work instructions.

[2548] 9. Start of the next process

[2549] The user starts the next work process and proceeds with the work while referring to the support suggestions and emotional response messages.

[2550] Specific examples

[2551] The operation of the system will be explained below based on a specific warehouse work scenario.

[2552] 1. Initial Server Setup

[2553] The server reads the "picking work procedure data" and "worker A's profile data."

[2554] 2. User Authentication

[2555] Worker A enters login information on the terminal, and the server performs authentication.

[2556] 3. Display of work procedures

[2557] The server sends the instruction "First, pick product B from shelf number A1" to the terminal.

[2558] The terminal displays the location of shelf number A1 on the audio glasses display and provides voice guidance.

[2559] 4. Work monitoring and feedback

[2560] Worker A begins to take out product B from shelf A1.

[2561] The device sends the work status to the server via the camera and sensors.

[2562] 5. Recognizing emotional states

[2563] The emotion engine analyzes worker A's facial expressions and tone of voice and determines that he is under stress.

[2564] 6. Warnings and Emotional Messages

[2565] If the server detects an incorrect operation, it generates a warning message saying, "Please remove product B from shelf A1."

[2566] The emotion engine generates an emotion-responsive message: "Relax and take a deep breath."

[2567] The device displays warnings and emotion-responsive messages on the audio glasses and provides voice guidance.

[2568] 7. Record and analyze work time

[2569] After worker A correctly picks up product B, the server records the task completion time.

[2570] The server compares the time with the standard work time and evaluates the skill level of worker A.

[2571] 8. Skill level assessment and support proposal generation

[2572] The server displays "tips for improving work efficiency before proceeding to the next step" on the terminal and provides audio explanations.

[2573] 9. Start of the next process

[2574] Worker A starts the next work process and proceeds with the work while referring to the support proposal and emotional response message.

[2575] This allows workers to perform new tasks with confidence, preventing operational errors and improving their skills. In addition, the emotion engine provides appropriate feedback according to the worker's emotional state, which is expected to further improve the work environment and work efficiency.

[2576] The processing flow will be explained below.

[2577] Program processing flow

[2578] Step 1: Initial Server Setup

[2579] server

[2580] The server starts up and loads the work process database, worker profile data, and emotional state data.

[2581] The work procedure data includes work steps, standard work times, and error conditions.

[2582] The server logs that initialization is complete.

[2583] Step 2: Authenticate the user

[2584] Terminal

[2585] The device (Audio Glasses) will start up and the login screen will be displayed.

[2586] The user enters authentication information (user ID and password).

[2587] server

[2588] The server receives the authentication information and checks it against a database.

[2589] If the authentication is successful, the worker's profile information is acquired and sent to the terminal.

[2590] Step 3: View the work instructions

[2591] server

[2592] The server transmits current work process information to the terminal.

[2593] Terminal

[2594] The device displays the work procedure on the audio glasses display as a video and provides audio instructions.

[2595] For example, the instruction displayed might be, "Please pick product B from shelf number A1."

[2596] Step 4: Getting started

[2597] User

[2598] The user follows the instructions and starts the task. For example, the user goes to shelf A1 and takes out product B.

[2599] Terminal

[2600] The device's camera and sensors monitor the user's movements in real time and send the monitoring data to a server.

[2601] Step 5: Operational monitoring and analysis

[2602] server

[2603] The server receives the monitoring data and the AI ​​module analyzes the work progress.

[2604] Determine whether any malfunctions or abnormalities are detected.

[2605] Step 6: Recognize your emotional state

[2606] Terminal

[2607] The device's emotion recognition sensors (e.g., facial expression recognition camera, voice analysis microphone) capture the user's visual expressions, voice tone, and physical movements.

[2608] The emotion data is sent to the server.

[2609] server

[2610] The server uses an emotion engine to analyze the user's emotional state.

[2611] Through analysis, it is determined that the user is feeling stressed.

[2612] Step 7: Misoperation detection and warning

[2613] server

[2614] If the server detects an incorrect operation, it generates a warning message saying, "Please remove product B from shelf A1."

[2615] Terminal

[2616] The device will visually display a warning message on the audio glasses and issue an audio warning.

[2617] User

[2618] The user checks the warning and corrects the error as instructed. For example, the user picks up the correct product from shelf A1.

[2619] Step 8: Generate and display emotion-aware messages

[2620] server

[2621] The emotion engine generates an emotion-responsive message such as "Relax and take a deep breath" depending on the user's emotional state.

[2622] The server sends an emotion-related message to the terminal.

[2623] Terminal

[2624] The device visually displays emotion-based messages on the audio glasses and provides audio guidance.

[2625] Step 9: Notify and record completion of work

[2626] Terminal

[2627] After the user has correctly completed the task, the terminal sends a task completion notification to the server.

[2628] server

[2629] The server records the time the task is completed.

[2630] Evaluate user skill levels by comparing them with standard work hours.

[2631] Step 10: Assess skill levels and generate support suggestions

[2632] server

[2633] The server generates support proposals for skill improvement based on the user's skill level.

[2634] Send support suggestions to the device along with the next work instructions.

[2635] Terminal

[2636] The device displays the next steps and tips for improving skills on the audio glasses and provides audio guidance.

[2637] Step 11: Start the next process

[2638] User

[2639] The user starts the next work process and proceeds with the work while referring to the support suggestions and emotional response messages.

[2640] The above is the specific flow of each processing step of the program that combines the emotion engine. This allows workers to work with peace of mind while receiving appropriate feedback according to their emotional state. This prevents operational errors, improves skills, and increases work efficiency.

[2641] Example 2

[2642] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2643] Conventional work support systems often focus on providing immediate feedback to workers regarding operational errors and improving work efficiency, but lack a mechanism to respond to the worker's emotional state. As a result, worker stress and a decline in motivation can have a negative impact on work efficiency and quality. Furthermore, it has been difficult to effectively navigate by combining real-time emotional feedback and work instructions.

[2644] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for accessing the work process database and managing each work step; wearable terminal means for outputting instructions related to the work process visually and audibly; AI module means for monitoring the worker's operations in real time and providing feedback via the wearable terminal means; control means for issuing visual and audio warnings and preventing the worker from proceeding to the next step when an erroneous operation is detected; emotion engine means for analyzing data from an emotion recognition sensor and determining the worker's emotional state; means for generating messages according to the emotional state and providing them via the wearable terminal means; and evaluation means for recording the completion time of each work step and comparing it with the standard work time to evaluate the worker's skill level. This makes it possible to comprehensively manage the worker's real-time work status and emotional state and provide appropriate feedback and emotion-responsive messages.

[2645] The "work process database" is a database for recording and managing information about each work step.

[2646] A "wearable terminal" is a device that can be worn by a user and provides work instructions and feedback.

[2647] An "AI module" is a device that includes an artificial intelligence program that monitors the operations of workers in real time and provides feedback based on the analysis results.

[2648] "Control means" refers to a device or function that detects operational errors, issues visual and audio warnings as a countermeasure, and controls the progress of work.

[2649] An "emotion recognition sensor" is a sensor that detects a worker's facial expression, tone of voice, physical movements, etc., and determines their emotional state.

[2650] An "emotion engine" is software or hardware that analyzes data from emotion recognition sensors and determines the user's emotional state.

[2651] The "evaluation means" is a device or function that records the time required to complete each work step and compares it with the standard work time to evaluate the skill level of the worker.

[2652] "Feedback" refers to information or instructions that provide an evaluation of the worker's operations and areas for improvement.

[2653] "Emotion-responsive messages" are messages that instruct users to relax or increase their motivation, which are generated in response to the user's emotional state.

[2654] "Real-time" means processing or responding immediately with little or no delay.

[2655] MODE FOR CARRYING OUT THE INVENTION

[2656] The present invention provides a system for providing feedback and navigation according to the emotional state of a worker. The system includes a server, a wearable device, an AI module, an emotion recognition sensor, and an emotion engine.

[2657] System Configuration

[2658] The system consists of the following main elements:

[2659] 1. Server

[2660] Access the work process database and manage each work step.

[2661] Stores worker profile data and retrieves necessary information.

[2662] Monitor the progress of work steps in real time and generate feedback.

[2663] 2. Wearable devices

[2664] This device outputs work instructions visually and audibly. An example of such a device is audio glasses.

[2665] A login screen is provided for user authentication.

[2666] Work is monitored in real time using cameras and various sensors.

[2667] It is equipped with an emotion recognition sensor that analyzes the user's facial expressions and tone of voice.

[2668] 3. AI Module

[2669] Analyzes work progress and provides feedback if an error is detected.

[2670] 4. Emotion Recognition Sensor

[2671] Sensors for detecting the user's emotional state in real time include a facial recognition camera and a voice analysis microphone.

[2672] 5. Emotion Engine

[2673] Data from emotion recognition sensors is analyzed to determine the user's emotional state.

[2674] Messages are generated according to the emotional state and provided via a wearable device.

[2675] Program processing overview

[2676] The following describes how each component of the system works together.

[2677] Initial Server Configuration

[2678] When the server starts up, it loads the work process database and the worker profile database. The work process data includes the work procedure for each step, the standard work time, the list of required materials, error conditions, etc.

[2679] User Authentication

[2680] When a user enters their authentication information on the login screen of their wearable device, the device sends the information to the server, which verifies the authentication information and retrieves the worker's profile.

[2681] Displaying work procedures

[2682] Once authentication is complete, the server sends the work procedure information to the device, which then displays visual instructions on the audio glasses and provides audio guidance.

[2683] Work monitoring and feedback

[2684] As the user performs a task, the device's camera and sensors monitor the progress and send the data in real time to a server, which analyzes the data and provides feedback via the wearable device if any errors are detected.

[2685] Recognition of emotional states

[2686] The emotion engine analyzes emotion recognition sensor data from the device and determines the user's emotional state. Based on the results, the server generates an appropriate emotion-responsive message and sends it to the device.

[2687] Warnings and Emotional Messag...

Claims

1. a means for accessing a work process database and managing each work step; a wearable terminal means for outputting instructions relating to the work process by visual display and voice; an AI module means for monitoring the operation of the worker in real time and providing feedback via the wearable terminal means; a control means for issuing a visual and audio warning when an erroneous operation is detected and preventing the process from proceeding to the next step; and an evaluation means for recording the completion time of each work step and comparing it with a standard work time to evaluate the skill level of the worker.

2. The system according to claim 1 , further comprising means for providing support proposals according to skill levels through the wearable terminal in order to improve the skills of the worker.

3. The system of claim 1 , further comprising means for receiving worker authentication information and providing customized workflows based on the worker profile.

Images