system

Abstract

Provide a system. 【Solution means】 Means for obtaining information from an information storage; Means for analyzing the obtained information to extract relevant information; Means for automatically generating an instruction based on the extracted information; Means for presenting the instruction to the worker and performing necessary preparation confirmation; Means for receiving input from the worker during work and providing information in real time; Means for recording the report content after completion of work and generating improvement plans for the next work; Additional means for visually presenting auxiliary information through a human - engineered interface; A system including the above.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of the chatbot's character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance that responds to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In on-site work, there are problems that it takes a great deal of time and manpower to create work procedures and work instructions. Also, it is difficult to provide real-time information and respond to situation changes during work, and there are concerns about mistakes and efficiency degradation due to this. Therefore, it is required to improve the overall work efficiency from work preparation to implementation and completion, and reduce human resources.

Means for Solving the Problems

[0005] This invention provides a system that automatically generates instruction manuals by acquiring relevant information from a database, analyzing it, and extracting necessary information. Furthermore, this system has a function to present the generated instruction manual to the worker and confirm the necessary preparations. In addition, it accepts worker input in real time during work and provides necessary information in a timely manner to ensure safety and efficiency. Moreover, it records the report content after the completion of work and generates improvement suggestions for the next work, thereby promoting continuous improvement of work quality. Through these means, the system achieves increased efficiency and reduced errors in on-site work.

[0006] A "database" is a system that stores related information and allows you to search for and retrieve necessary data.

[0007] "Means of obtaining information" refers to the function of accessing a database and retrieving necessary documents and data.

[0008] "Methods for analyzing information and extracting relevant information" refers to the process of analyzing acquired data and selecting data that is useful for the task.

[0009] "Methods for automatically generating instruction manuals" refers to technologies that automatically create work procedures and instruction manuals based on extracted information.

[0010] "Means of presenting instructions to workers and confirming necessary preparations" refers to the process of showing the generated instructions to the workers and confirming whether they are ready.

[0011] "A means of receiving input from workers during work and providing information in real time" refers to a function that immediately returns the necessary information in response to the worker's requests or questions.

[0012] "A means of recording report details after work completion and generating improvement proposals for future work" refers to the process of saving a detailed report after work is completed and using it to create proposals for improving future work efficiency. [Brief explanation of the drawing]

[0013] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2, which combines an emotion engine. [Modes for carrying out the invention]

[0014] An example of an embodiment of the system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0015] First, the terms used in the following description will be explained.

[0016] In the following embodiments, a tagged processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0017] In the following embodiments, a tagged RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0018] In the following embodiments, a tagged storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0019] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0020] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0021] [First Embodiment]

[0022] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0023] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0024] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0025] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0026] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0027] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0028] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0029] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0030] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0031] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0032] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0033] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0034] The present invention provides an automation solution for streamlining operations in network centers and telecommunications exchanges. This system primarily consists of coordinated operation between servers, terminals, and users. Specific embodiments of the system are described below.

[0035] First, the server accesses a vast database of network and communication equipment, collecting detailed information on each piece of equipment, past work history, relevant laws and regulations, and guidelines. Next, the server analyzes the collected data and extracts relevant information necessary for on-site work. In this process, the analysis engine utilizes text mining technology to mechanically identify important procedures and points of caution from the data.

[0036] Based on the analyzed information, the server automatically generates instruction manuals and work instructions. The generated instructions are structured to be easy for workers to understand and include detailed information on efficient work procedures, necessary tool lists, and safety checks.

[0037] Next, the terminal presents the generated instructions to the field worker. The worker uses the terminal to review the instructions and check the checklist to ensure they are prepared. At this stage, the terminal records the worker's preparation status and provides an interface to check for any deficiencies in preparation.

[0038] Once work begins, the user (worker) can interact with the AI ​​agent in real time through a terminal. The terminal receives input and voice commands from the worker, and based on this, the AI ​​agent provides information and advice regarding the work. During this time, the server continuously monitors the progress of the work and immediately proposes countermeasures if any abnormalities occur. To ensure the safety of the worker and to ensure efficient work execution, the server also provides voice-based safety confirmation support as needed.

[0039] After completing a task, the user submits a completion report to their terminal. The completed work, any problems, and areas for improvement are entered, and the server evaluates the work based on this information. The evaluation results are stored in a database and used to improve efficiency and efficiency in future work. Based on the report, the server analyzes the performance of each task and generates improvement suggestions to incorporate the feedback into future work instructions.

[0040] Thus, the present invention provides consistent support from the preparation stage to the completion of a task, thereby improving work efficiency and reducing errors.

[0041] The following describes the processing flow.

[0042] Step 1:

[0043] The server accesses databases at network centers and communication exchanges to collect equipment information, work history, legal regulations, and security guidelines. By obtaining this information, the server gathers the basic data necessary for its operations.

[0044] Step 2:

[0045] The server analyzes the collected data and extracts information relevant to the task. Text mining techniques are used to identify important procedures and points to note, and the information is organized into categories.

[0046] Step 3:

[0047] The server automatically generates instructions based on the analysis results, including efficient work procedures, necessary tools, and safety checks. These instructions are systematically organized and presented in a format that is easy for workers to understand.

[0048] Step 4:

[0049] The terminal displays the generated instructions to the worker and prompts them to confirm the contents. The worker reviews the instructions on the terminal and completes a checklist on the terminal to confirm that they are ready. The terminal records the results of the preparation status check.

[0050] Step 5:

[0051] The user begins work on-site, and the terminal supports this process. The terminal receives input and voice commands from the worker and provides necessary information in real time through an AI agent.

[0052] Step 6:

[0053] The server monitors the progress of the work, provides real-time information, and detects and addresses anomalies. It can also provide workers with voice safety confirmations and warnings as needed.

[0054] Step 7:

[0055] After completing a task, the user submits a completion report using a terminal. The user inputs details of the work, any problems encountered, and areas for improvement into the terminal, which the server records and evaluates. This evaluation serves as the basis for future work instructions.

[0056] (Example 1)

[0057] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0058] Managing modern communications and network equipment requires processing large amounts of data and complex procedures. This necessitates efficient work procedures, worker safety, and real-time situational response. However, traditional methods are time-consuming for information gathering and analysis, lacking accuracy and speed. These issues raise concerns about decreased work efficiency and increased errors.

[0059] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0060] In this invention, the server includes means for an information processing device to access information sources via a network and collect information about equipment; means for analyzing the collected information and identifying work-related information using text processing technology; and means for creating instructions in electronic format based on the identified information and generating structured instructions. This enables each worker to perform tasks quickly and accurately, resulting in an efficient work process and improved worker safety.

[0061] An "information processing device" is a combination of hardware and software used to acquire, collect, and analyze information via a network.

[0062] "Information source" refers to external or internal databases or information systems that provide data about networks and communication equipment.

[0063] "Text processing technology" refers to techniques used to analyze relevant information from large amounts of text data and extract useful data.

[0064] "Work-related information" refers to information regarding instructions, procedures, and precautions that are essential for performing a particular task.

[0065] "Electronic instructions" refers to work instructions or guides that are displayed or delivered on a digital device.

[0066] "Structured instructions" are information or guidelines that are organized and systematically structured so that workers can easily understand and follow them.

[0067] "Worker" refers to a technician or operator who inspects, repairs, or maintains equipment on-site.

[0068] A "display device" refers to an electronic device that visually presents work instructions and other related information.

[0069] A "checklist" is a table or list that lists items used to prepare for work or to confirm procedures.

[0070] "Knowledge provision" refers to activities that provide necessary information and advice to workers in response to their requests.

[0071] This invention aims to improve work efficiency and safety using an information processing system. Specific embodiments are described below.

[0072] The server accesses the database system via the network to retrieve the necessary information. For example, it collects information about network equipment, past work history, and relevant laws and regulations. This is done using an open-source database management system, which queries the information to retrieve it. The collected data is then analyzed using a text analysis engine running on the server. During this process, natural language processing libraries are utilized to extract important procedures and precautions.

[0073] The server automatically generates work instructions in electronic format based on the extracted information. The generated instructions are structured in a format such as PDF and distributed from the server to the terminal. The work instructions include work procedures, a list of necessary tools, and safety checks to make them easy for workers to understand.

[0074] The terminal, acting as a client device, displays instructions received from the server to the on-site worker. This terminal may be a mobile information terminal equipped with a touchscreen. The worker reviews the displayed instructions and confirms the necessary preparations for the work. The preparation status is confirmed using a checklist displayed on the terminal, ensuring there are no deficiencies.

[0075] During the work process, the user (worker) can interact with the AI ​​agent through a terminal. For example, by typing "What should I do next?", the worker can receive answers and advice from the AI ​​agent. In this process, the server continuously monitors the progress of the work and immediately notifies the user if any anomalies are detected. The server can also provide voice guidance to ensure the worker's safety.

[0076] After completing a task, the user submits a completion report via their terminal. This report includes a summary of the work, any problems encountered, and suggestions for improvement for future tasks. The server evaluates the work based on this report and saves the results to a database. The accumulated data is used as feedback for future work instructions to further improve work efficiency.

[0077] A concrete example is the inspection of communication equipment. The server automatically creates inspection instructions based on past history and regulatory information, and a terminal displays them to the field worker. During the work, the worker can receive instructions from the AI ​​agent by typing "Please tell me the next step" into the terminal, allowing them to proceed with the work efficiently. In this way, the present invention utilizes information technology to support and optimize all stages of the work.

[0078] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0079] Step 1:

[0080] The server accesses information sources via the network to retrieve the necessary data. This input consists of detailed information about the equipment, including past work history and relevant regulatory information. The server queries this information using a database management system and outputs it as text data.

[0081] Step 2:

[0082] The server analyzes the acquired data. The input is the text data acquired in Step 1. The server analyzes this data using natural language processing technology and extracts necessary work procedures and precautions. As part of the data processing, the text analysis engine identifies important indicators and outputs them as structured information.

[0083] Step 3:

[0084] The server generates an electronic instruction sheet based on the analysis results. The input is the structured information obtained in step 2. The instruction sheet is formatted using LaTeX or similar and output in PDF format. The instruction sheet includes work procedures, necessary tools, and precautions, preparing them for presentation in the next stage.

[0085] Step 4:

[0086] The terminal receives instructions in PDF format sent from the server. The input is the electronic instructions generated in step 3. The instructions are displayed on the terminal using display software, making them easily accessible to the worker. This allows the worker to review the work content and prepare accordingly.

[0087] Step 5:

[0088] The user checks the instructions displayed on the terminal. The input here is the instructions displayed on the terminal in step 4. Based on the instructions, the user checks the preparation status via a checklist and checks the necessary boxes. The output is the confirmation result of whether preparation is complete.

[0089] Step 6:

[0090] During the task, the user interacts with the AI ​​agent in real time via a terminal. Input consists of voice commands and text input from the worker. The terminal transmits this to a server, where the generated AI model outputs optimal advice based on the input. The output is guidance information from the AI ​​agent.

[0091] Step 7:

[0092] The server monitors the progress of the work and detects anomalies. Input is data sent from the terminal during the work. Based on this, the server performs real-time data analysis and immediately outputs a notification in case of anomalies. Voice guidance may be output as safety confirmation support.

[0093] Step 8:

[0094] After completing a task, the user submits a completion report to the server from their terminal. The input is report data including the work performed, problems encountered, and areas for improvement. The server receives this data and stores it in a database. The evaluation results are analyzed and used as feedback when creating the next work order to improve work efficiency. The output is data for creating the next order, including improvement suggestions.

[0095] (Application Example 1)

[0096] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0097] In modern data management facilities, it is extremely difficult for workers to efficiently and safely perform tasks that require diverse procedures and split-second decisions. In such environments, rapid information provision and appropriate instructions are crucial for supporting work, but there is a lack of tools and systems to achieve this. Furthermore, optimization of visual and audible information presentation methods at the worksite is also required. Therefore, a system is needed that simultaneously improves work efficiency and strengthens safety measures.

[0098] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0099] In this invention, the server includes means for acquiring information from an information storage facility, means for analyzing the acquired information and extracting relevant information, and means for automatically generating instructions based on the extracted information. This enables workers to receive appropriate visual and audible instructions in real time, allowing them to perform their work quickly and safely.

[0100] An "information storage facility" refers to a storage device or database used for acquiring and managing data.

[0101] "Analysis" refers to the process of performing calculations to identify and extract relevant data from acquired information.

[0102] "Relevant information" refers to data and knowledge that are useful under specific conditions, as discovered through analysis.

[0103] An "instruction sheet" is a document that contains details and procedures for the tasks that workers are to perform.

[0104] "Workers" refers to individuals or teams who perform tasks based on instructions in a specific situation.

[0105] An "interface" refers to a device or software that provides a means of input and output for a user to interact with a system.

[0106] "Visual presentation" refers to the process of helping people understand information by displaying it in a visual form.

[0107] "Audio guidelines" refer to methods of providing guidance and instructions to workers through audio.

[0108] This invention provides a system for improving operational efficiency in data centers. It supports the entire process from start to finish through coordinated operation between servers, terminals, and users.

[0109] The server retrieves necessary data from the information storage. This information includes equipment details and past work history. The server analyzes this data and extracts relevant information. Text mining techniques are used for this analysis. Based on the relevant information, the server automatically generates instructions for workers and sends them to their terminals. The instructions include work procedures, safety checks, and necessary tools.

[0110] The terminal visually displays instructions to workers and provides an interface to check if the workers are ready. Furthermore, it accepts voice input and can provide voice guidelines and supplementary explanations from the server based on that input. Workers, as users, can check information in real time through the terminal and perform their tasks according to the instructions.

[0111] During the work process, the server monitors the work status through an AI agent and provides supplementary advice as needed. If an anomaly is detected, it can immediately propose countermeasures. In doing so, it provides information to the worker visually and audibly using ergonomic interfaces such as smart glasses.

[0112] Upon completion of a task, the user submits a report to their terminal, which is then sent to the server. The server evaluates the report and generates suggestions for improvement for future tasks. This process aims to improve the quality and efficiency of the work.

[0113] As a concrete example, in a standard server rack maintenance procedure within a data center, a worker wearing smart glasses can issue a voice command such as "Show me the details of the procedure," and the detailed procedure and related data will be displayed in real time. Furthermore, a generative AI model could utilize the prompt phrase, "Please tell me the standard server rack maintenance procedure in a data center."

[0114] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0115] Step 1:

[0116] The server retrieves data from the information storage, including detailed equipment information and past work history. This data is used as input for analysis on the server. At this stage, information is retrieved through APIs and database queries.

[0117] Step 2:

[0118] The server analyzes the acquired data and extracts relevant information. Here, text mining techniques are used to identify important procedures and points of caution within the information. In this analysis step, natural language processing techniques are used to compute the data, and the extracted information based on the results is used as input for the next step.

[0119] Step 3:

[0120] The server automatically generates work instructions based on the extracted relevant information. These instructions are organized documents containing work procedures, safety checks, and a list of necessary tools. The generated instructions are provided as output to the terminal and serve as input data for the next process. Here, a template-based document generation function is used.

[0121] Step 4:

[0122] The terminal visually displays the work instructions received from the server to the worker. An interface is also provided to prompt the worker (user) to make the necessary preparations, and the worker proceeds with preparations based on this information. Here, the instructions are displayed via a screen.

[0123] Step 5:

[0124] During the process, the user gives instructions to the AI ​​agent via voice input through the terminal. The terminal sends this voice information to a server, which analyzes the input. The generating AI model calculates appropriate advice and returns the result to the terminal as output. This step includes processing by the speech recognition system.

[0125] Step 6:

[0126] After the user completes their task, the terminal sends the work details and report to the server. The server uses this input information to evaluate the performance of the task and calculates suggestions for improvement for the next task. The final output is an updated instruction sheet to be used for the next task.

[0127] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0128] This invention aims to improve work efficiency and create a safe and secure work environment by combining an emotion engine with work support systems in network centers and communication exchanges, thereby recognizing, analyzing, and responding to the emotional state of workers in real time. This system consists of coordinated operation between servers, terminals, and users, and is implemented as follows.

[0129] First, the server, as before, collects and analyzes relevant information such as equipment information and work history from various databases. Then, an automatically generated instruction sheet based on this information is presented to the terminal of the field worker.

[0130] In this system, the terminal is equipped with an emotion engine that analyzes the worker's facial expressions and voice in real time through the camera and microphone. For example, if a user is feeling fatigued or stressed while working, the emotion engine detects this and sends the analysis results to the server.

[0131] The server receives data from the emotion engine and provides appropriate feedback to the worker. This feedback includes relaxation techniques to reduce stress, advice on how to proceed with work, and suggestions for appropriate breaks. For example, if the server detects that the worker's concentration is waning, it can notify them to take a break and provide voice guidance on stretching during that time.

[0132] Furthermore, users (workers) can check the emotional engine's feedback through their devices and manage their own work pace and state. The data recorded by the emotional engine is analyzed over the long term and used to improve the work environment and enhance performance. This makes it possible to provide more user-friendly work instructions that take the emotional aspects of workers into consideration.

[0133] By combining this with an emotional engine, it becomes possible to add a higher level of safety and personalized support to conventional work efficiency systems. This system reduces the burden on workers and provides a safe and efficient work environment.

[0134] The following describes the processing flow.

[0135] Step 1:

[0136] The server accesses databases related to the daily operations of network centers and communication exchanges, collecting necessary equipment information, past work history, and information on legal regulations. This allows for the acquisition of basic information necessary for preparing for work.

[0137] Step 2:

[0138] The server analyzes the collected data to extract specific procedures and precautions necessary for the work. Text mining technology is used in this process, and work instructions are automatically generated based on the analysis results. These instructions include efficient work procedures and safety checks.

[0139] Step 3:

[0140] The terminal displays instructions generated on the server to the worker. The worker reviews the instructions and uses a checklist to verify that all necessary preparations are complete. The terminal records this preparation status and the results of the verification.

[0141] Step 4:

[0142] The emotion engine built into the terminal uses a camera and microphone to monitor the worker's facial expressions and voice, recognizing their current emotional state. For example, if a worker is showing signs of anxiety or stress, it can be detected immediately.

[0143] Step 5:

[0144] Based on data from the emotion engine, the server creates feedback based on the worker's emotional state. For example, if the server detects that the worker is stressed, it will suggest relaxation techniques or advise them to take a short break.

[0145] Step 6:

[0146] The user (worker) receives emotional feedback through the terminal and continues working or takes a break according to the advice provided by the system. This promotes self-management of pace and improves safety.

[0147] Step 7:

[0148] After completing a task, the user submits a completion report using their terminal. The report details the progress of the task, any problems encountered, and the user's emotional state, which the server then records in a database. This recorded data is used to improve future tasks, leading to continuous efficiency improvements.

[0149] (Example 2)

[0150] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0151] To improve work efficiency and safety on-site, it is necessary to understand workers' emotional states in real time and provide appropriate feedback based on that understanding. However, conventional systems have not been sufficient in understanding and responding to such emotional states. Proceeding with work without considering emotional states can lead to decreased work efficiency and safety problems. Therefore, work support that takes emotional states into account is necessary.

[0152] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0153] In this invention, the server includes means for acquiring data from a data storage device, means for analyzing the acquired data and extracting relevant data, and means for detecting the emotional state of an operator and transmitting the analyzed data to a higher-level device. This makes it possible to provide situation-appropriate feedback in real time based on the emotional state of the operator, thereby improving work efficiency and safety.

[0154] "Data storage device" is a general term for storage media and devices used to store various types of data, and includes databases and server storage.

[0155] "Means of retrieving data" refers to the processes and technologies used to access and retrieve stored data, and includes database operations using query languages.

[0156] "Means of analyzing data and extracting relevant data" refers to methods of computationally processing acquired data and extracting useful information, and includes the use of algorithms and statistical models.

[0157] "Means for automatically generating instruction information" refers to a system that automatically creates necessary procedures and points to note using a computer based on collected data.

[0158] "Means of displaying instructional information to workers and performing necessary preparation checks" refers to the process of presenting information on a screen or other device to help workers follow instructions and confirming their readiness.

[0159] "A means of receiving information from workers and providing that information in real time" refers to a system that processes the information entered by workers immediately and provides the results right away.

[0160] "Means for detecting emotional states and transmitting the analysis data to a higher-level device" refers to means for measuring and analyzing the emotions of workers and transmitting the results to a central system.

[0161] "Means for generating and presenting feedback" refers to the process of automatically creating useful advice and information based on analysis results and communicating it to the worker.

[0162] "Means of recording report contents and making improvements for the next work" refers to an approach that saves work history and uses it to improve the efficiency and safety of the next work.

[0163] The present invention is a system for improving work efficiency and safety on site, and its embodiments are as follows.

[0164] The server first retrieves the necessary data from the data storage device. This data includes equipment records and past work history, and is retrieved using SQL queries. The server then analyzes the data, extracts relevant information, and automatically generates instruction information. Decision tree models and random forests are commonly used as machine learning algorithms. The generated instruction information is then sent to the worker's terminal.

[0165] The terminal is equipped with an emotion engine to analyze the emotional state of the worker. This engine uses libraries such as OpenCV and PyAudio to analyze the worker's facial expressions and voice in real time through the camera and microphone. The terminal has the function to send the analyzed emotional state data to a server, and the emotion analysis engine uses a convolutional neural network (CNN).

[0166] When the server receives emotional state data sent from the terminal, it creates appropriate feedback based on this data. Using a generative AI model, it generates advice according to the prompt. An example of a prompt is, "Please provide effective relaxation methods if the user is feeling fatigued." The feedback content, including suggestions for breaks and improvements to work methods, is sent to the worker's terminal.

[0167] The user, as the worker, can check the feedback provided through the terminal and optimally manage their work status. For example, by taking breaks according to the recommended relaxation methods, the worker can regain their concentration. Furthermore, the feedback content may also be guided to the worker by a voice assistant, making it highly convenient even in work situations where visual input is unavailable.

[0168] Thus, the system of the present invention can provide real-time feedback based on the worker's emotional state and work history, thereby supporting an efficient and safe work environment.

[0169] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0170] Step 1:

[0171] The server retrieves equipment information and work history from data storage devices. The input is a database, and the specific technology used is SQL queries. The output is an aggregation of work-related data. This prepares the data necessary for subsequent analysis and instruction generation.

[0172] Step 2:

[0173] The server analyzes the acquired data and extracts relevant data. Specifically, machine learning algorithms (e.g., decision trees and random forests) are used. The input is the data aggregated in step 1, and the output is automatically generated instruction information. This process helps to develop efficient work procedures.

[0174] Step 3:

[0175] The server converts the generated instruction information into JSON format and sends it to the terminal. The input is the instruction information, and the output is the data displayed on the terminal. At this stage, the worker can confirm the necessary preparations.

[0176] Step 4:

[0177] The terminal uses its built-in camera and microphone to collect the worker's facial expressions and voice in real time. The input is the worker's visual and audio data, and the output is data for analyzing their emotional state. Specific technologies used include the OpenCV and PyAudio libraries.

[0178] Step 5:

[0179] The terminal feeds the collected data into an emotion engine to analyze the emotional state. A convolutional neural network (CNN) model is used here. The input is the data obtained in step 4, and the output is the analyzed emotional state data. This analysis quantifies the worker's emotional state.

[0180] Step 6:

[0181] The terminal sends the results of the emotional state analysis to the server. The input is the analyzed emotional data, and the output is the data to be sent to the higher-level device. HTTPS is used as the specific communication protocol.

[0182] Step 7:

[0183] The server uses a generative AI model based on emotional data to create feedback. The input is emotional state data, and the output is feedback information provided to the worker. An example of a prompt is, "Provide effective relaxation methods for when the user is feeling fatigued."

[0184] Step 8:

[0185] The server sends the generated feedback to the terminal, which then presents it to the worker visually and audibly. The input is the feedback information, and the output is the guidance information received by the worker. Specifically, the feedback is communicated via screen displays and voice assistants.

[0186] Step 9:

[0187] The user (worker) manages their work status based on the feedback provided and adjusts their work methods and breaks as needed. The input is feedback information from the terminal, and the output is the managed work status. This step allows the worker to continue working efficiently and safely.

[0188] (Application Example 2)

[0189] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0190] In modern production environments, it is known that the emotional state of workers significantly impacts work efficiency and safety. However, in many workplaces, this factor is often overlooked, and worker fatigue and stress are sometimes ignored. This can lead to decreased work efficiency and compromised safety, highlighting the need for a system that monitors workers' emotional states in real time and provides appropriate feedback.

[0191] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0192] In this invention, the server includes means for receiving information from a data storage device, means for analyzing the received information to derive relevant information, and means for analyzing the emotional state of workers using an emotion analysis engine and providing appropriate feedback based on the analysis results. This makes it possible to grasp the emotional state of workers in real time and improve work efficiency and safety through appropriate feedback.

[0193] A "data storage device" is a device that stores various types of information over a long period of time and allows it to be searched and retrieved as needed.

[0194] "Means of receiving information" refers to the processes and technologies used to obtain necessary data through external data communication.

[0195] "Means of information analysis" refer to the technologies and algorithms used to analyze received data and derive useful information.

[0196] "Means for deriving relevant information" refers to methods for extracting meaningful information related to a specific purpose from analyzed data.

[0197] "Means for automatically generating instructions" refers to technology that automatically creates guidelines defining the tasks and actions that humans should perform based on the derived information.

[0198] "Means of presenting instructions to workers" refers to procedures or devices for explicitly communicating generated instructions to the person in charge of the work.

[0199] "Means of supplying information" refers to a system that provides necessary data and feedback immediately during work.

[0200] An "emotion analysis engine" is a technology that automatically detects and analyzes the emotional state of workers based on their facial expressions, voice characteristics, and other factors.

[0201] "Means of providing feedback" refer to techniques and devices for communicating advice and information based on sentiment analysis to workers.

[0202] "Means for accumulating report content" refers to a system that records data and results after work is completed, and uses them for later analysis and improvement.

[0203] "Methods for creating improvement measures" refers to the process of using accumulated work data and sentiment analysis results to derive areas for improvement in work procedures and the work environment.

[0204] The system that realizes this invention mainly consists of a server, a terminal (in this case, smart glasses), and an operator.

[0205] The server receives information stored in data storage devices, analyzes the information, and derives relevant data. Specifically, it functions as the core for automatically generating efficient instructions based on past work data and environmental information. The server also plays a role in providing feedback to workers, such as emotion-based advice and relaxation techniques, based on data provided by the emotion analysis engine. For this purpose, a cloud-based data processing platform and an analysis programming environment such as Python or R are likely to be used.

[0206] The smart glasses used as terminals are equipped with a camera and microphone, capturing the worker's facial expressions and voice in real time. Based on this, an emotion analysis engine analyzes the worker's emotional state. Based on the analysis results, instructions and feedback created by the server are displayed on the glasses' screen. For example, if a worker is determined to be irritated during work, they may receive instructions such as, "Take a deep breath and try meditating for one minute." An example of a prompt message is, "Please determine your stress level based on the current voice and facial expression data."

[0207] By receiving this feedback, users (workers) can manage their own work pace and status in real time. Furthermore, the data recorded by the emotion engine is accumulated over the long term and used for analysis to improve the work environment and enhance individual performance. This enables flexible and efficient work support tailored to individual needs.

[0208] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0209] Step 1:

[0210] The server retrieves work-related information and past work data from the database as input data. It analyzes this data and automatically generates instructions tailored to the work content. The generated instructions are then used as output data to be sent to the terminal. This process includes executing database queries and organizing and analyzing the obtained data.

[0211] Step 2:

[0212] The terminal uses its built-in camera and microphone to collect the worker's facial expressions and voice as input. Based on this input data, an emotion analysis engine calculates the worker's emotional state and outputs the analysis results to the server. Specifically, it performs facial expression analysis using image recognition technology and voice recognition.

[0213] Step 3:

[0214] The server uses emotion analysis results received from terminals and worker productivity data as input to evaluate stress levels and concentration levels. Based on this, it generates appropriate feedback and break instructions for the workers. Output data includes relaxation techniques and concise instructions. This step utilizes a generative AI model to integrate and analyze the collected data.

[0215] Step 4:

[0216] The terminal displays feedback and instructions received from the server on the smart glasses' display, notifying the worker. This display content is then used as output to prompt the user to take appropriate action. Specifically, this involves popping up messages or playing audio guidance.

[0217] Step 5:

[0218] The user (worker) checks notifications on their device and adjusts their work pace or takes breaks according to the feedback. This step requires the worker to take actions that maintain their own productivity and safety.

[0219] Step 6:

[0220] The server receives the report as input after the work is completed and derives improvement measures for the next task. This generates output data containing proposed improvements for future work instructions. The data is stored and analyzed over the long term to optimize the work environment.

[0221] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0222] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0223] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0224] [Second Embodiment]

[0225] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0226] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0227] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0228] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0229] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0230] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0231] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0232] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0233] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0234] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0235] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0236] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0237] The present invention provides an automation solution for streamlining operations in network centers and telecommunications exchanges. This system primarily consists of coordinated operation between servers, terminals, and users. Specific embodiments of the system are described below.

[0238] First, the server accesses a vast database of network and communication equipment, collecting detailed information on each piece of equipment, past work history, relevant laws and regulations, and guidelines. Next, the server analyzes the collected data and extracts relevant information necessary for on-site work. In this process, the analysis engine utilizes text mining technology to mechanically identify important procedures and points of caution from the data.

[0239] Based on the analyzed information, the server automatically generates instruction manuals and work instructions. The generated instructions are structured to be easy for workers to understand and include detailed information on efficient work procedures, necessary tool lists, and safety checks.

[0240] Next, the terminal presents the generated instructions to the field worker. The worker uses the terminal to review the instructions and check the checklist to ensure they are prepared. At this stage, the terminal records the worker's preparation status and provides an interface to check for any deficiencies in preparation.

[0241] Once work begins, the user (worker) can interact with the AI ​​agent in real time through a terminal. The terminal receives input and voice commands from the worker, and based on this, the AI ​​agent provides information and advice regarding the work. During this time, the server continuously monitors the progress of the work and immediately proposes countermeasures if any abnormalities occur. To ensure the safety of the worker and to ensure efficient work execution, the server also provides voice-based safety confirmation support as needed.

[0242] After completing a task, the user submits a completion report to their terminal. The completed work, any problems, and areas for improvement are entered, and the server evaluates the work based on this information. The evaluation results are stored in a database and used to improve efficiency and efficiency in future work. Based on the report, the server analyzes the performance of each task and generates improvement suggestions to incorporate the feedback into future work instructions.

[0243] Thus, the present invention provides consistent support from the preparation stage to the completion of a task, thereby improving work efficiency and reducing errors.

[0244] The following describes the processing flow.

[0245] Step 1:

[0246] The server accesses databases at network centers and communication exchanges to collect equipment information, work history, legal regulations, and security guidelines. By obtaining this information, the server gathers the basic data necessary for its operations.

[0247] Step 2:

[0248] The server analyzes the collected data and extracts information relevant to the task. Text mining techniques are used to identify important procedures and points to note, and the information is organized into categories.

[0249] Step 3:

[0250] The server automatically generates instructions based on the analysis results, including efficient work procedures, necessary tools, and safety checks. These instructions are systematically organized and presented in a format that is easy for workers to understand.

[0251] Step 4:

[0252] The terminal displays the generated instructions to the worker and prompts them to confirm the contents. The worker reviews the instructions on the terminal and completes a checklist on the terminal to confirm that they are ready. The terminal records the results of the preparation status check.

[0253] Step 5:

[0254] The user begins work on-site, and the terminal supports this process. The terminal receives input and voice commands from the worker and provides necessary information in real time through an AI agent.

[0255] Step 6:

[0256] The server monitors the progress of the work, provides real-time information, and detects and addresses anomalies. It can also provide workers with voice safety confirmations and warnings as needed.

[0257] Step 7:

[0258] After completing a task, the user submits a completion report using a terminal. The user inputs details of the work, any problems encountered, and areas for improvement into the terminal, which the server records and evaluates. This evaluation serves as the basis for future work instructions.

[0259] (Example 1)

[0260] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0261] Managing modern communications and network equipment requires processing large amounts of data and complex procedures. This necessitates efficient work procedures, worker safety, and real-time situational response. However, traditional methods are time-consuming for information gathering and analysis, lacking accuracy and speed. These issues raise concerns about decreased work efficiency and increased errors.

[0262] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0263] In this invention, the server includes means for an information processing device to access information sources via a network and collect information about equipment; means for analyzing the collected information and identifying work-related information using text processing technology; and means for creating instructions in electronic format based on the identified information and generating structured instructions. This enables each worker to perform tasks quickly and accurately, resulting in an efficient work process and improved worker safety.

[0264] An "information processing device" is a combination of hardware and software used to acquire, collect, and analyze information via a network.

[0265] "Information source" refers to external or internal databases or information systems that provide data about networks and communication equipment.

[0266] "Text processing technology" refers to techniques used to analyze relevant information from large amounts of text data and extract useful data.

[0267] "Work-related information" refers to information regarding instructions, procedures, and precautions that are essential for performing a particular task.

[0268] "Electronic instructions" refers to work instructions or guides that are displayed or delivered on a digital device.

[0269] "Structured instructions" are information or guidelines that are organized and systematically structured so that workers can easily understand and follow them.

[0270] "Worker" refers to a technician or operator who inspects, repairs, or maintains equipment on-site.

[0271] A "display device" refers to an electronic device that visually presents work instructions and other related information.

[0272] A "checklist" is a table or list that lists items used to prepare for work or to confirm procedures.

[0273] "Knowledge provision" refers to activities that provide necessary information and advice to workers in response to their requests.

[0274] This invention aims to improve work efficiency and safety using an information processing system. Specific embodiments are described below.

[0275] The server accesses the database system via the network to retrieve the necessary information. For example, it collects information about network equipment, past work history, and relevant laws and regulations. This is done using an open-source database management system, which queries the information to retrieve it. The collected data is then analyzed using a text analysis engine running on the server. During this process, natural language processing libraries are utilized to extract important procedures and precautions.

[0276] The server automatically generates work instructions in electronic format based on the extracted information. The generated instructions are structured in a format such as PDF and distributed from the server to the terminal. The work instructions include work procedures, a list of necessary tools, and safety checks to make them easy for workers to understand.

[0277] The terminal, acting as a client device, displays instructions received from the server to the on-site worker. This terminal may be a mobile information terminal equipped with a touchscreen. The worker reviews the displayed instructions and confirms the necessary preparations for the work. The preparation status is confirmed using a checklist displayed on the terminal, ensuring there are no deficiencies.

[0278] During the operation, the user, i.e., the operator, can interact with the AI agent through the terminal. For example, by entering "What should I do next?" the operator can receive answers and advice from the AI agent. In this process, the server continuously monitors the progress of the operation and immediately notifies if any abnormalities are detected. The server can also provide voice guidance to ensure the safety of the operator.

[0279] After the operation is completed, the user reports the completion of the operation through the terminal. This report includes a summary of the operation content, problems, and improvement points for the next time. The server evaluates the operation based on this report and saves the results in the database. The accumulated data is utilized as feedback in the next work instruction to further improve work efficiency.

[0280] As a specific example, there is an inspection operation for communication equipment. The server automatically creates an inspection instruction based on past history and regulatory information, and the terminal displays it to the on-site operator. During the operation, by entering "Please tell me the next step" into the terminal, the operator can receive instructions from the AI agent and proceed with the operation efficiently. Thus, the present invention utilizes information technology to support and optimize all stages of the operation.

[0281] The flow of the specific process in Example 1 will be described using FIG. 11.

[0282] Step 1:

[0283] The server accesses the information source via the network to obtain the necessary data. This input is detailed information about the equipment, including past work history and relevant regulatory information. The server queries this information using a database management system and outputs it as text-formatted data.

[0284] Step 2:

[0285] The server analyzes the acquired data. The input is the text-formatted data acquired in Step 1. The server uses natural language processing technology to analyze this data and extracts the necessary work procedures and precautions. As data processing, the text analysis engine identifies important indicators and outputs them as structured information.

[0286] Step 3:

[0287] The server generates an electronic instruction based on the analysis results. The input is the structured information obtained in Step 2. The instruction is formatted using LaTeX or the like and output in PDF format. The instruction includes work procedures, necessary tools, and precautions, and is prepared for presentation in the next stage.

[0288] Step 4:

[0289] The terminal receives the PDF-formatted instruction sent from the server. The input is the electronic instruction generated in Step 3. The instruction is displayed using display software on the terminal to make it easily accessible to the operator. Thereby, the operator can check the work content and get ready.

[0290] Step 5:

[0291] The user checks the instruction displayed on the terminal. The input here is the instruction displayed on the terminal in Step 4. The user checks the preparation status based on the instruction via a checklist and checks the necessary columns. The output is the confirmation result of whether the preparation is complete.

[0292] Step 6:

[0293] During the work, the user interacts with the AI agent in real time via the terminal. The input is voice commands or text input from the operator. The terminal transfers this to the server, and the generated AI model outputs optimal advice based on the input. The output is the guidance information from the AI agent.

[0294] Step 7:

[0295] The server monitors the progress of the work and detects anomalies. Input is data sent from the terminal during the work. Based on this, the server performs real-time data analysis and immediately outputs a notification in case of anomalies. Voice guidance may be output as safety confirmation support.

[0296] Step 8:

[0297] After completing a task, the user submits a completion report to the server from their terminal. The input is report data including the work performed, problems encountered, and areas for improvement. The server receives this data and stores it in a database. The evaluation results are analyzed and used as feedback when creating the next work order to improve work efficiency. The output is data for creating the next order, including improvement suggestions.

[0298] (Application Example 1)

[0299] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0300] In modern data management facilities, it is extremely difficult for workers to efficiently and safely perform tasks that require diverse procedures and split-second decisions. In such environments, rapid information provision and appropriate instructions are crucial for supporting work, but there is a lack of tools and systems to achieve this. Furthermore, optimization of visual and audible information presentation methods at the worksite is also required. Therefore, a system is needed that simultaneously improves work efficiency and strengthens safety measures.

[0301] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0302] In this invention, the server includes means for acquiring information from an information storage, means for analyzing the acquired information to extract relevant information, and means for automatically generating an instruction sheet based on the extracted information. As a result, it becomes possible for workers to receive appropriate instructions visually and aurally in real time and to perform work quickly and safely.

[0303] The "information storage" refers to a storage device or database for acquiring and managing data.

[0304] "Analysis" means performing a computational process for identifying and extracting relevant data from the acquired information.

[0305] "Relevant information" is data or knowledge useful under specific conditions found through analysis.

[0306] An "instruction sheet" is a document containing details and procedures of the work to be performed by workers.

[0307] "Workers" refer to an individual or team who perform work based on an instruction sheet in a specific scenario.

[0308] An "interface" refers to a device or software that provides means for input and output for a user to interact with a system.

[0309] "Visual presentation" refers to a process of assisting the understanding of content by visualizing and displaying information.

[0310] "Aural guidelines" is a method for guiding and instructing workers through sound.

[0311] This invention provides a system for improving work efficiency in a data center. Through the collaborative operation among the server, the terminal, and the user, it supports the process from the start to the completion of work.

[0312] The server retrieves necessary data from the information storage. This information includes equipment details and past work history. The server analyzes this data and extracts relevant information. Text mining techniques are used for this analysis. Based on the relevant information, the server automatically generates instructions for workers and sends them to their terminals. The instructions include work procedures, safety checks, and necessary tools.

[0313] The terminal visually displays instructions to workers and provides an interface to check if the workers are ready. Furthermore, it accepts voice input and can provide voice guidelines and supplementary explanations from the server based on that input. Workers, as users, can check information in real time through the terminal and perform their tasks according to the instructions.

[0314] During the work process, the server monitors the work status through an AI agent and provides supplementary advice as needed. If an anomaly is detected, it can immediately propose countermeasures. In doing so, it provides information to the worker visually and audibly using ergonomic interfaces such as smart glasses.

[0315] Upon completion of a task, the user submits a report to their terminal, which is then sent to the server. The server evaluates the report and generates suggestions for improvement for future tasks. This process aims to improve the quality and efficiency of the work.

[0316] As a concrete example, in a standard server rack maintenance procedure within a data center, a worker wearing smart glasses can issue a voice command such as "Show me the details of the procedure," and the detailed procedure and related data will be displayed in real time. Furthermore, a generative AI model could utilize the prompt phrase, "Please tell me the standard server rack maintenance procedure in a data center."

[0317] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0318] Step 1:

[0319] The server retrieves data from the information storage, including detailed equipment information and past work history. This data is used as input for analysis on the server. At this stage, information is retrieved through APIs and database queries.

[0320] Step 2:

[0321] The server analyzes the acquired data and extracts relevant information. Here, text mining techniques are used to identify important procedures and points of caution within the information. In this analysis step, natural language processing techniques are used to compute the data, and the extracted information based on the results is used as input for the next step.

[0322] Step 3:

[0323] The server automatically generates work instructions based on the extracted relevant information. These instructions are organized documents containing work procedures, safety checks, and a list of necessary tools. The generated instructions are provided as output to the terminal and serve as input data for the next process. Here, a template-based document generation function is used.

[0324] Step 4:

[0325] The terminal visually displays the work instructions received from the server to the worker. An interface is also provided to prompt the worker (user) to make the necessary preparations, and the worker proceeds with preparations based on this information. Here, the instructions are displayed via a screen.

[0326] Step 5:

[0327] During the process, the user gives instructions to the AI ​​agent via voice input through the terminal. The terminal sends this voice information to a server, which analyzes the input. The generating AI model calculates appropriate advice and returns the result to the terminal as output. This step includes processing by the speech recognition system.

[0328] Step 6:

[0329] After the user completes their task, the terminal sends the work details and report to the server. The server uses this input information to evaluate the performance of the task and calculates suggestions for improvement for the next task. The final output is an updated instruction sheet to be used for the next task.

[0330] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0331] This invention aims to improve work efficiency and create a safe and secure work environment by combining an emotion engine with work support systems in network centers and communication exchanges, thereby recognizing, analyzing, and responding to the emotional state of workers in real time. This system consists of coordinated operation between servers, terminals, and users, and is implemented as follows.

[0332] First, the server, as before, collects and analyzes relevant information such as equipment information and work history from various databases. Then, an automatically generated instruction sheet based on this information is presented to the terminal of the field worker.

[0333] In this system, the terminal is equipped with an emotion engine that analyzes the worker's facial expressions and voice in real time through the camera and microphone. For example, if a user is feeling fatigued or stressed while working, the emotion engine detects this and sends the analysis results to the server.

[0334] The server receives data from the emotion engine and provides appropriate feedback to the worker. This feedback includes relaxation techniques to reduce stress, advice on how to proceed with work, and suggestions for appropriate breaks. For example, if the server detects that the worker's concentration is waning, it can notify them to take a break and provide voice guidance on stretching during that time.

[0335] Furthermore, users (workers) can check the emotional engine's feedback through their devices and manage their own work pace and state. The data recorded by the emotional engine is analyzed over the long term and used to improve the work environment and enhance performance. This makes it possible to provide more user-friendly work instructions that take the emotional aspects of workers into consideration.

[0336] By combining this with an emotional engine, it becomes possible to add a higher level of safety and personalized support to conventional work efficiency systems. This system reduces the burden on workers and provides a safe and efficient work environment.

[0337] The following describes the processing flow.

[0338] Step 1:

[0339] The server accesses databases related to the daily operations of network centers and communication exchanges, collecting necessary equipment information, past work history, and information on legal regulations. This allows for the acquisition of basic information necessary for preparing for work.

[0340] Step 2:

[0341] The server analyzes the collected data to extract specific procedures and precautions necessary for the work. Text mining technology is used in this process, and work instructions are automatically generated based on the analysis results. These instructions include efficient work procedures and safety checks.

[0342] Step 3:

[0343] The terminal displays instructions generated on the server to the worker. The worker reviews the instructions and uses a checklist to verify that all necessary preparations are complete. The terminal records this preparation status and the results of the verification.

[0344] Step 4:

[0345] The emotion engine built into the terminal uses a camera and microphone to monitor the worker's facial expressions and voice, recognizing their current emotional state. For example, if a worker is showing signs of anxiety or stress, it can be detected immediately.

[0346] Step 5:

[0347] Based on data from the emotion engine, the server creates feedback based on the worker's emotional state. For example, if the server detects that the worker is stressed, it will suggest relaxation techniques or advise them to take a short break.

[0348] Step 6:

[0349] The user (worker) receives emotional feedback through the terminal and continues working or takes a break according to the advice provided by the system. This promotes self-management of pace and improves safety.

[0350] Step 7:

[0351] After completing a task, the user submits a completion report using their terminal. The report details the progress of the task, any problems encountered, and the user's emotional state, which the server then records in a database. This recorded data is used to improve future tasks, leading to continuous efficiency improvements.

[0352] (Example 2)

[0353] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0354] To improve work efficiency and safety on-site, it is necessary to understand workers' emotional states in real time and provide appropriate feedback based on that understanding. However, conventional systems have not been sufficient in understanding and responding to such emotional states. Proceeding with work without considering emotional states can lead to decreased work efficiency and safety problems. Therefore, work support that takes emotional states into account is necessary.

[0355] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0356] In this invention, the server includes means for acquiring data from a data storage device, means for analyzing the acquired data and extracting relevant data, and means for detecting the emotional state of an operator and transmitting the analyzed data to a higher-level device. This makes it possible to provide situation-appropriate feedback in real time based on the emotional state of the operator, thereby improving work efficiency and safety.

[0357] "Data storage device" is a general term for storage media and devices used to store various types of data, and includes databases and server storage.

[0358] "Means of retrieving data" refers to the processes and technologies used to access and retrieve stored data, and includes database operations using query languages.

[0359] "Means of analyzing data and extracting relevant data" refers to methods of computationally processing acquired data and extracting useful information, and includes the use of algorithms and statistical models.

[0360] "Means for automatically generating instruction information" refers to a system that automatically creates necessary procedures and points to note using a computer based on collected data.

[0361] "Means of displaying instructional information to workers and performing necessary preparation checks" refers to the process of presenting information on a screen or other device to help workers follow instructions and confirming their readiness.

[0362] "A means of receiving information from workers and providing that information in real time" refers to a system that processes the information entered by workers immediately and provides the results right away.

[0363] "Means for detecting emotional states and transmitting the analysis data to a higher-level device" refers to means for measuring and analyzing the emotions of workers and transmitting the results to a central system.

[0364] "Means for generating and presenting feedback" refers to the process of automatically creating useful advice and information based on analysis results and communicating it to the worker.

[0365] "Means of recording report contents and making improvements for the next work" refers to an approach that saves work history and uses it to improve the efficiency and safety of the next work.

[0366] The present invention is a system for improving work efficiency and safety on site, and its embodiments are as follows.

[0367] The server first retrieves the necessary data from the data storage device. This data includes equipment records and past work history, and is retrieved using SQL queries. The server then analyzes the data, extracts relevant information, and automatically generates instruction information. Decision tree models and random forests are commonly used as machine learning algorithms. The generated instruction information is then sent to the worker's terminal.

[0368] The terminal is equipped with an emotion engine to analyze the emotional state of the worker. This engine uses libraries such as OpenCV and PyAudio to analyze the worker's facial expressions and voice in real time through the camera and microphone. The terminal has the function to send the analyzed emotional state data to a server, and the emotion analysis engine uses a convolutional neural network (CNN).

[0369] When the server receives emotional state data sent from the terminal, it creates appropriate feedback based on this data. Using a generative AI model, it generates advice according to the prompt. An example of a prompt is, "Please provide effective relaxation methods if the user is feeling fatigued." The feedback content, including suggestions for breaks and improvements to work methods, is sent to the worker's terminal.

[0370] The user, as the worker, can check the feedback provided through the terminal and optimally manage their work status. For example, by taking breaks according to the recommended relaxation methods, the worker can regain their concentration. Furthermore, the feedback content may also be guided to the worker by a voice assistant, making it highly convenient even in work situations where visual input is unavailable.

[0371] Thus, the system of the present invention can provide real-time feedback based on the worker's emotional state and work history, thereby supporting an efficient and safe work environment.

[0372] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0373] Step 1:

[0374] The server retrieves equipment information and work history from data storage devices. The input is a database, and the specific technology used is SQL queries. The output is an aggregation of work-related data. This prepares the data necessary for subsequent analysis and instruction generation.

[0375] Step 2:

[0376] The server analyzes the acquired data and extracts relevant data. Specifically, machine learning algorithms (e.g., decision trees and random forests) are used. The input is the data aggregated in step 1, and the output is automatically generated instruction information. This process helps to develop efficient work procedures.

[0377] Step 3:

[0378] The server converts the generated instruction information into JSON format and sends it to the terminal. The input is the instruction information, and the output is the data displayed on the terminal. At this stage, the worker can confirm the necessary preparations.

[0379] Step 4:

[0380] The terminal uses its built-in camera and microphone to collect the worker's facial expressions and voice in real time. The input is the worker's visual and audio data, and the output is data for analyzing their emotional state. Specific technologies used include the OpenCV and PyAudio libraries.

[0381] Step 5:

[0382] The terminal feeds the collected data into an emotion engine to analyze the emotional state. A convolutional neural network (CNN) model is used here. The input is the data obtained in step 4, and the output is the analyzed emotional state data. This analysis quantifies the worker's emotional state.

[0383] Step 6:

[0384] The terminal sends the results of the emotional state analysis to the server. The input is the analyzed emotional data, and the output is the data to be sent to the higher-level device. HTTPS is used as the specific communication protocol.

[0385] Step 7:

[0386] The server uses a generative AI model based on emotional data to create feedback. The input is emotional state data, and the output is feedback information provided to the worker. An example of a prompt is, "Provide effective relaxation methods for when the user is feeling fatigued."

[0387] Step 8:

[0388] The server sends the generated feedback to the terminal, which then presents it to the worker visually and audibly. The input is the feedback information, and the output is the guidance information received by the worker. Specifically, the feedback is communicated via screen displays and voice assistants.

[0389] Step 9:

[0390] The user (worker) manages their work status based on the feedback provided and adjusts their work methods and breaks as needed. The input is feedback information from the terminal, and the output is the managed work status. This step allows the worker to continue working efficiently and safely.

[0391] (Application Example 2)

[0392] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0393] In modern production environments, it is known that the emotional state of workers significantly impacts work efficiency and safety. However, in many workplaces, this factor is often overlooked, and worker fatigue and stress are sometimes ignored. This can lead to decreased work efficiency and compromised safety, highlighting the need for a system that monitors workers' emotional states in real time and provides appropriate feedback.

[0394] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0395] In this invention, the server includes means for receiving information from a data storage device, means for analyzing the received information to derive relevant information, and means for analyzing the emotional state of workers using an emotion analysis engine and providing appropriate feedback based on the analysis results. This makes it possible to grasp the emotional state of workers in real time and improve work efficiency and safety through appropriate feedback.

[0396] A "data storage device" is a device that stores various types of information over a long period of time and allows it to be searched and retrieved as needed.

[0397] "Means of receiving information" refers to the processes and technologies used to obtain necessary data through external data communication.

[0398] "Means of information analysis" refer to the technologies and algorithms used to analyze received data and derive useful information.

[0399] "Means for deriving relevant information" refers to methods for extracting meaningful information related to a specific purpose from analyzed data.

[0400] "Means for automatically generating instructions" refers to technology that automatically creates guidelines defining the tasks and actions that humans should perform based on the derived information.

[0401] "Means of presenting instructions to workers" refers to procedures or devices for explicitly communicating generated instructions to the person in charge of the work.

[0402] "Means of supplying information" refers to a system that provides necessary data and feedback immediately during work.

[0403] An "emotion analysis engine" is a technology that automatically detects and analyzes the emotional state of workers based on their facial expressions, voice characteristics, and other factors.

[0404] "Means of providing feedback" refer to techniques and devices for communicating advice and information based on sentiment analysis to workers.

[0405] "Means for accumulating report content" refers to a system that records data and results after work is completed, and uses them for later analysis and improvement.

[0406] "Methods for creating improvement measures" refers to the process of using accumulated work data and sentiment analysis results to derive areas for improvement in work procedures and the work environment.

[0407] The system that realizes this invention mainly consists of a server, a terminal (in this case, smart glasses), and an operator.

[0408] The server receives information stored in data storage devices, analyzes the information, and derives relevant data. Specifically, it functions as the core for automatically generating efficient instructions based on past work data and environmental information. The server also plays a role in providing feedback to workers, such as emotion-based advice and relaxation techniques, based on data provided by the emotion analysis engine. For this purpose, a cloud-based data processing platform and an analysis programming environment such as Python or R are likely to be used.

[0409] The smart glasses used as terminals are equipped with a camera and microphone, capturing the worker's facial expressions and voice in real time. Based on this, an emotion analysis engine analyzes the worker's emotional state. Based on the analysis results, instructions and feedback created by the server are displayed on the glasses' screen. For example, if a worker is determined to be irritated during work, they may receive instructions such as, "Take a deep breath and try meditating for one minute." An example of a prompt message is, "Please determine your stress level based on the current voice and facial expression data."

[0410] By receiving this feedback, users (workers) can manage their own work pace and status in real time. Furthermore, the data recorded by the emotion engine is accumulated over the long term and used for analysis to improve the work environment and enhance individual performance. This enables flexible and efficient work support tailored to individual needs.

[0411] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0412] Step 1:

[0413] The server retrieves work-related information and past work data from the database as input data. It analyzes this data and automatically generates instructions tailored to the work content. The generated instructions are then used as output data to be sent to the terminal. This process includes executing database queries and organizing and analyzing the obtained data.

[0414] Step 2:

[0415] The terminal uses its built-in camera and microphone to collect the worker's facial expressions and voice as input. Based on this input data, an emotion analysis engine calculates the worker's emotional state and outputs the analysis results to the server. Specifically, it performs facial expression analysis using image recognition technology and voice recognition.

[0416] Step 3:

[0417] The server uses emotion analysis results received from terminals and worker productivity data as input to evaluate stress levels and concentration levels. Based on this, it generates appropriate feedback and break instructions for the workers. Output data includes relaxation techniques and concise instructions. This step utilizes a generative AI model to integrate and analyze the collected data.

[0418] Step 4:

[0419] The terminal displays feedback and instructions received from the server on the smart glasses' display, notifying the worker. This display content is then used as output to prompt the user to take appropriate action. Specifically, this involves popping up messages or playing audio guidance.

[0420] Step 5:

[0421] The user (worker) checks notifications on their device and adjusts their work pace or takes breaks according to the feedback. This step requires the worker to take actions that maintain their own productivity and safety.

[0422] Step 6:

[0423] The server receives the report as input after the work is completed and derives improvement measures for the next task. This generates output data containing proposed improvements for future work instructions. The data is stored and analyzed over the long term to optimize the work environment.

[0424] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0425] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0426] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0427] [Third Embodiment]

[0428] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0429] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0430] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0431] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0432] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0433] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0434] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0435] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0436] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0437] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0438] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0439] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0440] The present invention provides an automation solution for streamlining operations in network centers and telecommunications exchanges. This system primarily consists of coordinated operation between servers, terminals, and users. Specific embodiments of the system are described below.

[0441] First, the server accesses a vast database of network and communication equipment, collecting detailed information on each piece of equipment, past work history, relevant laws and regulations, and guidelines. Next, the server analyzes the collected data and extracts relevant information necessary for on-site work. In this process, the analysis engine utilizes text mining technology to mechanically identify important procedures and points of caution from the data.

[0442] Based on the analyzed information, the server automatically generates instruction manuals and work instructions. The generated instructions are structured to be easy for workers to understand and include detailed information on efficient work procedures, necessary tool lists, and safety checks.

[0443] Next, the terminal presents the generated instructions to the field worker. The worker uses the terminal to review the instructions and check the checklist to ensure they are prepared. At this stage, the terminal records the worker's preparation status and provides an interface to check for any deficiencies in preparation.

[0444] Once work begins, the user (worker) can interact with the AI ​​agent in real time through a terminal. The terminal receives input and voice commands from the worker, and based on this, the AI ​​agent provides information and advice regarding the work. During this time, the server continuously monitors the progress of the work and immediately proposes countermeasures if any abnormalities occur. To ensure the safety of the worker and to ensure efficient work execution, the server also provides voice-based safety confirmation support as needed.

[0445] After completing a task, the user submits a completion report to their terminal. The completed work, any problems, and areas for improvement are entered, and the server evaluates the work based on this information. The evaluation results are stored in a database and used to improve efficiency and efficiency in future work. Based on the report, the server analyzes the performance of each task and generates improvement suggestions to incorporate the feedback into future work instructions.

[0446] Thus, the present invention provides consistent support from the preparation stage to the completion of a task, thereby improving work efficiency and reducing errors.

[0447] The following describes the processing flow.

[0448] Step 1:

[0449] The server accesses databases at network centers and communication exchanges to collect equipment information, work history, legal regulations, and security guidelines. By obtaining this information, the server gathers the basic data necessary for its operations.

[0450] Step 2:

[0451] The server analyzes the collected data and extracts information relevant to the task. Text mining techniques are used to identify important procedures and points to note, and the information is organized into categories.

[0452] Step 3:

[0453] The server automatically generates instructions based on the analysis results, including efficient work procedures, necessary tools, and safety checks. These instructions are systematically organized and presented in a format that is easy for workers to understand.

[0454] Step 4:

[0455] The terminal displays the generated instructions to the worker and prompts them to confirm the contents. The worker reviews the instructions on the terminal and completes a checklist on the terminal to confirm that they are ready. The terminal records the results of the preparation status check.

[0456] Step 5:

[0457] The user begins work on-site, and the terminal supports this process. The terminal receives input and voice commands from the worker and provides necessary information in real time through an AI agent.

[0458] Step 6:

[0459] The server monitors the progress of the work, provides real-time information, and detects and addresses anomalies. It can also provide workers with voice safety confirmations and warnings as needed.

[0460] Step 7:

[0461] After completing a task, the user submits a completion report using a terminal. The user inputs details of the work, any problems encountered, and areas for improvement into the terminal, which the server records and evaluates. This evaluation serves as the basis for future work instructions.

[0462] (Example 1)

[0463] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0464] Managing modern communications and network equipment requires processing large amounts of data and complex procedures. This necessitates efficient work procedures, worker safety, and real-time situational response. However, traditional methods are time-consuming for information gathering and analysis, lacking accuracy and speed. These issues raise concerns about decreased work efficiency and increased errors.

[0465] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0466] In this invention, the server includes means for an information processing device to access information sources via a network and collect information about equipment; means for analyzing the collected information and identifying work-related information using text processing technology; and means for creating instructions in electronic format based on the identified information and generating structured instructions. This enables each worker to perform tasks quickly and accurately, resulting in an efficient work process and improved worker safety.

[0467] An "information processing device" is a combination of hardware and software used to acquire, collect, and analyze information via a network.

[0468] "Information source" refers to external or internal databases or information systems that provide data about networks and communication equipment.

[0469] "Text processing technology" refers to techniques used to analyze relevant information from large amounts of text data and extract useful data.

[0470] "Work-related information" refers to information regarding instructions, procedures, and precautions that are essential for performing a particular task.

[0471] "Electronic instructions" refers to work instructions or guides that are displayed or delivered on a digital device.

[0472] "Structured instructions" are information or guidelines that are organized and systematically structured so that workers can easily understand and follow them.

[0473] "Worker" refers to a technician or operator who inspects, repairs, or maintains equipment on-site.

[0474] A "display device" refers to an electronic device that visually presents work instructions and other related information.

[0475] A "checklist" is a table or list that lists items used to prepare for work or to confirm procedures.

[0476] "Knowledge provision" refers to activities that provide necessary information and advice to workers in response to their requests.

[0477] This invention aims to improve work efficiency and safety using an information processing system. Specific embodiments are described below.

[0478] The server accesses the database system via the network to retrieve the necessary information. For example, it collects information about network equipment, past work history, and relevant laws and regulations. This is done using an open-source database management system, which queries the information to retrieve it. The collected data is then analyzed using a text analysis engine running on the server. During this process, natural language processing libraries are utilized to extract important procedures and precautions.

[0479] The server automatically generates work instructions in electronic format based on the extracted information. The generated instructions are structured in a format such as PDF and distributed from the server to the terminal. The work instructions include work procedures, a list of necessary tools, and safety checks to make them easy for workers to understand.

[0480] The terminal, acting as a client device, displays instructions received from the server to the on-site worker. This terminal may be a mobile information terminal equipped with a touchscreen. The worker reviews the displayed instructions and confirms the necessary preparations for the work. The preparation status is confirmed using a checklist displayed on the terminal, ensuring there are no deficiencies.

[0481] During the work process, the user (worker) can interact with the AI ​​agent through a terminal. For example, by typing "What should I do next?", the worker can receive answers and advice from the AI ​​agent. In this process, the server continuously monitors the progress of the work and immediately notifies the user if any anomalies are detected. The server can also provide voice guidance to ensure the worker's safety.

[0482] After completing a task, the user submits a completion report via their terminal. This report includes a summary of the work, any problems encountered, and suggestions for improvement for future tasks. The server evaluates the work based on this report and saves the results to a database. The accumulated data is used as feedback for future work instructions to further improve work efficiency.

[0483] A concrete example is the inspection of communication equipment. The server automatically creates inspection instructions based on past history and regulatory information, and a terminal displays them to the field worker. During the work, the worker can receive instructions from the AI ​​agent by typing "Please tell me the next step" into the terminal, allowing them to proceed with the work efficiently. In this way, the present invention utilizes information technology to support and optimize all stages of the work.

[0484] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0485] Step 1:

[0486] The server accesses information sources via the network to retrieve the necessary data. This input consists of detailed information about the equipment, including past work history and relevant regulatory information. The server queries this information using a database management system and outputs it as text data.

[0487] Step 2:

[0488] The server analyzes the acquired data. The input is the text data acquired in Step 1. The server analyzes this data using natural language processing technology and extracts necessary work procedures and precautions. As part of the data processing, the text analysis engine identifies important indicators and outputs them as structured information.

[0489] Step 3:

[0490] The server generates an electronic instruction sheet based on the analysis results. The input is the structured information obtained in step 2. The instruction sheet is formatted using LaTeX or similar and output in PDF format. The instruction sheet includes work procedures, necessary tools, and precautions, preparing them for presentation in the next stage.

[0491] Step 4:

[0492] The terminal receives instructions in PDF format sent from the server. The input is the electronic instructions generated in step 3. The instructions are displayed on the terminal using display software, making them easily accessible to the worker. This allows the worker to review the work content and prepare accordingly.

[0493] Step 5:

[0494] The user checks the instructions displayed on the terminal. The input here is the instructions displayed on the terminal in step 4. Based on the instructions, the user checks the preparation status via a checklist and checks the necessary boxes. The output is the confirmation result of whether preparation is complete.

[0495] Step 6:

[0496] During the task, the user interacts with the AI ​​agent in real time via a terminal. Input consists of voice commands and text input from the worker. The terminal transmits this to a server, where the generated AI model outputs optimal advice based on the input. The output is guidance information from the AI ​​agent.

[0497] Step 7:

[0498] The server monitors the progress of the work and detects anomalies. Input is data sent from the terminal during the work. Based on this, the server performs real-time data analysis and immediately outputs a notification in case of anomalies. Voice guidance may be output as safety confirmation support.

[0499] Step 8:

[0500] After completing a task, the user submits a completion report to the server from their terminal. The input is report data including the work performed, problems encountered, and areas for improvement. The server receives this data and stores it in a database. The evaluation results are analyzed and used as feedback when creating the next work order to improve work efficiency. The output is data for creating the next order, including improvement suggestions.

[0501] (Application Example 1)

[0502] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0503] In modern data management facilities, it is extremely difficult for workers to efficiently and safely perform tasks that require diverse procedures and split-second decisions. In such environments, rapid information provision and appropriate instructions are crucial for supporting work, but there is a lack of tools and systems to achieve this. Furthermore, optimization of visual and audible information presentation methods at the worksite is also required. Therefore, a system is needed that simultaneously improves work efficiency and strengthens safety measures.

[0504] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0505] In this invention, the server includes means for acquiring information from an information storage facility, means for analyzing the acquired information and extracting relevant information, and means for automatically generating instructions based on the extracted information. This enables workers to receive appropriate visual and audible instructions in real time, allowing them to perform their work quickly and safely.

[0506] An "information storage facility" refers to a storage device or database used for acquiring and managing data.

[0507] "Analysis" refers to the process of performing calculations to identify and extract relevant data from acquired information.

[0508] "Relevant information" refers to data and knowledge that are useful under specific conditions, as discovered through analysis.

[0509] An "instruction sheet" is a document that contains details and procedures for the tasks that workers are to perform.

[0510] "Workers" refers to individuals or teams who perform tasks based on instructions in a specific situation.

[0511] An "interface" refers to a device or software that provides a means of input and output for a user to interact with a system.

[0512] "Visual presentation" refers to the process of helping people understand information by displaying it in a visual form.

[0513] "Audio guidelines" refer to methods of providing guidance and instructions to workers through audio.

[0514] This invention provides a system for improving operational efficiency in data centers. It supports the entire process from start to finish through coordinated operation between servers, terminals, and users.

[0515] The server retrieves necessary data from the information storage. This information includes equipment details and past work history. The server analyzes this data and extracts relevant information. Text mining techniques are used for this analysis. Based on the relevant information, the server automatically generates instructions for workers and sends them to their terminals. The instructions include work procedures, safety checks, and necessary tools.

[0516] The terminal visually displays instructions to workers and provides an interface to check if the workers are ready. Furthermore, it accepts voice input and can provide voice guidelines and supplementary explanations from the server based on that input. Workers, as users, can check information in real time through the terminal and perform their tasks according to the instructions.

[0517] During the work process, the server monitors the work status through an AI agent and provides supplementary advice as needed. If an anomaly is detected, it can immediately propose countermeasures. In doing so, it provides information to the worker visually and audibly using ergonomic interfaces such as smart glasses.

[0518] Upon completion of a task, the user submits a report to their terminal, which is then sent to the server. The server evaluates the report and generates suggestions for improvement for future tasks. This process aims to improve the quality and efficiency of the work.

[0519] As a concrete example, in a standard server rack maintenance procedure within a data center, a worker wearing smart glasses can issue a voice command such as "Show me the details of the procedure," and the detailed procedure and related data will be displayed in real time. Furthermore, a generative AI model could utilize the prompt phrase, "Please tell me the standard server rack maintenance procedure in a data center."

[0520] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0521] Step 1:

[0522] The server retrieves data from the information storage, including detailed equipment information and past work history. This data is used as input for analysis on the server. At this stage, information is retrieved through APIs and database queries.

[0523] Step 2:

[0524] The server analyzes the acquired data and extracts relevant information. Here, text mining techniques are used to identify important procedures and points of caution within the information. In this analysis step, natural language processing techniques are used to compute the data, and the extracted information based on the results is used as input for the next step.

[0525] Step 3:

[0526] The server automatically generates work instructions based on the extracted relevant information. These instructions are organized documents containing work procedures, safety checks, and a list of necessary tools. The generated instructions are provided as output to the terminal and serve as input data for the next process. Here, a template-based document generation function is used.

[0527] Step 4:

[0528] The terminal visually displays the work instructions received from the server to the worker. An interface is also provided to prompt the worker (user) to make the necessary preparations, and the worker proceeds with preparations based on this information. Here, the instructions are displayed via a screen.

[0529] Step 5:

[0530] During the process, the user gives instructions to the AI ​​agent via voice input through the terminal. The terminal sends this voice information to a server, which analyzes the input. The generating AI model calculates appropriate advice and returns the result to the terminal as output. This step includes processing by the speech recognition system.

[0531] Step 6:

[0532] After the user completes their task, the terminal sends the work details and report to the server. The server uses this input information to evaluate the performance of the task and calculates suggestions for improvement for the next task. The final output is an updated instruction sheet to be used for the next task.

[0533] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0534] This invention aims to improve work efficiency and create a safe and secure work environment by combining an emotion engine with work support systems in network centers and communication exchanges, thereby recognizing, analyzing, and responding to the emotional state of workers in real time. This system consists of coordinated operation between servers, terminals, and users, and is implemented as follows.

[0535] First, the server, as before, collects and analyzes relevant information such as equipment information and work history from various databases. Then, an automatically generated instruction sheet based on this information is presented to the terminal of the field worker.

[0536] In this system, the terminal is equipped with an emotion engine that analyzes the worker's facial expressions and voice in real time through the camera and microphone. For example, if a user is feeling fatigued or stressed while working, the emotion engine detects this and sends the analysis results to the server.

[0537] The server receives data from the emotion engine and provides appropriate feedback to the worker. This feedback includes relaxation techniques to reduce stress, advice on how to proceed with work, and suggestions for appropriate breaks. For example, if the server detects that the worker's concentration is waning, it can notify them to take a break and provide voice guidance on stretching during that time.

[0538] Furthermore, users (workers) can check the emotional engine's feedback through their devices and manage their own work pace and state. The data recorded by the emotional engine is analyzed over the long term and used to improve the work environment and enhance performance. This makes it possible to provide more user-friendly work instructions that take the emotional aspects of workers into consideration.

[0539] By combining this with an emotional engine, it becomes possible to add a higher level of safety and personalized support to conventional work efficiency systems. This system reduces the burden on workers and provides a safe and efficient work environment.

[0540] The following describes the processing flow.

[0541] Step 1:

[0542] The server accesses databases related to the daily operations of network centers and communication exchanges, collecting necessary equipment information, past work history, and information on legal regulations. This allows for the acquisition of basic information necessary for preparing for work.

[0543] Step 2:

[0544] The server analyzes the collected data to extract specific procedures and precautions necessary for the work. Text mining technology is used in this process, and work instructions are automatically generated based on the analysis results. These instructions include efficient work procedures and safety checks.

[0545] Step 3:

[0546] The terminal displays instructions generated on the server to the worker. The worker reviews the instructions and uses a checklist to verify that all necessary preparations are complete. The terminal records this preparation status and the results of the verification.

[0547] Step 4:

[0548] The emotion engine built into the terminal uses a camera and microphone to monitor the worker's facial expressions and voice, recognizing their current emotional state. For example, if a worker is showing signs of anxiety or stress, it can be detected immediately.

[0549] Step 5:

[0550] Based on data from the emotion engine, the server creates feedback based on the worker's emotional state. For example, if the server detects that the worker is stressed, it will suggest relaxation techniques or advise them to take a short break.

[0551] Step 6:

[0552] The user (worker) receives emotional feedback through the terminal and continues working or takes a break according to the advice provided by the system. This promotes self-management of pace and improves safety.

[0553] Step 7:

[0554] After completing a task, the user submits a completion report using their terminal. The report details the progress of the task, any problems encountered, and the user's emotional state, which the server then records in a database. This recorded data is used to improve future tasks, leading to continuous efficiency improvements.

[0555] (Example 2)

[0556] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0557] To improve work efficiency and safety on-site, it is necessary to understand workers' emotional states in real time and provide appropriate feedback based on that understanding. However, conventional systems have not been sufficient in understanding and responding to such emotional states. Proceeding with work without considering emotional states can lead to decreased work efficiency and safety problems. Therefore, work support that takes emotional states into account is necessary.

[0558] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0559] In this invention, the server includes means for acquiring data from a data storage device, means for analyzing the acquired data and extracting relevant data, and means for detecting the emotional state of an operator and transmitting the analyzed data to a higher-level device. This makes it possible to provide situation-appropriate feedback in real time based on the emotional state of the operator, thereby improving work efficiency and safety.

[0560] "Data storage device" is a general term for storage media and devices used to store various types of data, and includes databases and server storage.

[0561] "Means of retrieving data" refers to the processes and technologies used to access and retrieve stored data, and includes database operations using query languages.

[0562] "Means of analyzing data and extracting relevant data" refers to methods of computationally processing acquired data and extracting useful information, and includes the use of algorithms and statistical models.

[0563] "Means for automatically generating instruction information" refers to a system that automatically creates necessary procedures and points to note using a computer based on collected data.

[0564] "Means of displaying instructional information to workers and performing necessary preparation checks" refers to the process of presenting information on a screen or other device to help workers follow instructions and confirming their readiness.

[0565] "A means of receiving information from workers and providing that information in real time" refers to a system that processes the information entered by workers immediately and provides the results right away.

[0566] "Means for detecting emotional states and transmitting the analysis data to a higher-level device" refers to means for measuring and analyzing the emotions of workers and transmitting the results to a central system.

[0567] "Means for generating and presenting feedback" refers to the process of automatically creating useful advice and information based on analysis results and communicating it to the worker.

[0568] "Means of recording report contents and making improvements for the next work" refers to an approach that saves work history and uses it to improve the efficiency and safety of the next work.

[0569] The present invention is a system for improving work efficiency and safety on site, and its embodiments are as follows.

[0570] The server first retrieves the necessary data from the data storage device. This data includes equipment records and past work history, and is retrieved using SQL queries. The server then analyzes the data, extracts relevant information, and automatically generates instruction information. Decision tree models and random forests are commonly used as machine learning algorithms. The generated instruction information is then sent to the worker's terminal.

[0571] The terminal is equipped with an emotion engine to analyze the emotional state of the worker. This engine uses libraries such as OpenCV and PyAudio to analyze the worker's facial expressions and voice in real time through the camera and microphone. The terminal has the function to send the analyzed emotional state data to a server, and the emotion analysis engine uses a convolutional neural network (CNN).

[0572] When the server receives emotional state data sent from the terminal, it creates appropriate feedback based on this data. Using a generative AI model, it generates advice according to the prompt. An example of a prompt is, "Please provide effective relaxation methods if the user is feeling fatigued." The feedback content, including suggestions for breaks and improvements to work methods, is sent to the worker's terminal.

[0573] The user, as the worker, can check the feedback provided through the terminal and optimally manage their work status. For example, by taking breaks according to the recommended relaxation methods, the worker can regain their concentration. Furthermore, the feedback content may also be guided to the worker by a voice assistant, making it highly convenient even in work situations where visual input is unavailable.

[0574] Thus, the system of the present invention can provide real-time feedback based on the worker's emotional state and work history, thereby supporting an efficient and safe work environment.

[0575] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0576] Step 1:

[0577] The server retrieves equipment information and work history from data storage devices. The input is a database, and the specific technology used is SQL queries. The output is an aggregation of work-related data. This prepares the data necessary for subsequent analysis and instruction generation.

[0578] Step 2:

[0579] The server analyzes the acquired data and extracts relevant data. Specifically, machine learning algorithms (e.g., decision trees and random forests) are used. The input is the data aggregated in step 1, and the output is automatically generated instruction information. This process helps to develop efficient work procedures.

[0580] Step 3:

[0581] The server converts the generated instruction information into JSON format and sends it to the terminal. The input is the instruction information, and the output is the data displayed on the terminal. At this stage, the worker can confirm the necessary preparations.

[0582] Step 4:

[0583] The terminal uses its built-in camera and microphone to collect the worker's facial expressions and voice in real time. The input is the worker's visual and audio data, and the output is data for analyzing their emotional state. Specific technologies used include the OpenCV and PyAudio libraries.

[0584] Step 5:

[0585] The terminal feeds the collected data into an emotion engine to analyze the emotional state. A convolutional neural network (CNN) model is used here. The input is the data obtained in step 4, and the output is the analyzed emotional state data. This analysis quantifies the worker's emotional state.

[0586] Step 6:

[0587] The terminal sends the results of the emotional state analysis to the server. The input is the analyzed emotional data, and the output is the data to be sent to the higher-level device. HTTPS is used as the specific communication protocol.

[0588] Step 7:

[0589] The server uses a generative AI model based on emotional data to create feedback. The input is emotional state data, and the output is feedback information provided to the worker. An example of a prompt is, "Provide effective relaxation methods for when the user is feeling fatigued."

[0590] Step 8:

[0591] The server sends the generated feedback to the terminal, which then presents it to the worker visually and audibly. The input is the feedback information, and the output is the guidance information received by the worker. Specifically, the feedback is communicated via screen displays and voice assistants.

[0592] Step 9:

[0593] The user (worker) manages their work status based on the feedback provided and adjusts their work methods and breaks as needed. The input is feedback information from the terminal, and the output is the managed work status. This step allows the worker to continue working efficiently and safely.

[0594] (Application Example 2)

[0595] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0596] In modern production environments, it is known that the emotional state of workers significantly impacts work efficiency and safety. However, in many workplaces, this factor is often overlooked, and worker fatigue and stress are sometimes ignored. This can lead to decreased work efficiency and compromised safety, highlighting the need for a system that monitors workers' emotional states in real time and provides appropriate feedback.

[0597] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0598] In this invention, the server includes means for receiving information from a data storage device, means for analyzing the received information to derive relevant information, and means for analyzing the emotional state of workers using an emotion analysis engine and providing appropriate feedback based on the analysis results. This makes it possible to grasp the emotional state of workers in real time and improve work efficiency and safety through appropriate feedback.

[0599] A "data storage device" is a device that stores various types of information over a long period of time and allows it to be searched and retrieved as needed.

[0600] "Means of receiving information" refers to the processes and technologies used to obtain necessary data through external data communication.

[0601] "Means of information analysis" refer to the technologies and algorithms used to analyze received data and derive useful information.

[0602] "Means for deriving relevant information" refers to methods for extracting meaningful information related to a specific purpose from analyzed data.

[0603] "Means for automatically generating instructions" refers to technology that automatically creates guidelines defining the tasks and actions that humans should perform based on the derived information.

[0604] "Means of presenting instructions to workers" refers to procedures or devices for explicitly communicating generated instructions to the person in charge of the work.

[0605] "Means of supplying information" refers to a system that provides necessary data and feedback immediately during work.

[0606] An "emotion analysis engine" is a technology that automatically detects and analyzes the emotional state of workers based on their facial expressions, voice characteristics, and other factors.

[0607] "Means of providing feedback" refer to techniques and devices for communicating advice and information based on sentiment analysis to workers.

[0608] "Means for accumulating report content" refers to a system that records data and results after work is completed, and uses them for later analysis and improvement.

[0609] "Methods for creating improvement measures" refers to the process of using accumulated work data and sentiment analysis results to derive areas for improvement in work procedures and the work environment.

[0610] The system that realizes this invention mainly consists of a server, a terminal (in this case, smart glasses), and an operator.

[0611] The server receives information stored in data storage devices, analyzes the information, and derives relevant data. Specifically, it functions as the core for automatically generating efficient instructions based on past work data and environmental information. The server also plays a role in providing feedback to workers, such as emotion-based advice and relaxation techniques, based on data provided by the emotion analysis engine. For this purpose, a cloud-based data processing platform and an analysis programming environment such as Python or R are likely to be used.

[0612] The smart glasses used as terminals are equipped with a camera and microphone, capturing the worker's facial expressions and voice in real time. Based on this, an emotion analysis engine analyzes the worker's emotional state. Based on the analysis results, instructions and feedback created by the server are displayed on the glasses' screen. For example, if a worker is determined to be irritated during work, they may receive instructions such as, "Take a deep breath and try meditating for one minute." An example of a prompt message is, "Please determine your stress level based on the current voice and facial expression data."

[0613] By receiving this feedback, users (workers) can manage their own work pace and status in real time. Furthermore, the data recorded by the emotion engine is accumulated over the long term and used for analysis to improve the work environment and enhance individual performance. This enables flexible and efficient work support tailored to individual needs.

[0614] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0615] Step 1:

[0616] The server retrieves work-related information and past work data from the database as input data. It analyzes this data and automatically generates instructions tailored to the work content. The generated instructions are then used as output data to be sent to the terminal. This process includes executing database queries and organizing and analyzing the obtained data.

[0617] Step 2:

[0618] The terminal uses its built-in camera and microphone to collect the worker's facial expressions and voice as input. Based on this input data, an emotion analysis engine calculates the worker's emotional state and outputs the analysis results to the server. Specifically, it performs facial expression analysis using image recognition technology and voice recognition.

[0619] Step 3:

[0620] The server uses emotion analysis results received from terminals and worker productivity data as input to evaluate stress levels and concentration levels. Based on this, it generates appropriate feedback and break instructions for the workers. Output data includes relaxation techniques and concise instructions. This step utilizes a generative AI model to integrate and analyze the collected data.

[0621] Step 4:

[0622] The terminal displays feedback and instructions received from the server on the smart glasses' display, notifying the worker. This display content is then used as output to prompt the user to take appropriate action. Specifically, this involves popping up messages or playing audio guidance.

[0623] Step 5:

[0624] The user (worker) checks notifications on their device and adjusts their work pace or takes breaks according to the feedback. This step requires the worker to take actions that maintain their own productivity and safety.

[0625] Step 6:

[0626] The server receives the report as input after the work is completed and derives improvement measures for the next task. This generates output data containing proposed improvements for future work instructions. The data is stored and analyzed over the long term to optimize the work environment.

[0627] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0628] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0629] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0630] [Fourth Embodiment]

[0631] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0632] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0633] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0634] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0635] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0636] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0637] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0638] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0639] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0640] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0641] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0642] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0643] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0644] The present invention provides an automation solution for streamlining operations in network centers and telecommunications exchanges. This system primarily consists of coordinated operation between servers, terminals, and users. Specific embodiments of the system are described below.

[0645] First, the server accesses a vast database of network and communication equipment, collecting detailed information on each piece of equipment, past work history, relevant laws and regulations, and guidelines. Next, the server analyzes the collected data and extracts relevant information necessary for on-site work. In this process, the analysis engine utilizes text mining technology to mechanically identify important procedures and points of caution from the data.

[0646] Based on the analyzed information, the server automatically generates instruction manuals and work instructions. The generated instructions are structured to be easy for workers to understand and include detailed information on efficient work procedures, necessary tool lists, and safety checks.

[0647] Next, the terminal presents the generated instructions to the field worker. The worker uses the terminal to review the instructions and check the checklist to ensure they are prepared. At this stage, the terminal records the worker's preparation status and provides an interface to check for any deficiencies in preparation.

[0648] Once work begins, the user (worker) can interact with the AI ​​agent in real time through a terminal. The terminal receives input and voice commands from the worker, and based on this, the AI ​​agent provides information and advice regarding the work. During this time, the server continuously monitors the progress of the work and immediately proposes countermeasures if any abnormalities occur. To ensure the safety of the worker and to ensure efficient work execution, the server also provides voice-based safety confirmation support as needed.

[0649] After completing a task, the user submits a completion report to their terminal. The completed work, any problems, and areas for improvement are entered, and the server evaluates the work based on this information. The evaluation results are stored in a database and used to improve efficiency and efficiency in future work. Based on the report, the server analyzes the performance of each task and generates improvement suggestions to incorporate the feedback into future work instructions.

[0650] Thus, the present invention provides consistent support from the preparation stage to the completion of a task, thereby improving work efficiency and reducing errors.

[0651] The following describes the processing flow.

[0652] Step 1:

[0653] The server accesses databases at network centers and communication exchanges to collect equipment information, work history, legal regulations, and security guidelines. By obtaining this information, the server gathers the basic data necessary for its operations.

[0654] Step 2:

[0655] The server analyzes the collected data and extracts information relevant to the task. Text mining techniques are used to identify important procedures and points to note, and the information is organized into categories.

[0656] Step 3:

[0657] The server automatically generates instructions based on the analysis results, including efficient work procedures, necessary tools, and safety checks. These instructions are systematically organized and presented in a format that is easy for workers to understand.

[0658] Step 4:

[0659] The terminal displays the generated instructions to the worker and prompts them to confirm the contents. The worker reviews the instructions on the terminal and completes a checklist on the terminal to confirm that they are ready. The terminal records the results of the preparation status check.

[0660] Step 5:

[0661] The user begins work on-site, and the terminal supports this process. The terminal receives input and voice commands from the worker and provides necessary information in real time through an AI agent.

[0662] Step 6:

[0663] The server monitors the progress of the work, provides real-time information, and detects and addresses anomalies. It can also provide workers with voice safety confirmations and warnings as needed.

[0664] Step 7:

[0665] After completing a task, the user submits a completion report using a terminal. The user inputs details of the work, any problems encountered, and areas for improvement into the terminal, which the server records and evaluates. This evaluation serves as the basis for future work instructions.

[0666] (Example 1)

[0667] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0668] Managing modern communications and network equipment requires processing large amounts of data and complex procedures. This necessitates efficient work procedures, worker safety, and real-time situational response. However, traditional methods are time-consuming for information gathering and analysis, lacking accuracy and speed. These issues raise concerns about decreased work efficiency and increased errors.

[0669] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0670] In this invention, the server includes means for an information processing device to access information sources via a network and collect information about equipment; means for analyzing the collected information and identifying work-related information using text processing technology; and means for creating instructions in electronic format based on the identified information and generating structured instructions. This enables each worker to perform tasks quickly and accurately, resulting in an efficient work process and improved worker safety.

[0671] An "information processing device" is a combination of hardware and software used to acquire, collect, and analyze information via a network.

[0672] "Information source" refers to external or internal databases or information systems that provide data about networks and communication equipment.

[0673] "Text processing technology" refers to techniques used to analyze relevant information from large amounts of text data and extract useful data.

[0674] "Work-related information" refers to information regarding instructions, procedures, and precautions that are essential for performing a particular task.

[0675] "Electronic instructions" refers to work instructions or guides that are displayed or delivered on a digital device.

[0676] "Structured instructions" are information or guidelines that are organized and systematically structured so that workers can easily understand and follow them.

[0677] "Worker" refers to a technician or operator who inspects, repairs, or maintains equipment on-site.

[0678] A "display device" refers to an electronic device that visually presents work instructions and other related information.

[0679] A "checklist" is a table or list that lists items used to prepare for work or to confirm procedures.

[0680] "Knowledge provision" refers to activities that provide necessary information and advice to workers in response to their requests.

[0681] This invention aims to improve work efficiency and safety using an information processing system. Specific embodiments are described below.

[0682] The server accesses the database system via the network to retrieve the necessary information. For example, it collects information about network equipment, past work history, and relevant laws and regulations. This is done using an open-source database management system, which queries the information to retrieve it. The collected data is then analyzed using a text analysis engine running on the server. During this process, natural language processing libraries are utilized to extract important procedures and precautions.

[0683] The server automatically generates work instructions in electronic format based on the extracted information. The generated instructions are structured in a format such as PDF and distributed from the server to the terminal. The work instructions include work procedures, a list of necessary tools, and safety checks to make them easy for workers to understand.

[0684] The terminal, acting as a client device, displays instructions received from the server to the on-site worker. This terminal may be a mobile information terminal equipped with a touchscreen. The worker reviews the displayed instructions and confirms the necessary preparations for the work. The preparation status is confirmed using a checklist displayed on the terminal, ensuring there are no deficiencies.

[0685] During the work process, the user (worker) can interact with the AI ​​agent through a terminal. For example, by typing "What should I do next?", the worker can receive answers and advice from the AI ​​agent. In this process, the server continuously monitors the progress of the work and immediately notifies the user if any anomalies are detected. The server can also provide voice guidance to ensure the worker's safety.

[0686] After completing a task, the user submits a completion report via their terminal. This report includes a summary of the work, any problems encountered, and suggestions for improvement for future tasks. The server evaluates the work based on this report and saves the results to a database. The accumulated data is used as feedback for future work instructions to further improve work efficiency.

[0687] A concrete example is the inspection of communication equipment. The server automatically creates inspection instructions based on past history and regulatory information, and a terminal displays them to the field worker. During the work, the worker can receive instructions from the AI ​​agent by typing "Please tell me the next step" into the terminal, allowing them to proceed with the work efficiently. In this way, the present invention utilizes information technology to support and optimize all stages of the work.

[0688] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0689] Step 1:

[0690] The server accesses information sources via the network to retrieve the necessary data. This input consists of detailed information about the equipment, including past work history and relevant regulatory information. The server queries this information using a database management system and outputs it as text data.

[0691] Step 2:

[0692] The server analyzes the acquired data. The input is the text data acquired in Step 1. The server analyzes this data using natural language processing technology and extracts necessary work procedures and precautions. As part of the data processing, the text analysis engine identifies important indicators and outputs them as structured information.

[0693] Step 3:

[0694] The server generates an electronic instruction sheet based on the analysis results. The input is the structured information obtained in step 2. The instruction sheet is formatted using LaTeX or similar and output in PDF format. The instruction sheet includes work procedures, necessary tools, and precautions, preparing them for presentation in the next stage.

[0695] Step 4:

[0696] The terminal receives instructions in PDF format sent from the server. The input is the electronic instructions generated in step 3. The instructions are displayed on the terminal using display software, making them easily accessible to the worker. This allows the worker to review the work content and prepare accordingly.

[0697] Step 5:

[0698] The user checks the instructions displayed on the terminal. The input here is the instructions displayed on the terminal in step 4. Based on the instructions, the user checks the preparation status via a checklist and checks the necessary boxes. The output is the confirmation result of whether preparation is complete.

[0699] Step 6:

[0700] During the task, the user interacts with the AI ​​agent in real time via a terminal. Input consists of voice commands and text input from the worker. The terminal transmits this to a server, where the generated AI model outputs optimal advice based on the input. The output is guidance information from the AI ​​agent.

[0701] Step 7:

[0702] The server monitors the progress of the work and detects anomalies. Input is data sent from the terminal during the work. Based on this, the server performs real-time data analysis and immediately outputs a notification in case of anomalies. Voice guidance may be output as safety confirmation support.

[0703] Step 8:

[0704] After completing a task, the user submits a completion report to the server from their terminal. The input is report data including the work performed, problems encountered, and areas for improvement. The server receives this data and stores it in a database. The evaluation results are analyzed and used as feedback when creating the next work order to improve work efficiency. The output is data for creating the next order, including improvement suggestions.

[0705] (Application Example 1)

[0706] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0707] In modern data management facilities, it is extremely difficult for workers to efficiently and safely perform tasks that require diverse procedures and split-second decisions. In such environments, rapid information provision and appropriate instructions are crucial for supporting work, but there is a lack of tools and systems to achieve this. Furthermore, optimization of visual and audible information presentation methods at the worksite is also required. Therefore, a system is needed that simultaneously improves work efficiency and strengthens safety measures.

[0708] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0709] In this invention, the server includes means for acquiring information from an information storage facility, means for analyzing the acquired information and extracting relevant information, and means for automatically generating instructions based on the extracted information. This enables workers to receive appropriate visual and audible instructions in real time, allowing them to perform their work quickly and safely.

[0710] An "information storage facility" refers to a storage device or database used for acquiring and managing data.

[0711] "Analysis" refers to the process of performing calculations to identify and extract relevant data from acquired information.

[0712] "Relevant information" refers to data and knowledge that are useful under specific conditions, as discovered through analysis.

[0713] An "instruction sheet" is a document that contains details and procedures for the tasks that workers are to perform.

[0714] "Workers" refers to individuals or teams who perform tasks based on instructions in a specific situation.

[0715] An "interface" refers to a device or software that provides a means of input and output for a user to interact with a system.

[0716] "Visual presentation" refers to the process of helping people understand information by displaying it in a visual form.

[0717] "Audio guidelines" refer to methods of providing guidance and instructions to workers through audio.

[0718] This invention provides a system for improving operational efficiency in data centers. It supports the entire process from start to finish through coordinated operation between servers, terminals, and users.

[0719] The server retrieves necessary data from the information storage. This information includes equipment details and past work history. The server analyzes this data and extracts relevant information. Text mining techniques are used for this analysis. Based on the relevant information, the server automatically generates instructions for workers and sends them to their terminals. The instructions include work procedures, safety checks, and necessary tools.

[0720] The terminal visually displays instructions to workers and provides an interface to check if the workers are ready. Furthermore, it accepts voice input and can provide voice guidelines and supplementary explanations from the server based on that input. Workers, as users, can check information in real time through the terminal and perform their tasks according to the instructions.

[0721] During the work process, the server monitors the work status through an AI agent and provides supplementary advice as needed. If an anomaly is detected, it can immediately propose countermeasures. In doing so, it provides information to the worker visually and audibly using ergonomic interfaces such as smart glasses.

[0722] Upon completion of a task, the user submits a report to their terminal, which is then sent to the server. The server evaluates the report and generates suggestions for improvement for future tasks. This process aims to improve the quality and efficiency of the work.

[0723] As a concrete example, in a standard server rack maintenance procedure within a data center, a worker wearing smart glasses can issue a voice command such as "Show me the details of the procedure," and the detailed procedure and related data will be displayed in real time. Furthermore, a generative AI model could utilize the prompt phrase, "Please tell me the standard server rack maintenance procedure in a data center."

[0724] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0725] Step 1:

[0726] The server retrieves data from the information storage, including detailed equipment information and past work history. This data is used as input for analysis on the server. At this stage, information is retrieved through APIs and database queries.

[0727] Step 2:

[0728] The server analyzes the acquired data and extracts relevant information. Here, text mining techniques are used to identify important procedures and points of caution within the information. In this analysis step, natural language processing techniques are used to compute the data, and the extracted information based on the results is used as input for the next step.

[0729] Step 3:

[0730] The server automatically generates work instructions based on the extracted relevant information. These instructions are organized documents containing work procedures, safety checks, and a list of necessary tools. The generated instructions are provided as output to the terminal and serve as input data for the next process. Here, a template-based document generation function is used.

[0731] Step 4:

[0732] The terminal visually displays the work instructions received from the server to the worker. An interface is also provided to prompt the worker (user) to make the necessary preparations, and the worker proceeds with preparations based on this information. Here, the instructions are displayed via a screen.

[0733] Step 5:

[0734] During the process, the user gives instructions to the AI ​​agent via voice input through the terminal. The terminal sends this voice information to a server, which analyzes the input. The generating AI model calculates appropriate advice and returns the result to the terminal as output. This step includes processing by the speech recognition system.

[0735] Step 6:

[0736] After the user completes their task, the terminal sends the work details and report to the server. The server uses this input information to evaluate the performance of the task and calculates suggestions for improvement for the next task. The final output is an updated instruction sheet to be used for the next task.

[0737] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0738] This invention aims to improve work efficiency and create a safe and secure work environment by combining an emotion engine with work support systems in network centers and communication exchanges, thereby recognizing, analyzing, and responding to the emotional state of workers in real time. This system consists of coordinated operation between servers, terminals, and users, and is implemented as follows.

[0739] First, the server, as before, collects and analyzes relevant information such as equipment information and work history from various databases. Then, an automatically generated instruction sheet based on this information is presented to the terminal of the field worker.

[0740] In this system, the terminal is equipped with an emotion engine that analyzes the worker's facial expressions and voice in real time through the camera and microphone. For example, if a user is feeling fatigued or stressed while working, the emotion engine detects this and sends the analysis results to the server.

[0741] The server receives data from the emotion engine and provides appropriate feedback to the worker. This feedback includes relaxation techniques to reduce stress, advice on how to proceed with work, and suggestions for appropriate breaks. For example, if the server detects that the worker's concentration is waning, it can notify them to take a break and provide voice guidance on stretching during that time.

[0742] Furthermore, users (workers) can check the emotional engine's feedback through their devices and manage their own work pace and state. The data recorded by the emotional engine is analyzed over the long term and used to improve the work environment and enhance performance. This makes it possible to provide more user-friendly work instructions that take the emotional aspects of workers into consideration.

[0743] By combining this with an emotional engine, it becomes possible to add a higher level of safety and personalized support to conventional work efficiency systems. This system reduces the burden on workers and provides a safe and efficient work environment.

[0744] The following describes the processing flow.

[0745] Step 1:

[0746] The server accesses databases related to the daily operations of network centers and communication exchanges, collecting necessary equipment information, past work history, and information on legal regulations. This allows for the acquisition of basic information necessary for preparing for work.

[0747] Step 2:

[0748] The server analyzes the collected data to extract specific procedures and precautions necessary for the work. Text mining technology is used in this process, and work instructions are automatically generated based on the analysis results. These instructions include efficient work procedures and safety checks.

[0749] Step 3:

[0750] The terminal displays instructions generated on the server to the worker. The worker reviews the instructions and uses a checklist to verify that all necessary preparations are complete. The terminal records this preparation status and the results of the verification.

[0751] Step 4:

[0752] The emotion engine built into the terminal uses a camera and microphone to monitor the worker's facial expressions and voice, recognizing their current emotional state. For example, if a worker is showing signs of anxiety or stress, it can be detected immediately.

[0753] Step 5:

[0754] Based on data from the emotion engine, the server creates feedback based on the worker's emotional state. For example, if the server detects that the worker is stressed, it will suggest relaxation techniques or advise them to take a short break.

[0755] Step 6:

[0756] The user (worker) receives emotional feedback through the terminal and continues working or takes a break according to the advice provided by the system. This promotes self-management of pace and improves safety.

[0757] Step 7:

[0758] After completing a task, the user submits a completion report using their terminal. The report details the progress of the task, any problems encountered, and the user's emotional state, which the server then records in a database. This recorded data is used to improve future tasks, leading to continuous efficiency improvements.

[0759] (Example 2)

[0760] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0761] To improve work efficiency and safety on-site, it is necessary to understand workers' emotional states in real time and provide appropriate feedback based on that understanding. However, conventional systems have not been sufficient in understanding and responding to such emotional states. Proceeding with work without considering emotional states can lead to decreased work efficiency and safety problems. Therefore, work support that takes emotional states into account is necessary.

[0762] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0763] In this invention, the server includes means for acquiring data from a data storage device, means for analyzing the acquired data and extracting relevant data, and means for detecting the emotional state of an operator and transmitting the analyzed data to a higher-level device. This makes it possible to provide situation-appropriate feedback in real time based on the emotional state of the operator, thereby improving work efficiency and safety.

[0764] "Data storage device" is a general term for storage media and devices used to store various types of data, and includes databases and server storage.

[0765] "Means of retrieving data" refers to the processes and technologies used to access and retrieve stored data, and includes database operations using query languages.

[0766] "Means of analyzing data and extracting relevant data" refers to methods of computationally processing acquired data and extracting useful information, and includes the use of algorithms and statistical models.

[0767] "Means for automatically generating instruction information" refers to a system that automatically creates necessary procedures and points to note using a computer based on collected data.

[0768] "Means of displaying instructional information to workers and performing necessary preparation checks" refers to the process of presenting information on a screen or other device to help workers follow instructions and confirming their readiness.

[0769] "A means of receiving information from workers and providing that information in real time" refers to a system that processes the information entered by workers immediately and provides the results right away.

[0770] "Means for detecting emotional states and transmitting the analysis data to a higher-level device" refers to means for measuring and analyzing the emotions of workers and transmitting the results to a central system.

[0771] "Means for generating and presenting feedback" refers to the process of automatically creating useful advice and information based on analysis results and communicating it to the worker.

[0772] "Means of recording report contents and making improvements for the next work" refers to an approach that saves work history and uses it to improve the efficiency and safety of the next work.

[0773] The present invention is a system for improving work efficiency and safety on site, and its embodiments are as follows.

[0774] The server first retrieves the necessary data from the data storage device. This data includes equipment records and past work history, and is retrieved using SQL queries. The server then analyzes the data, extracts relevant information, and automatically generates instruction information. Decision tree models and random forests are commonly used as machine learning algorithms. The generated instruction information is then sent to the worker's terminal.

[0775] The terminal is equipped with an emotion engine to analyze the emotional state of the worker. This engine uses libraries such as OpenCV and PyAudio to analyze the worker's facial expressions and voice in real time through the camera and microphone. The terminal has the function to send the analyzed emotional state data to a server, and the emotion analysis engine uses a convolutional neural network (CNN).

[0776] When the server receives emotional state data sent from the terminal, it creates appropriate feedback based on this data. Using a generative AI model, it generates advice according to the prompt. An example of a prompt is, "Please provide effective relaxation methods if the user is feeling fatigued." The feedback content, including suggestions for breaks and improvements to work methods, is sent to the worker's terminal.

[0777] The user, as the worker, can check the feedback provided through the terminal and optimally manage their work status. For example, by taking breaks according to the recommended relaxation methods, the worker can regain their concentration. Furthermore, the feedback content may also be guided to the worker by a voice assistant, making it highly convenient even in work situations where visual input is unavailable.

[0778] Thus, the system of the present invention can provide real-time feedback based on the worker's emotional state and work history, thereby supporting an efficient and safe work environment.

[0779] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0780] Step 1:

[0781] The server retrieves equipment information and work history from data storage devices. The input is a database, and the specific technology used is SQL queries. The output is an aggregation of work-related data. This prepares the data necessary for subsequent analysis and instruction generation.

[0782] Step 2:

[0783] The server analyzes the acquired data and extracts relevant data. Specifically, machine learning algorithms (e.g., decision trees and random forests) are used. The input is the data aggregated in step 1, and the output is automatically generated instruction information. This process helps to develop efficient work procedures.

[0784] Step 3:

[0785] The server converts the generated instruction information into JSON format and sends it to the terminal. The input is the instruction information, and the output is the data displayed on the terminal. At this stage, the worker can confirm the necessary preparations.

[0786] Step 4:

[0787] The terminal uses its built-in camera and microphone to collect the worker's facial expressions and voice in real time. The input is the worker's visual and audio data, and the output is data for analyzing their emotional state. Specific technologies used include the OpenCV and PyAudio libraries.

[0788] Step 5:

[0789] The terminal feeds the collected data into an emotion engine to analyze the emotional state. A convolutional neural network (CNN) model is used here. The input is the data obtained in step 4, and the output is the analyzed emotional state data. This analysis quantifies the worker's emotional state.

[0790] Step 6:

[0791] The terminal sends the results of the emotional state analysis to the server. The input is the analyzed emotional data, and the output is the data to be sent to the higher-level device. HTTPS is used as the specific communication protocol.

[0792] Step 7:

[0793] The server uses a generative AI model based on emotional data to create feedback. The input is emotional state data, and the output is feedback information provided to the worker. An example of a prompt is, "Provide effective relaxation methods for when the user is feeling fatigued."

[0794] Step 8:

[0795] The server sends the generated feedback to the terminal, which then presents it to the worker visually and audibly. The input is the feedback information, and the output is the guidance information received by the worker. Specifically, the feedback is communicated via screen displays and voice assistants.

[0796] Step 9:

[0797] The user (worker) manages their work status based on the feedback provided and adjusts their work methods and breaks as needed. The input is feedback information from the terminal, and the output is the managed work status. This step allows the worker to continue working efficiently and safely.

[0798] (Application Example 2)

[0799] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0800] In modern production environments, it is known that the emotional state of workers significantly impacts work efficiency and safety. However, in many workplaces, this factor is often overlooked, and worker fatigue and stress are sometimes ignored. This can lead to decreased work efficiency and compromised safety, highlighting the need for a system that monitors workers' emotional states in real time and provides appropriate feedback.

[0801] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0802] In this invention, the server includes means for receiving information from a data storage device, means for analyzing the received information to derive relevant information, and means for analyzing the emotional state of workers using an emotion analysis engine and providing appropriate feedback based on the analysis results. This makes it possible to grasp the emotional state of workers in real time and improve work efficiency and safety through appropriate feedback.

[0803] A "data storage device" is a device that stores various types of information over a long period of time and allows it to be searched and retrieved as needed.

[0804] "Means of receiving information" refers to the processes and technologies used to obtain necessary data through external data communication.

[0805] "Means of information analysis" refer to the technologies and algorithms used to analyze received data and derive useful information.

[0806] "Means for deriving relevant information" refers to methods for extracting meaningful information related to a specific purpose from analyzed data.

[0807] "Means for automatically generating instructions" refers to technology that automatically creates guidelines defining the tasks and actions that humans should perform based on the derived information.

[0808] "Means of presenting instructions to workers" refers to procedures or devices for explicitly communicating generated instructions to the person in charge of the work.

[0809] "Means of supplying information" refers to a system that provides necessary data and feedback immediately during work.

[0810] An "emotion analysis engine" is a technology that automatically detects and analyzes the emotional state of workers based on their facial expressions, voice characteristics, and other factors.

[0811] "Means of providing feedback" refer to techniques and devices for communicating advice and information based on sentiment analysis to workers.

[0812] "Means for accumulating report content" refers to a system that records data and results after work is completed, and uses them for later analysis and improvement.

[0813] "Methods for creating improvement measures" refers to the process of using accumulated work data and sentiment analysis results to derive areas for improvement in work procedures and the work environment.

[0814] The system that realizes this invention mainly consists of a server, a terminal (in this case, smart glasses), and an operator.

[0815] The server receives information stored in data storage devices, analyzes the information, and derives relevant data. Specifically, it functions as the core for automatically generating efficient instructions based on past work data and environmental information. The server also plays a role in providing feedback to workers, such as emotion-based advice and relaxation techniques, based on data provided by the emotion analysis engine. For this purpose, a cloud-based data processing platform and an analysis programming environment such as Python or R are likely to be used.

[0816] The smart glasses used as terminals are equipped with a camera and microphone, capturing the worker's facial expressions and voice in real time. Based on this, an emotion analysis engine analyzes the worker's emotional state. Based on the analysis results, instructions and feedback created by the server are displayed on the glasses' screen. For example, if a worker is determined to be irritated during work, they may receive instructions such as, "Take a deep breath and try meditating for one minute." An example of a prompt message is, "Please determine your stress level based on the current voice and facial expression data."

[0817] By receiving this feedback, users (workers) can manage their own work pace and status in real time. Furthermore, the data recorded by the emotion engine is accumulated over the long term and used for analysis to improve the work environment and enhance individual performance. This enables flexible and efficient work support tailored to individual needs.

[0818] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0819] Step 1:

[0820] The server retrieves work-related information and past work data from the database as input data. It analyzes this data and automatically generates instructions tailored to the work content. The generated instructions are then used as output data to be sent to the terminal. This process includes executing database queries and organizing and analyzing the obtained data.

[0821] Step 2:

[0822] The terminal uses its built-in camera and microphone to collect the worker's facial expressions and voice as input. Based on this input data, an emotion analysis engine calculates the worker's emotional state and outputs the analysis results to the server. Specifically, it performs facial expression analysis using image recognition technology and voice recognition.

[0823] Step 3:

[0824] The server uses emotion analysis results received from terminals and worker productivity data as input to evaluate stress levels and concentration levels. Based on this, it generates appropriate feedback and break instructions for the workers. Output data includes relaxation techniques and concise instructions. This step utilizes a generative AI model to integrate and analyze the collected data.

[0825] Step 4:

[0826] The terminal displays feedback and instructions received from the server on the smart glasses' display, notifying the worker. This display content is then used as output to prompt the user to take appropriate action. Specifically, this involves popping up messages or playing audio guidance.

[0827] Step 5:

[0828] The user (worker) checks notifications on their device and adjusts their work pace or takes breaks according to the feedback. This step requires the worker to take actions that maintain their own productivity and safety.

[0829] Step 6:

[0830] The server receives the report as input after the work is completed and derives improvement measures for the next task. This generates output data containing proposed improvements for future work instructions. The data is stored and analyzed over the long term to optimize the work environment.

[0831] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0832] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0833] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0834] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0835] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0836] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0837] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0838] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0839] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0840] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0841] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0842] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0843] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0844] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0845] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0846] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0847] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0848] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0849] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0850] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0851] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0852] The following is further disclosed regarding the embodiments described above.

[0853] (Claim 1)

[0854] Means for retrieving information from a database,

[0855] A means of analyzing acquired information and extracting relevant information,

[0856] A means of automatically generating instructions based on extracted information,

[0857] A means of presenting instructions to workers and performing necessary preparations and checks,

[0858] A means of receiving input from workers during work and providing information in real time,

[0859] A method for recording the report details after the work is completed and generating improvement plans for the next work,

[0860] A system that includes this.

[0861] (Claim 2)

[0862] The system according to claim 1, which modifies work procedures based on collected information.

[0863] (Claim 3)

[0864] The system according to claim 1, which responds to voice input from an operator and provides voice guidance.

[0865] "Example 1"

[0866] (Claim 1)

[0867] A means by which an information processing device accesses information sources via a network and collects information about the equipment,

[0868] A means of analyzing collected information and identifying necessary work information using text processing technology,

[0869] Means for creating instructions in electronic format based on identified information and generating structured instructions,

[0870] A means of displaying instructions on a display device used by the worker and confirming the preparation status using a checklist,

[0871] A means of receiving voice or other input from workers during work and providing knowledge in real time,

[0872] A method for recording the report details after the work is completed, generating improvement measures based on that data, and reflecting them in the next work,

[0873] A system that includes this.

[0874] (Claim 2)

[0875] The system according to claim 1, which adjusts and optimizes work procedures based on analysis.

[0876] (Claim 3)

[0877] The system according to claim 1, which adapts to the operator's voice instructions and provides guidance by voice.

[0878] "Application Example 1"

[0879] (Claim 1)

[0880] Means of obtaining information from an information storage facility,

[0881] A means of analyzing acquired information and extracting relevant information,

[0882] A means of automatically generating instructions based on extracted information,

[0883] A means of presenting instructions to workers and confirming necessary preparations,

[0884] A means of receiving input from workers during work and providing information in real time,

[0885] A means of recording the report content after the completion of the work and generating improvement plans for the next work,

[0886] An additional means of visually presenting auxiliary information through an ergonomic interface,

[0887] A system that includes this.

[0888] (Claim 2)

[0889] The system according to claim 1, which modifies work procedures based on collected information and provides visual support for the work.

[0890] (Claim 3)

[0891] The system according to claim 1, which responds to voice input from workers and provides voice guidance and visual guidelines.

[0892] "Example 2 of combining an emotion engine"

[0893] (Claim 1)

[0894] Means for acquiring data from a data storage device,

[0895] A means of analyzing acquired data and extracting relevant data,

[0896] A means of automatically generating instruction information based on extracted data,

[0897] A means of displaying instruction information to workers and performing necessary preparation checks,

[0898] A means of receiving information from workers during work and providing that information in real time,

[0899] A means for detecting the emotional state of a worker and transmitting the analysis data to a higher-level device,

[0900] A means by which a higher-level device generates and presents feedback based on the worker's emotion analysis data,

[0901] A method for recording the report details after completing the work and making improvements for the next work,

[0902] A system that includes this.

[0903] (Claim 2)

[0904] The system according to claim 1, which modifies work procedures based on accumulated data.

[0905] (Claim 3)

[0906] The system according to claim 1, which analyzes the voice input and facial expression data of an operator and provides voice and visual guidance.

[0907] "Application example 2 when combining with an emotional engine"

[0908] (Claim 1)

[0909] A means of receiving information from a data storage device,

[0910] A means of analyzing received information and deriving relevant information,

[0911] A means of automatically generating instructions based on the derived information,

[0912] A means of presenting instructions to workers and performing necessary checks for the work,

[0913] A means of receiving information from workers during work and supplying that information immediately,

[0914] A means for analyzing the emotional state of workers using an emotion analysis engine and providing appropriate feedback based on the analysis results,

[0915] A method for accumulating report content after work completion and creating improvement measures for the next work,

[0916] A system that includes this.

[0917] (Claim 2)

[0918] The system according to claim 1, which modifies work procedures based on collected information and sentiment analysis results, and provides feedback that corresponds to the worker's emotions.

[0919] (Claim 3)

[0920] The system according to claim 1, which responds to voice input from workers, provides voice guidance, and also provides relaxation guidance according to their emotional state. [Explanation of Symbols]

[0921] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. Means of obtaining information from an information storage facility, A means of analyzing acquired information and extracting relevant information, A means of automatically generating instructions based on extracted information, A means of presenting instructions to workers and confirming necessary preparations, A means of receiving input from workers during work and providing information in real time, A means of recording the report content after the completion of the work and generating improvement plans for the next work, An additional means of visually presenting auxiliary information through an ergonomic interface, A system that includes this.

2. The system according to claim 1, which modifies work procedures based on collected information and provides visual support for the work.

3. The system according to claim 1, which responds to voice input from workers and provides voice guidance and visual guidelines.

Images