System

Abstract

To provide a system.SOLUTION: A system comprises: data collection means for collecting data which indicates a work state in a site and a task feeling of a user; means for creating an educational training content based on the collected data, using a generative AI; and display means for displaying the created educational training content to the user.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including a description and related instruction sentence regarding the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-180282 A Summary of the Invention [Problem to be solved by the invention]

[0004] Companies in the railroad, manufacturing, and medical industries have introduced educational training content using VR goggles, but since content creation is expensive, many companies use the same content for a certain number of years. As a result, there is a challenge in that it is difficult to update the educational training content according to changes in the on-site situation or the issues faced by the target employees. [Means for solving the problem]

[0005] As a means to solve this problem, we provide a generation AI that generates educational training content that takes into account the on-site situation and the sense of issues of the target employees, and VR goggles that display the educational training content generated by the generation AI. This makes it possible to flexibly update the educational training content according to the on-site situation and the sense of issues of the target employees, and realize more effective training. [Brief description of the drawings]

[0006] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Diagram 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. FIG. [Diagram 3] FIG. 11 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Diagram 5] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 13 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 2 is a sequence diagram showing a process flow of the data processing system in the first embodiment of the first form example. [Figure 12] FIG. 11 is a sequence diagram showing a process flow of the data processing system in the application example 1 of the first embodiment. [Figure 13]FIG. 11 is a sequence diagram showing a process flow of the data processing system in the second embodiment of the second embodiment. [Figure 14] FIG. 13 is a sequence diagram showing a process flow of the data processing system in application example 2 of embodiment 2. [Figure 15] FIG. 13 is a sequence diagram showing a process flow of a data processing system in a third embodiment of the present invention. [Figure 16] FIG. 13 is a sequence diagram showing a process flow of the data processing system in the third application example of the third embodiment. [Figure 17] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. [Figure 18] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Form Example 1 when combined with an emotion engine. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0008] First, the terms used in the following description will be explained.

[0009] In the following embodiments, a signed processor (hereinafter simply referred to as a "processor") may be one arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be one type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

[0010] In the following embodiments, a signed RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by the processor.

[0011] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0012] In the following embodiments, a communication I / F (Interface) with a code is an interface including a communication processor and an antenna. The communication I / F controls communication between multiple computers. An example of a communication standard applied to the communication I / F is a wireless communication standard including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same idea as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0014] [First embodiment]

[0015] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0016] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0017] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0018] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0019] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (e.g., a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0020] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (e.g., voice and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs voice according to instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a Complementary Metal-Oxide-Semiconductor (CMOS) image sensor or a Charge Coupled Device (CCD) image sensor.

[0021] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54.

[0022] FIG. 2 shows an example of main functions of the data processing device 12 and the smart device 14.

[0023] As shown in Fig. 2, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32. The specific process program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific process program 56 from the storage 32, and executes the read specific process program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific process program 56 executed on the RAM 30.

[0024] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0025] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores a reception output program 60. The reception output program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads out the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0026] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0027] "Example 1"

[0028] As one embodiment of the present invention, there is a system that provides a generation AI that receives as input the on-site situation and the target employee's sense of issues, and VR goggles that display the educational training content generated by the generation AI. The generation AI generates educational training content that takes into account the on-site situation and the target employee's sense of issues. Specifically, the on-site situation may be, for example, the work situation on the production line or the operation status of the railway, and the target employee's sense of issues may be, for example, the difficulty of the work or the need for safety measures, as input. Then, the educational training content is generated based on this information. The generated educational training content is displayed to the target employee through the VR goggles, and supports the training of the target employee.

[0029] "Example 2"

[0030] As one embodiment of the present invention, there is a system that provides a generation AI that receives as input the on-site situation and the target employee's sense of issues, and VR goggles that display the educational training content generated by the generation AI. The generation AI generates educational training content that takes into account the on-site situation and the target employee's sense of issues. Specifically, the on-site situation may be, for example, the work situation on the production line or the operation status of the railway, and the target employee's sense of issues may be, for example, the difficulty of the work or the need for safety measures, as input. Then, the educational training content is generated based on this information. The generated educational training content is displayed to the target employee through the VR goggles, and supports the training of the target employee.

[0031] The process flow of each embodiment will be described below.

[0032] "Example 1"

[0033] Step 1: The generation AI receives as input the on-site situation and the target employee's perception of the problem. For example, the on-site situation may be the work situation on the production line or the train operation status, while for example, the target employee's perception of the problem may be the difficulty of the work or the need for safety measures.

[0034] Step 2: The AI ​​generates training content based on the received on-site situation and the employee's sense of challenge. For example, if the production line is congested, the AI ​​generates training content that includes work procedures and precautions to deal with the congestion.

[0035] Step 3: The generated educational training content is displayed to the target employees through VR goggles. The target employees experience the educational training content through the VR goggles and receive the training.

[0036] Example 1

[0037] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal".

[0038] With conventional education and training systems, it was difficult to provide content that fully reflected the specific situation at the workplace and the individual issues faced by the target employees. As a result, the effectiveness of the training was limited, and it was difficult to acquire skills and knowledge that were suited to actual work. In addition, there was also the problem that it took a lot of time and effort to generate and distribute training content.

[0039] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0040] In this invention, the server includes a generating AI means for generating educational training content that takes into account the on-site situation and the target employee's sense of issues, a visual display device means for displaying the educational training content generated by the generating AI, an input device means for inputting the on-site situation and the target employee's sense of issues, and a communication means for transmitting the generated educational training content to the visual display device. This makes it possible to quickly and efficiently generate and display educational training content that reflects the specific on-site situation and the individual sense of issues of the target employee.

[0041] "On-site conditions" refer to the specific states and conditions in a particular business environment, such as the working conditions on a manufacturing line or the operation status of a train.

[0042] "The target employees' sense of challenge" refers to the work-related challenges and problems that specific employees feel, such as the difficulty of the work or the need for safety measures.

[0043] "Educational and training content" refers to learning materials and training programs aimed at improving employee skills and acquiring knowledge.

[0044] "Generative AI" refers to artificial intelligence that automatically generates educational and training content based on on-site conditions and the issues perceived by target employees.

[0045] A "visual display device" is a device for displaying the generated educational and training content, specifically referring to VR goggles and the like.

[0046] An "input device" refers to a device that provides an interface for users to input on-site conditions and the issues perceived by target employees.

[0047] "Communication means" refers to the technology or protocols used to transmit the generated educational and training content from the server to the visual display device.

[0048] The present invention is a system for generating educational and training content based on the situation at the workplace and the sense of issues faced by target employees, and displaying the content on a visual display device. A specific embodiment of this system will be described below.

[0049] server

[0050] The server hosts the generative AI model and receives the on-site situation and the target employees' sense of issues as input. The generative AI model generates educational training content based on this input data. Specifically, the server performs the following processes:

[0051] 1. Data Reception

[0052] The server receives the on-site situation and the employee's sense of challenge sent from the terminal, such as data on the work situation on the manufacturing line, the operation status of trains, the difficulty of the work, and the need for safety measures.

[0053] 2. Data Analysis

[0054] The server analyzes the received data and converts it into a format suitable for the generative AI model, for example by converting text data into numerical data and extracting necessary features.

[0055] 3. Content Creation

[0056] The server generates educational and training content using a generative AI model, which creates optimal educational and training content based on the on-site situation and perceived issues.

[0057] 4. Content Submission

[0058] The server transmits the generated educational / training content to the terminal, specifically, by converting it into a data format suitable for display on a visual display device.

[0059] Terminal

[0060] The terminal provides an interface for users to input information about the on-site situation and the issues the target employees face. The data entered by the user is sent from the terminal to the server. After the generation AI on the server generates the educational training content, the terminal transfers the content to the visual display device. The specific operation is as follows:

[0061] 1. Data Entry

[0062] The terminal provides an interface for the user to input information about the situation at the site and the issues faced by the target employees. For example, data is entered using text boxes and drop-down menus.

[0063] 2. Data transmission

[0064] The terminal transmits the input data to the server. Specifically, the terminal transmits the data by using an HTTP request.

[0065] 3. Receiving Content

[0066] The terminal receives the educational and training content transmitted from the server and prepares the received data for transfer to the visual display device.

[0067] 4. Content Display

[0068] The terminal transfers the received educational / training content to the visual display device for display, specifically, by transmitting data for displaying the content on the display of the visual display device.

[0069] User

[0070] The user uses the terminal to input the on-site situation and the issues the target employees feel. For example, they can input information such as "The speed of Line A is too fast" or "New employees are not used to the work." The input data is sent to the server, and the generation AI generates educational and training content. The generated content is displayed to the user through a visual display device. The specific operation is as follows:

[0071] 1. Data Entry

[0072] The user uses the terminal to input information about the on-site situation and issues, such as "The speed of Line A is too fast" or "New employees are not used to the work."

[0073] 2. Data verification

[0074] The user reviews the data they entered to ensure it is accurate and, if necessary, corrects it.

[0075] 3. Training

[0076] The user wears the visual display device and receives the generated educational and training content. Specifically, the user views the content displayed on the display of the visual display device and performs operations according to the instructions.

[0077] Examples

[0078] Example 1

[0079] The user inputs the work situation of the production line as "Line A is running too fast" and the employee's sense of challenge as "New employees are not used to the work." The generation AI on the server uses this information to generate educational training content for new employees. The generated content is shown to new employees through a visual display device, helping them with training on work procedures and safety measures.

[0080] Example 2

[0081] The user inputs the train operation status as "frequent delays" and the target employee's sense of the problem as "efficiency in traffic management is necessary." The generation AI uses this information to generate educational and training content on the efficiency of traffic management. The generated content is shown to traffic management personnel via a visual display device, supporting training on efficient traffic management methods.

[0082] Examples of prompt statements

[0083] "Please enter the operation status of the production line. Example: Line A is running fast."

[0084] "Please enter the problem that the target employee feels. Example: New employees are not familiar with the work."

[0085] "Please enter the train operation status. Example: Frequent delays."

[0086] "Please enter the issues the target employees face. Example: Traffic management needs to be made more efficient."

[0087] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0088] Step 1:

[0089] Data entry (terminal)

[0090] The terminal provides an interface for the user to input the situation at the site and the perceived issues of the target employees. For example, the user inputs information such as "The speed of Line A is fast" or "New employees are not familiar with the work." The input data is saved in the terminal via text boxes and drop-down menus. The input data is text information about the situation at the site and perceived issues.

[0091] Step 2:

[0092] Data transmission (terminal)

[0093] The terminal transmits the input data on the field situation and the sense of issues to the server. Specifically, the data is transmitted using an HTTP request. The transmitted data is in a structured data format such as JSON. The input data is the text information transmitted to the server, and the output is the data transmitted to the server.

[0094] Step 3:

[0095] Data reception (server)

[0096] The server receives data on the field situation and issues sent from the terminal. The received data is analyzed on the server. The input data is the text information sent from the terminal, and the output is the data ready for analysis.

[0097] Step 4:

[0098] Data analysis (server)

[0099] The server analyzes the received data and converts it into a format suitable for the generative AI model. For example, it converts text data into numerical data and extracts the necessary features. The input data is the received text information, and the output is numerical data to be input into the generative AI model.

[0100] Step 5:

[0101] Content generation (server)

[0102] The server generates educational training content using a generative AI model. The generative AI model creates optimal educational training content based on the on-site situation and sense of issues. The input data is analyzed numerical data, and the output is the generated educational training content.

[0103] Step 6:

[0104] Content transmission (server)

[0105] The server transmits the generated educational / training content to the terminal. Specifically, it converts the content into a data format suitable for display on a visual display device and transmits it. The input data is the generated educational / training content, and the output is the content data transmitted to the terminal.

[0106] Step 7:

[0107] Content reception (terminal)

[0108] The terminal receives the educational and training content transmitted from the server. The terminal prepares the received data for transfer to the visual display device. The input data is the content data transmitted from the server, and the output is the data ready for transfer to the visual display device.

[0109] Step 8:

[0110] Content display (terminal)

[0111] The terminal transfers the received educational and training content to the visual display device for display. Specifically, the terminal transmits data for displaying the content on the display of the visual display device. The input data is the data ready to be transferred to the visual display device, and the output is the educational and training content displayed on the visual display device.

[0112] Step 9:

[0113] Attending training (user)

[0114] The user wears the visual display device and takes the generated educational and training content. Specifically, the user watches the content displayed on the display of the visual display device and performs operations according to the instructions. The input data is the educational and training content displayed on the visual display device, and the output is the user's progress in taking the training.

[0115] (Application example 1)

[0116] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0117] In order to effectively provide educational and training content based on the situation at the workplace and the sense of issues of the target employees, a means of generating and displaying appropriate information in real time is required. However, with conventional systems, it was difficult to provide educational and training content that fully reflected the situation at the workplace and the sense of issues of each employee, and the effectiveness of the training was limited. In addition, because the means of display were limited, it was not possible to provide a flexible training environment.

[0118] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0119] In this invention, the server includes a generating AI means for generating educational training content that takes into account the on-site situation and the sense of issues of the target employee, a VR goggle means for displaying the educational training content generated by the generating AI, and a smart glasses means for displaying the educational training content generated by the generating AI. This makes it possible to generate customized educational training content based on the on-site situation and the sense of issues of each employee in real time and provide it through a flexible display means.

[0120] "Site conditions" refers to the current work progress and environmental conditions at the work site.

[0121] "Target Employees" refers to employees who are intended to receive the educational and training content.

[0122] "Sense of challenge" refers to the concerns felt by the employees, such as the difficulty of the work and the need for safety measures.

[0123] "Educational and training content" refers to educational materials and training programs aimed at improving the skills and knowledge of target employees.

[0124] "Generative AI" refers to artificial intelligence that takes on-site conditions and the target employees' sense of challenges as input and generates educational training content based on that.

[0125] "VR goggles" refers to a head-mounted display for displaying a virtual reality environment.

[0126] "Smart glasses" refers to a glasses-type device that has augmented reality and information display functions.

[0127] The system for carrying out the present invention includes a generating AI means for generating educational training content that takes into account the on-site situation and the sense of issues of the target employees, and a VR goggle means and a smart glasses means for displaying the educational training content generated by the generating AI. A specific embodiment of this system will be described below.

[0128] System Programming

[0129] The server receives the on-site situation and the target employee's sense of challenge as input, and executes the generation AI that generates educational training content based on that. The generation AI sends prompts using the OpenAI (registered trademark) API and generates educational training content. The generated content is displayed on the VR goggles and smart glasses.

[0130] Processing Description

[0131] The server first obtains the on-site situation and the target employee's sense of challenge as input data. This input data includes, for example, the working situation on the production line, the difficulty of the work, and the need for safety measures. Next, the server sends a prompt to the generation AI to generate education and training content. This prompt specifically describes the on-site situation and the sense of challenge. The generated education and training content is sent to the VR goggles and smart glasses and displayed to the target employee.

[0132] Hardware and Software Used

[0133] Hardware: Servers, VR goggles, smart glasses

[0134] Software: OpenAI API, Python (registered trademark)

[0135] Examples

[0136] For example, if the work situation on a production line is "assembly of parts" and the worker's sense of challenge is "installation of parts is difficult," the generative AI will generate educational and training content including "detailed steps on how to install parts" and "safety measures." The generated content is displayed to the worker through VR goggles or smart glasses.

[0137] Examples of prompt statements

[0138] Site situation: Parts assembly work

[0139] Problem: Difficult to install parts

[0140] Generate educational training content based on this.

[0141] In this way, the server can generate customized educational and training content in real time based on the situation at the workplace and the issues each employee faces, and provide it through flexible display means.

[0142] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0143] Step 1:

[0144] The server receives input data on the on-site situation and the employees' perceptions of the issues. Specifically, it collects information such as the work situation on the production line, the difficulty of the work, and the need for safety measures through sensors and user input. The input data is sent to the server in JSON format.

[0145] Step 2:

[0146] The server generates a prompt sentence based on the acquired on-site situation and problem perception. Specifically, it converts the on-site situation and problem perception into text format and creates a prompt sentence to send to the generation AI. For example, the format might be "On-site situation: Parts assembly work, Problem perception: Parts installation is difficult."

[0147] Step 3:

[0148] The server sends the prompt text to the generation AI and generates the educational training content. Specifically, the server uses the OpenAI API to send the prompt text and receives the generated text data. The generation AI generates appropriate educational training content based on the input prompt text.

[0149] Step 4:

[0150] The server transmits the generated educational and training content to the VR goggles and smart glasses. Specifically, the server converts the generated text data into a display format for the VR goggles and smart glasses and transmits it to the device. The device displays the received content.

[0151] Step 5:

[0152] Users visually check the educational training content through VR goggles or smart glasses. Specifically, they read the content displayed on the device's display and receive the training. Users perform tasks based on the displayed information and acquire the necessary skills and knowledge.

[0153] Example 2

[0154] Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0155] There is a need to efficiently generate educational and training content that takes into account the situation at the workplace and the issues faced by the target employees, and to provide effective training to the target employees. However, with conventional systems, it was difficult to generate content that appropriately reflected the situation at the workplace and the issues faced, and there were also limited means to effectively display the generated content. This made it difficult to provide training that met the specific situation at the workplace and the needs of each individual employee.

[0156] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0157] In this invention, the server includes an interface means for the user to input the on-site situation and the sense of issues of the target employee, a means for the server to receive the input data and generate a prompt sentence for the generative AI model, and a means for the server to transmit the generated education and training content to the terminal. This makes it possible to efficiently generate education and training content that reflects the on-site situation and the sense of issues, and to provide effective training to the target employee through the VR goggles.

[0158] "On-site conditions" refer to the specific states and conditions in a particular business environment, such as the working conditions on a manufacturing line or the operation status of a train.

[0159] "Target Employees" refers to the specific employees intended to receive the educational and training content.

[0160] "Sense of challenge" refers to specific problems or issues in the work, such as the difficulty of the work or the need for safety measures as perceived by the employees.

[0161] "Educational and training content" refers to content that is created based on on-site conditions and issues, and is intended for the education and training of target employees.

[0162] "Generative AI means" refers to the function of artificial intelligence that receives on-site conditions and sense of issues as input and generates educational training content based on that.

[0163] "VR goggles means" refers to a device that displays the educational and training content generated by the generating AI means and provides training in a virtual reality environment to target employees.

[0164] "Interface means" refers to the input devices and software of the system that allow the user to input the on-site situation and the target employees' sense of issues.

[0165] A "prompt sentence" is an instruction sentence to be input into a generative AI model, and refers to text that specifically describes the on-site situation and the sense of the problem.

[0166] "Terminal" refers to a device that receives educational and training content sent from the server and displays it on the VR goggles.

[0167] This invention is a system that generates educational training content based on the on-site situation and the employee's sense of issues, and displays it to the employee through VR goggles. A specific embodiment of this system will be described below.

[0168] The user inputs the on-site situation and the employee's perception of the issues through the system interface. For example, the speed of the production line, the difficulty of the work, the need for safety measures, etc. The interface can be a web browser or dedicated application software.

[0169] The server receives the on-site situation and sense of the problem input by the user, and generates a prompt sentence to be input to the generative AI model based on this information. As an example of the generative AI model, OpenAI's GPT-4 (registered trademark) is used. An example of a prompt sentence is, "The speed of the manufacturing line is fast, and new employees are having trouble remembering the work procedures. Please generate educational training content for this."

[0170] The server inputs the generated prompt sentence into the generative AI model, and the generative AI model generates educational training content based on the prompt sentence. The generated educational training content is, for example, VR content that provides detailed explanations of work procedures on a manufacturing line.

[0171] The server transmits the generated educational and training content to the terminal. The terminal is a device (e.g., a PC or a smartphone) that is linked to the VR goggles. The terminal displays the received educational and training content on the VR goggles.

[0172] Users (target employees) wear VR goggles and receive training through the educational and training content displayed. For example, they can learn the work procedures of a manufacturing line while experiencing them in a virtual environment. This provides a realistic training experience that reflects the actual situation and challenges on the ground.

[0173] This system makes it possible to efficiently generate educational and training content that reflects on-site conditions and issues, and to provide effective training to target employees through VR goggles.

[0174] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0175] Step 1:

[0176] The user inputs the situation on-site and the target employee's perception of the issues.

[0177] Through the system's interface, users input information about the on-site situation and the issues the target employees face. For example, they input information such as the speed of the production line, the difficulty of the work, and the need for safety measures. The input data is sent to the server. An example of input data is, "The speed of the production line is so fast that new employees are having trouble learning the work procedures."

[0178] Step 2:

[0179] The server receives the input data and generates a prompt for the generative AI model.

[0180] The server receives data on the on-site situation and problem perception sent by the user. Next, based on this information, it generates a prompt sentence to be input to the generative AI model. Specifically, the server analyzes the input data and generates a prompt sentence such as, "The speed of the manufacturing line is fast, and new employees are having difficulty learning the work procedures. Please generate educational training content for this." The generated prompt sentence is sent to the generative AI model.

[0181] Step 3:

[0182] The server inputs prompt sentences into the generative AI model to generate educational training content.

[0183] The server inputs the generated prompt sentence into a generative AI model (e.g., OpenAI's GPT-4). The generative AI model generates educational training content based on the prompt sentence. Specifically, the generative AI model analyzes the prompt sentence and generates VR content that explains the work procedures of the manufacturing line in detail. The generated educational training content is returned to the server.

[0184] Step 4:

[0185] The server transmits the generated educational / training content to the terminal.

[0186] The server receives the educational training content generated from the generative AI model and transmits it to the terminal. The terminal is a device (e.g., a PC or a smartphone) that is linked to the VR goggles. The transmitted educational training content is stored in the terminal.

[0187] Step 5:

[0188] The terminal displays educational training content through VR goggles.

[0189] The terminal displays the received educational and training content on the VR goggles. Specifically, the terminal transmits the content to the VR goggles, and the VR goggles provide realistic images and sounds to the target employees.

[0190] Step 6:

[0191] Users (target employees) receive training using VR goggles.

[0192] Users (target employees) wear VR goggles and receive training through the educational and training content displayed. For example, they can learn the work procedures of a manufacturing line while experiencing them in a virtual environment. This provides a realistic training experience that reflects the actual situation and challenges on the ground.

[0193] (Application example 2)

[0194] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0195] There is a need to efficiently generate educational and training content based on the on-site work situation and the workers' sense of issues, and provide it to workers at the appropriate time. However, with conventional systems, it was difficult to quickly generate training content that reflected the real-time situation at the site, and it was also difficult to provide content that responded to the individual sense of issues of workers. This resulted in problems such as insufficient improvement of workers' skills and thorough implementation of safety measures.

[0196] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0197] In this invention, the server includes a generating AI means for generating educational training content that takes into account the on-site situation and the target employee's sense of problem, a VR goggle means for displaying the educational training content generated by the generating AI, a data collection means for collecting data from sensors in the factory, an input means for the worker to input the sense of problem, and a transmission means for transmitting the generated educational training content to the VR goggles. This makes it possible to quickly generate and provide educational training content based on the real-time situation of the on-site situation and the individual sense of problem of the worker.

[0198] "Site conditions" refers to the real-time work situation and environmental conditions in a factory or work site.

[0199] "Target employees' sense of issues" refers to individual issues and concerns felt by workers, such as the difficulty of the work and the need for safety measures.

[0200] "Educational and training content" refers to information and teaching materials for education and training aimed at improving worker skills and ensuring thorough safety measures.

[0201] "Generative AI means" refers to the function of artificial intelligence that takes on-site conditions and the target employees' sense of challenges as input and generates educational training content based on that.

[0202] "VR goggle means" refers to virtual reality goggles for displaying the educational and training content generated by the generation AI to workers.

[0203] "Data collection means" refers to the function of collecting information on on-site work conditions in real time using sensors and cameras within the factory.

[0204] "Input means" refers to devices such as smartphones or tablets that workers use to input their own perceptions of issues.

[0205] "Transmission means" refers to the communication function for transmitting the generated educational and training content to the VR goggles.

[0206] A system for implementing this invention includes a generation AI means for generating educational and training content that takes into account the on-site situation and the target employee's sense of challenges, a VR goggles means for displaying the educational and training content generated by the generation AI, a data collection means for collecting data from sensors within the factory, an input means for workers to input their sense of challenges, and a transmission means for transmitting the generated educational and training content to the VR goggles.

[0207] First, the server collects information on the work situation in the factory in real time using sensors and cameras. This makes it possible to grasp the real-time situation at the factory. Next, workers use their smartphones or tablets to input their own sense of the issues. This sense of the issues includes the difficulty of the work and the need for safety measures.

[0208] The collected information on the site conditions and the workers' sense of challenges are input into the generation AI. Based on this data, the generation AI generates optimal educational and training content. This generation AI uses a programming language such as Python. The generated educational and training content is sent to the VR goggles via a transmission means.

[0209] The VR goggles display the generated educational and training content to the workers, allowing them to receive educational and training based on the real-time situation at the site and their own sense of issues. This allows workers to efficiently improve their skills and ensure safety measures.

[0210] As a concrete example, consider a situation where the robot on the manufacturing line is operating normally, but the worker is feeling uneasy about the new operating procedure. In this case, the worker inputs the sense of challenge that the new operating procedure is difficult. Based on this information, the generative AI generates educational training content that makes it easy for the worker to understand the new operating procedure.

[0211] Examples of prompts for a generative AI model include:

[0212] Site situation: The robots on the manufacturing line are working normally.

[0213] Challenges: Workers find the new operating procedures difficult.

[0214] Use this information to generate educational training content that helps workers understand new operating procedures.

[0215] In this way, it becomes possible to quickly generate and provide educational and training content based on the real-time situation on-site and the individual challenges faced by workers.

[0216] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0217] Step 1:

[0218] The server collects information on the status of on-site work in real time using sensors and cameras in the factory. It receives data from the sensors and cameras as input and generates data indicating the status of on-site work as output. This data includes, for example, the operating status of the manufacturing line and the operating status of robots.

[0219] Step 2:

[0220] The user inputs their own sense of the problem using a smartphone or tablet. As input, the system receives information such as the difficulty of the task as perceived by the user and the need for safety measures, and as output, generates data indicating the sense of the problem. This data includes specific sense of the problem, such as "the new operating procedure is difficult."

[0221] Step 3:

[0222] The server inputs the collected on-site situation data and the user's problem perception data into the generation AI. It receives the on-site situation data and problem perception data as input, and generates educational training content as output. This generation AI is implemented using a programming language such as Python, and generates optimal educational training content based on the input data.

[0223] Step 4:

[0224] The server transmits the generated educational / training content to the VR goggles through a transmission means. As an input, the server receives the generated educational / training content, and as an output, generates data to be transmitted to the VR goggles. For this transmission means, a communication technology such as Wi-Fi or Bluetooth is used.

[0225] Step 5:

[0226] The terminal (VR goggles) displays the received educational and training content to the user. As input, it receives the transmitted educational and training content, and as output, it generates images and audio to be displayed to the user. This allows the user to receive educational and training based on the real-time situation in the field and their own sense of issues.

[0227] In this way, it becomes possible to rapidly generate and provide educational and training content based on the real-time situation on-site and the user's individual sense of the issues.

[0228] Furthermore, an emotion engine that estimates the emotion of the user may be combined. That is, the identification processing unit 290 may estimate the emotion of the user using the emotion identification model 59, and perform identification processing using the emotion of the user.

[0229] "Example 1"

[0230] In one embodiment of the present invention, the generation AI includes an emotion engine. The emotion engine recognizes the user's emotion and uses the emotion as an input for the generation AI. Specifically, the emotion is estimated from the user's facial expression, tone of voice, choice of words, etc., and the result is provided to the generation AI. The generation AI generates educational training content by considering this emotion information together with the on-site situation and the target employee's sense of challenge. For example, if the user feels confused, the generation AI takes the emotion into account and generates educational training content including more detailed explanations and concrete examples. Also, if the user feels excited, the generation AI takes the emotion into account and generates educational training content including more advanced topics. In this way, the present invention combined with the emotion engine makes it possible to provide personalized educational training content according to the user's emotion.

[0231] "Example 2"

[0232] In one embodiment of the present invention, the generation AI includes an emotion engine. The emotion engine recognizes the user's emotion and uses the emotion as an input for the generation AI. Specifically, the emotion is estimated from the user's facial expression, tone of voice, choice of words, etc., and the result is provided to the generation AI. The generation AI generates educational training content by considering this emotion information together with the on-site situation and the target employee's sense of challenge. For example, if the user feels confused, the generation AI takes the emotion into account and generates educational training content including more detailed explanations and concrete examples. Also, if the user feels excited, the generation AI takes the emotion into account and generates educational training content including more advanced topics. In this way, the present invention combined with the emotion engine makes it possible to provide personalized educational training content according to the user's emotion.

[0233] The process flow of each embodiment will be described below.

[0234] "Example 1"

[0235] Step 1: The emotion engine recognizes the user's emotions. Specifically, it infers emotions from the user's facial expressions, tone of voice, choice of words, etc.

[0236] Step 2: The emotion engine provides the recognized emotion information to the generation AI.

[0237] Step 3: The generation AI considers the emotion information along with the on-site situation and the target employee's sense of challenge to generate educational training content. For example, if the user feels confused, the generation AI considers that emotion to generate educational training content that includes more detailed explanations and concrete examples. Also, if the user feels excited, the generation AI considers that emotion to generate educational training content that includes more advanced topics.

[0238] Step 4: The educational and training content generated by the generation AI is sent to the VR goggles so that it can be viewed by the target employees.

[0239] Example 1

[0240] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal".

[0241] With conventional education and training systems, it was difficult to generate content that took into account the on-site situation and the issues faced by the target employees, and it was also not possible to provide personalized training content that reflected the user's emotions. As a result, the effectiveness of the training was limited, and there were problems with the level of understanding and satisfaction of the target employees decreasing.

[0242] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0243] In this invention, the server includes a generating AI means for generating educational training content that takes into account the on-site situation and the target employee's sense of problem, a VR goggle means for displaying the educational training content generated by the generating AI, and an emotion engine means for recognizing the user's emotion and using the emotion as an input for the generating AI. This makes it possible to provide personalized educational training content that reflects the user's emotion in addition to the on-site situation and the target employee's sense of problem.

[0244] "On-site conditions" refer to a specific work environment or the progress of work, including, for example, the work conditions on a manufacturing line or the operation status of a railway.

[0245] "The target employee's sense of challenge" refers to the work-related difficulties and areas for improvement that a particular employee feels, including, for example, the difficulty of the work or the need for safety measures.

[0246] "Generative AI" refers to artificial intelligence that automatically generates educational and training content based on input data.

[0247] "VR goggles" refers to a device that uses virtual reality technology to visually display information and is used to display educational training content to target employees.

[0248] An "emotion engine" refers to technology that recognizes emotions from a user's facial expressions, tone of voice, choice of words, etc., and uses that emotional information as input for generative AI.

[0249] "Educational and training content" refers to educational materials and training programs aimed at improving the skills and knowledge of target employees.

[0250] "User" refers to the person who operates the system and inputs the on-site situation and problem perception.

[0251] This invention is a system that provides a generation AI that receives the on-site situation and the subject employee's sense of issues as input and generates educational training content based on that, and a VR goggle that displays that content. It also includes an emotion engine that recognizes the user's emotions and uses those emotions as input for the generation AI.

[0252] System Configuration

[0253] 1. Generation AI

[0254] The generative AI generates educational training content by integrating the on-site situation, the target employees' sense of issues, and the user's emotional information. The software used is a natural language processing library (e.g., spaCy, NLTK) or a machine learning framework (e.g., TENSORFLOW (registered trademark), PyTorch).

[0255] 2. VR goggles

[0256] The VR goggles display educational training content generated by generative AI to target employees, providing an interactive training experience.

[0257] 3. Emotion Engine

[0258] The emotion engine recognizes emotions from the user's facial expressions, tone of voice, choice of words, etc., and provides the results to the generative AI, which then makes it possible to generate personalized educational and training content that reflects the user's emotions.

[0259] System Operation

[0260] 1. Data Entry

[0261] The user inputs the situation at the site and the employee's perception of the problem. For example, the user inputs information such as "Work on the production line is delayed," "The work is difficult," and "New safety measures are needed" into an input form on the terminal.

[0262] The device transmits the user's facial expressions, tone of voice, choice of words, etc. to the emotion engine. The device's camera and microphone capture the user's facial expressions and voice and transmit the data to the emotion engine in real time.

[0263] 2. Data Processing

[0264] The emotion engine estimates the user's emotions and provides the results to the generation AI. The emotion engine estimates that the user is confused from their facial expression and sends that information to the generation AI.

[0265] The generative AI integrates the on-site situation, the target employees' sense of issues, and the user's emotional information to generate educational and training content.The generative AI generates educational and training content including detailed explanations and concrete examples based on information such as "production line work is delayed," "the work is difficult," "new safety measures are needed," and "users are confused."

[0266] 3. Content Display

[0267] The server transmits the generated educational training content to the VR goggles. The server transfers data via a network to transmit the educational training content received from the generation AI to the VR goggles.

[0268] The VR goggles display the educational training content to the target employees. The VR goggles display the received educational training content in the target employees' field of vision, providing an interactive training experience.

[0269] Examples and prompts

[0270] Examples

[0271] Situation on the ground: Work on the production line is slow.

[0272] Challenges: The work is difficult and new safety measures are needed.

[0273] User sentiment: Confused.

[0274] In this case, the generative AI generates educational and training content with detailed explanations and concrete examples, and displays it on the VR goggles.

[0275] Examples of prompt statements

[0276] Situation on the ground: Work on the production line is slow.

[0277] Challenges: The work is difficult and new safety measures are needed.

[0278] User sentiment: Confused.

[0279] Based on this, generate educational training content with detailed explanations and concrete examples.

[0280] In this way, the system provides personalized educational and training content that takes into account user input and sentiment.

[0281] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0282] Step 1:

[0283] The user inputs the situation on-site and the target employee's perception of the issues.

[0284] Input: On-site situation (e.g., work on the production line is delayed), target employee's sense of challenge (e.g., the work is difficult, new safety measures are needed)

[0285] Output: Data on the situation on the ground and issues

[0286] Specific operations: The user enters information such as "Work on the production line is delayed," "The work is difficult," and "New safety measures are needed" into an input form on the terminal.

[0287] Step 2:

[0288] The device transmits the user's facial expressions, tone of voice, choice of words, etc. to the emotion engine.

[0289] Input: User's facial expressions, tone of voice, and word choice

[0290] Output: Emotion data sent to the emotion engine

[0291] How it works: The device's camera and microphone capture the user's facial expressions and voice, and send that data to the emotion engine in real time.

[0292] Step 3:

[0293] The emotion engine estimates the user's emotions and provides the results to the generative AI.

[0294] Input: User's facial expressions, tone of voice, and word choice

[0295] Output: Estimated emotion data

[0296] Specific operation: The emotion engine infers that the user is confused from their facial expression and sends that information to the generation AI.

[0297] Step 4:

[0298] The generative AI integrates the on-site situation, the target employees' sense of issues, and the user's emotional information to generate educational training content.

[0299] Input: On-site situation, target employee's sense of issues, user's emotional information

[0300] Output: Educational and training content

[0301] Specific operation: Based on information such as "work on the production line is delayed," "the work is difficult," "new safety measures are needed," and "users are confused," the generative AI generates educational and training content including detailed explanations and concrete examples.

[0302] Step 5:

[0303] The server transmits the generated educational and training content to the VR goggles.

[0304] Input: Educational and training content

[0305] Output: Data sent to the VR goggles

[0306] Specific operation: The server transfers data via the network to send the educational training content received from the generation AI to the VR goggles.

[0307] Step 6:

[0308] VR goggles display educational training content to target employees.

[0309] Input: Educational and training content

[0310] Output: Educational training content displayed to target employees

[0311] Specific operation: The VR goggles display the received educational training content in the target employee's field of vision, providing an interactive training experience.

[0312] (Application example 1)

[0313] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0314] In conventional education and training systems, it was difficult to provide content that fully reflected the specific situation at the workplace and the individual issues felt by the target employees. In addition, there was a lack of means to provide personalized training content that took into account the feelings of the target employees, so the effectiveness of the training was limited. This made it difficult to implement efficient and effective education and training, especially at workplaces such as factories.

[0315] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0316] In this invention, the server includes a generating AI means for generating educational training content that takes into account the on-site situation and the target employee's sense of issues, a VR goggle means for displaying the educational training content generated by the generating AI, a sensor means for collecting on-site situations in real time, and an emotion engine means for recognizing the emotions of the target employee. This makes it possible to provide personalized educational training content that takes into account the specific on-site situation, the individual sense of issues of the target employee, and even emotions.

[0317] "Site conditions" refers to the progress of work and the state of the environment at a factory or work site.

[0318] "Target employees" refers to employees or workers who are intended to receive the educational and training content.

[0319] "Sense of challenge" refers to the individual issues and concerns felt by the employees, such as the difficulty of the work or the need for safety measures.

[0320] "Educational and training content" refers to educational materials and training programs aimed at improving the skills and knowledge of target employees.

[0321] "Generative AI" refers to artificial intelligence that takes on-site conditions and the target employees' sense of challenges as input and generates educational training content based on that.

[0322] "VR goggles" refers to a virtual reality device for displaying the generated educational training content to target employees.

[0323] A "sensor" refers to a device or equipment used to collect information on site conditions in real time.

[0324] An "emotion engine" refers to a system that recognizes emotions from the facial expressions and tone of voice of the target employee, and provides that information to the generating AI.

[0325] A system for implementing this invention includes a generation AI means for generating educational training content that takes into account the on-site situation and the target employee's sense of challenges, a VR goggle means for displaying the educational training content generated by the generation AI, a sensor means for collecting on-site conditions in real time, and an emotion engine means for recognizing the emotions of the target employee.

[0326] The server first uses sensors to collect real-time information about the on-site situation. Sensors include cameras and environmental sensors in the factory, and obtain data on the progress of work and the state of the environment.

[0327] Next, voice input and tablets are used to collect the employees' perceptions of the issues they face. The employees can input their own perceptions of the difficulty of the work and the need for safety measures.

[0328] In addition, the system uses an emotion engine to recognize the emotions of the target employee. The emotion engine uses a camera and microphone to analyze the facial expressions and tone of voice of the target employee and estimates their emotions.

[0329] All of this data is input into the Generative AI, which generates optimal educational and training content for the target employees based on the on-site situation, sense of issues, and emotional information. The generated content includes specific operating procedures and safety measures, and helps the target employees improve their skills and acquire knowledge.

[0330] The generated educational and training content is displayed to the target employees through VR goggles, which are virtual reality devices that allow the target employees to visually experience the generated content.

[0331] As a specific example, when learning how to operate a new machine in a factory, if an employee feels that "this operation is difficult," the generative AI will take into account that sense of challenge and the employee's confused expression and generate educational training content that includes more detailed operating procedures and safety measures.

[0332] An example of a prompt is as follows:

[0333] Field situation: learning how to operate new machinery

[0334] Challenge: The task is difficult

[0335] Emotion: Confused

[0336] Content to be generated: Educational and training content including detailed operating procedures and safety measures

[0337] In this way, the system is able to provide personalized educational and training content that takes into account the specific situation on-site, the individual challenges faced by the target employees, and even their emotions.

[0338] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0339] Step 1:

[0340] The server uses sensors to collect real-time information about the site conditions. Sensors include cameras and environmental sensors in the factory, and obtain data about the progress of work and the state of the environment. The input is real-time data from the sensors, and the output is data that indicates the site conditions. The server collects this data and passes it on to the next processing step.

[0341] Step 2:

[0342] The user uses voice input or a tablet to collect the target employees' perceptions of the issues. The target employees input information such as their perceptions of the difficulty of the work and the need for safety measures. The input is data on the target employees' perceptions of the issues, and the output is data indicating the perceptions of the issues. The server collects this data and passes it on to the next processing step.

[0343] Step 3:

[0344] The server uses an emotion engine to recognize the target employee's emotions. The emotion engine uses a camera and microphone to analyze the target employee's facial expressions and tone of voice to infer emotions. The input is data from the camera and microphone, and the output is data indicating emotions. The server collects this data and passes it to the next processing step.

[0345] Step 4:

[0346] The server inputs the collected information on the on-site situation, sense of issues, and emotions into the generation AI. Based on this data, the generation AI generates the optimal educational training content for the target employees. The input is the on-site situation, sense of issues, and emotional information, and the output is the educational training content. The generation AI analyzes the data and generates the appropriate content.

[0347] Step 5:

[0348] The server transmits the generated educational / training content to the VR goggles. The VR goggles display the generated content to the target employee. The input is the generated educational / training content, and the output is the content displayed on the VR goggles. The target employee wears the VR goggles and visually experiences the training content.

[0349] In this way, the system is able to provide personalized educational and training content that takes into account the specific situation on-site, the individual challenges faced by the target employees, and even their emotions.

[0350] Example 2

[0351] Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0352] In conventional education and training systems, it was difficult to provide content that fully reflected the on-site situation and the issues faced by the target employees, and it was not possible to provide personalized training content that responded to the user's emotions. As a result, the effectiveness of the training was limited, and there was a problem of a decrease in the understanding and satisfaction of the target employees.

[0353] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0354] In this invention, the server includes a generating AI means for generating educational training content that takes into account the on-site situation and the target employee's sense of problem, a VR goggle means for displaying the educational training content generated by the generating AI, and an emotion engine means for recognizing the user's emotion and using the emotion as an input for the generating AI. This makes it possible to provide personalized educational training content that takes into account the on-site situation, the target employee's sense of problem, and the user's emotion.

[0355] "On-site conditions" refers to the actual conditions in a specific business environment, such as the working conditions on a manufacturing line or the operation status of a railway.

[0356] "The target employees' sense of challenge" refers to the work-related challenges and problems that specific employees feel, such as the difficulty of the work or the need for safety measures.

[0357] "Generative AI" refers to artificial intelligence that generates educational and training content based on on-site conditions and the issues perceived by target employees.

[0358] "VR goggles" refers to a virtual reality device for visually displaying generated educational and training content.

[0359] An "emotion engine" is a system that recognizes emotions from a user's facial expressions, tone of voice, choice of words, etc., and provides that information to the generative AI.

[0360] "Educational and training content" refers to teaching materials and training materials that enable target employees to learn the knowledge and skills necessary for their jobs.

[0361] "User" refers to employees and participants who use the educational and training content.

[0362] "Server" refers to the computer system that manages and operates the entire system, including the generative AI and emotion engine.

[0363] The present invention is a system that generates educational training content based on the on-site situation and the sense of issues of the target employees, and provides personalized training according to the user's emotions. A specific embodiment of this system will be described below.

[0364] System Configuration

[0365] The system consists of the following main components:

[0366] 1. Generative AI means: Artificial intelligence that generates educational and training content based on on-site conditions and the issues faced by target employees.

[0367] 2. VR goggles means: a virtual reality device that visually displays the generated educational training content.

[0368] 3. Emotion engine means: A system that recognizes emotions from the user's facial expressions, tone of voice, choice of words, etc., and provides that information to the generation AI.

[0369] Hardware and Software Configuration

[0370] The server manages and operates the entire system, including the generation AI and the emotion engine. The generation AI receives the on-site situation and the target employees' sense of issues as input, and generates educational training content. The emotion engine recognizes the user's emotions and provides that information to the generation AI. The VR goggles display the generated educational training content to the user.

[0371] Data processing and calculation

[0372] The server receives data on the on-site situation and perceived issues sent by the user and passes it to the generation AI. The generation AI generates educational training content based on this data. The generated content is sent from the server to the VR goggles and displayed to the user. The emotion engine recognizes the user's emotions and feeds that information back to the generation AI. The generation AI regenerates educational training content as necessary based on the fed-back emotion information.

[0373] Examples

[0374] For example, consider a case where an employee needs to learn how to operate a new machine on a manufacturing line. The user opens a dedicated application on their device and inputs "how to operate a new machine on the manufacturing line" as the on-site situation. They also input "it's difficult to operate" as the sense of challenge. The server receives these data and passes them to the generation AI. The generation AI generates educational training content including detailed operating procedures and safety measures. The generated content is sent to the VR goggles and displayed to the user. When the user puts on the VR goggles and watches the content, the emotion engine recognizes the user's confused facial expression and feeds that information back to the generation AI. The generation AI regenerates the content with more detailed explanations and concrete examples and sends it back to the VR goggles.

[0375] Examples of prompt statements

[0376] Below are examples of prompt sentences that the generative AI uses when generating educational and training content.

[0377] If you have confused employees who need to learn how to operate a new machine on your production line, generate educational training content with detailed explanations and concrete examples.

[0378] In this way, a system is realized in which the server, terminals, and users work together to provide personalized educational and training content that is tailored to the on-site situation, sense of issues, and even the user's emotions.

[0379] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0380] Step 1:

[0381] The user inputs the situation at the site and the issues that need addressing.

[0382] The user opens a dedicated application on the terminal and inputs "How to operate a new machine on the manufacturing line" as the on-site situation. Also, inputs "It's difficult to operate" as the problem. These input data are sent to the server.

[0383] Step 2:

[0384] The server receives the input data and passes it to the generation AI.

[0385] The server receives data on the site situation and issues sent by the user. The received data is passed to the generation AI. At this time, the data format is converted into a format that the generation AI can easily process.

[0386] Step 3:

[0387] Generative AI generates educational training content.

[0388] The generation AI generates educational and training content based on the received data on the site situation and issues. For example, it generates content that includes "detailed instructions and safety measures on how to operate a new machine." The generated content is returned to the server.

[0389] Step 4:

[0390] The server sends the generated content to the VR goggles.

[0391] The server receives the educational and training content returned by the generation AI and transmits it to the VR goggles, where it is stored in the internal memory of the VR goggles.

[0392] Step 5:

[0393] The user puts on the VR goggles and watches the educational training content.

[0394] The user puts on the VR goggles and watches the saved educational and training content. While watching, the user can learn how to operate a new machine through 3D images.

[0395] Step 6:

[0396] The emotion engine recognizes the user's emotions and provides feedback to the generative AI.

[0397] The emotion engine recognizes emotions from the user's facial expressions, tone of voice, choice of words, etc. For example, if the user is confused, it will feed that information back to the generating AI.

[0398] Step 7:

[0399] Generative AI will recreate content based on emotion information (if necessary).

[0400] The AI ​​regenerates the educational and training content as necessary based on the feedback of emotional information. For example, if the user is confused, it generates content that includes more detailed explanations and concrete examples. The regenerated content is sent again via the server to the VR goggles and displayed to the user.

[0401] (Application example 2)

[0402] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0403] There is a need to collect information on the work situation at the site and the workers' sense of issues in real time, generate optimal education and training content based on that information, and provide it to workers effectively. However, with conventional systems, it was difficult to provide personalized education and training content that fully considered the situation at the site and the feelings of the workers. This resulted in problems such as a decrease in the workers' level of understanding and work efficiency.

[0404] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0405] In this invention, the server includes a generating AI means for generating educational training content that takes into account the on-site situation and the subject employee's sense of problem, a VR goggle means for displaying the educational training content generated by the generating AI, a data collection means for collecting the on-site work situation and the subject employee's sense of problem in real time, a content generation means for the generating AI to generate optimal educational training content based on the collected data, and a display means for displaying the generated content to the worker through the VR goggles. This makes it possible to provide personalized educational training content that takes into account the on-site situation and the worker's feelings.

[0406] "Site status" refers to information such as the current progress of work at the work site and the operating status of machines.

[0407] "The target employees' sense of challenge" refers to the workers' subjective assessment of the difficulty of the work, the need for safety measures, etc.

[0408] "Generative AI means" refers to the function of artificial intelligence that takes on-site conditions and the target employees' sense of challenges as input and generates educational training content based on that.

[0409] "VR goggle means" refers to virtual reality goggles for displaying educational and training content generated by the generation AI.

[0410] "Data collection means" refers to devices such as sensors and cameras used to collect information on on-site work conditions and workers' sense of challenges in real time.

[0411] "Content generation means" refers to the function whereby the generative AI generates optimal educational and training content based on collected data.

[0412] "Display means" refers to the function for displaying the generated educational and training content to workers through VR goggles.

[0413] In order to implement this invention, the following system configuration is used: The server includes a generating AI means for generating educational training content that takes into account the on-site situation and the sense of issues of the target employees, a VR goggle means for displaying the educational training content generated by the generating AI, a data collection means for collecting on-site work conditions and the sense of issues of the workers in real time, a content generation means for the generating AI to generate optimal educational training content based on the collected data, and a display means for displaying the generated content to the workers through the VR goggles.

[0414] Hardware and Software Used

[0415] Hardware:

[0416] Camera: Used to collect on-site information.

[0417] VR goggles: Used to display the generated educational and training content.

[0418] Sensors: Used to collect real-time information on work conditions and worker challenges.

[0419] software:

[0420] OpenCV: An image processing library used to collect scene conditions.

[0421] EmotionRecognizer: An emotion engine used to recognize the emotions of workers.

[0422] ContentGenerator: Generative AI. Used to generate educational and training content.

[0423] VRDisplay: Used to display content on VR goggles.

[0424] Data processing and calculation

[0425] The server first collects information from cameras and sensors about the current situation on the site. For example, the camera captures the progress of the production line and the operating status of the machines, and processes the images using OpenCV. Next, the server uses EmotionRecognizer to recognize the emotions of the workers from their facial expressions and tone of voice, and collects their sense of the issues.

[0426] The collected data is input into the ContentGenerator, and the generative AI generates optimal educational training content. For example, if a worker is confused, the generative AI takes that emotion into account and generates educational training content with more detailed explanations and concrete examples. The generated content is displayed on VR goggles through the VRDisplay, providing feedback and educational training to workers in real time.

[0427] Examples

[0428] For example, when a worker is having trouble assembling a part on a production line, the camera and sensor will collect the situation, and the EmotionRecognizer will recognize the worker's confusion. Based on this, the ContentGenerator will generate a 3D model and simulation that explains the part installation procedure in detail, and display it to the worker through the VR goggles.

[0429] Examples of prompt statements

[0430] Scene: Parts assembly on a production line. A worker is having trouble fitting the part.

[0431] Generated Content: Generate 3D models and simulations that detail part installation steps.

[0432] In this way, it becomes possible to provide personalized educational and training content that takes into account the situation on-site and the emotions of workers.

[0433] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0434] Step 1:

[0435] The server collects on-site conditions using cameras and sensors. Specifically, the camera captures the progress of the production line and the operating status of the machines, and obtains data from the sensors. The input is camera images and sensor data, and the output is data on the on-site conditions. OpenCV is used to process the images and analyze the on-site conditions.

[0436] Step 2:

[0437] The server uses EmotionRecognizer to recognize the emotions of the workers. Specifically, it analyzes the facial expressions and tone of voice of the workers to estimate emotions such as confusion and excitement. The input is the facial expression and voice data of the workers, and the output is emotion data. Emotions are analyzed using EmotionRecognizer, and the worker's sense of the challenges is collected.

[0438] Step 3:

[0439] The server uses ContentGenerator to generate optimal education and training content based on the collected site situation data and emotion data. Specifically, it generates education and training content including detailed explanations and concrete examples, taking into account the site situation and the emotions of the workers. The input is site situation data and emotion data, and the output is education and training content. The data is analyzed using ContentGenerator to generate optimal content.

[0440] Step 4:

[0441] The server displays the generated educational and training content on the VR goggles via the VRDisplay. Specifically, the server transmits the generated 3D models and simulations to the VR goggles and provides real-time feedback and educational training to workers. The input is the educational and training content, and the output is the content displayed on the VR goggles. The content is displayed using the VRDisplay and provided to workers.

[0442] Step 5:

[0443] Users check the educational training content displayed through the VR goggles and learn work procedures and safety measures. Specifically, they put on the VR goggles and understand the work procedures through the displayed 3D models and simulations. The input is the content displayed on the VR goggles, and the output is the user's improvement in understanding and work efficiency. Users use the VR goggles to learn and then apply what they have learned to their actual work.

[0444] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing.

[0445] The microphone 38B acquires a voice indicating a user input for a result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0446] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0447] Another example of generative AI is Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) are mentioned.

[0448] In the above embodiment, an example was given in which the specific process was performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0449] [Second embodiment]

[0450] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0451] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0452] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0453] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0454] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0455] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0456] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0457] Fig. 4 shows an example of main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0458] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0459] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0460] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0461] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0462] "Example 1"

[0463] As one embodiment of the present invention, there is a system that provides a generation AI that receives as input the on-site situation and the target employee's sense of issues, and VR goggles that display the educational training content generated by the generation AI. The generation AI generates educational training content that takes into account the on-site situation and the target employee's sense of issues. Specifically, the on-site situation may be, for example, the work situation on the production line or the operation status of the railway, and the target employee's sense of issues may be, for example, the difficulty of the work or the need for safety measures, as input. Then, the educational training content is generated based on this information. The generated educational training content is displayed to the target employee through the VR goggles, and supports the training of the target employee.

[0464] "Example 2"

[0465] As one embodiment of the present invention, there is a system that provides a generation AI that receives as input the on-site situation and the target employee's sense of issues, and VR goggles that display the educational training content generated by the generation AI. The generation AI generates educational training content that takes into account the on-site situation and the target employee's sense of issues. Specifically, the on-site situation may be, for example, the work situation on the production line or the operation status of the railway, and the target employee's sense of issues may be, for example, the difficulty of the work or the need for safety measures, as input. Then, the educational training content is generated based on this information. The generated educational training content is displayed to the target employee through the VR goggles, and supports the training of the target employee.

[0466] The process flow of each embodiment will be described below.

[0467] "Example 1"

[0468] Step 1: The generation AI receives as input the on-site situation and the target employee's perception of the problem. For example, the on-site situation may be the work situation on the production line or the train operation status, while for example, the target employee's perception of the problem may be the difficulty of the work or the need for safety measures.

[0469] Step 2: The AI ​​generates training content based on the received on-site situation and the employee's sense of challenge. For example, if the production line is congested, the AI ​​generates training content that includes work procedures and precautions to deal with the congestion.

[0470] Step 3: The generated educational training content is displayed to the target employees through VR goggles. The target employees experience the educational training content through the VR goggles and receive the training.

[0471] Example 1

[0472] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0473] With conventional education and training systems, it was difficult to provide content that fully reflected the specific situation at the workplace and the individual issues faced by the target employees. As a result, the effectiveness of the training was limited, and it was difficult to acquire skills and knowledge that were suited to actual work. In addition, there was also the problem that it took a lot of time and effort to generate and distribute training content.

[0474] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0475] In this invention, the server includes a generating AI means for generating educational training content that takes into account the on-site situation and the target employee's sense of issues, a visual display device means for displaying the educational training content generated by the generating AI, an input device means for inputting the on-site situation and the target employee's sense of issues, and a communication means for transmitting the generated educational training content to the visual display device. This makes it possible to quickly and efficiently generate and display educational training content that reflects the specific on-site situation and the individual sense of issues of the target employee.

[0476] "On-site conditions" refer to the specific states and conditions in a particular business environment, such as the working conditions on a manufacturing line or the operation status of a train.

[0477] "The target employees' sense of challenge" refers to the work-related challenges and problems that specific employees feel, such as the difficulty of the work or the need for safety measures.

[0478] "Educational and training content" refers to learning materials and training programs aimed at improving employee skills and acquiring knowledge.

[0479] "Generative AI" refers to artificial intelligence that automatically generates educational and training content based on on-site conditions and the issues perceived by target employees.

[0480] A "visual display device" is a device for displaying the generated educational and training content, specifically referring to VR goggles and the like.

[0481] An "input device" refers to a device that provides an interface for users to input on-site conditions and the issues perceived by target employees.

[0482] "Communication means" refers to the technology or protocols used to transmit the generated educational and training content from the server to the visual display device.

[0483] The present invention is a system for generating educational and training content based on the situation at the workplace and the sense of issues faced by target employees, and displaying the content on a visual display device. A specific embodiment of this system will be described below.

[0484] server

[0485] The server hosts the generative AI model and receives the on-site situation and the target employees' sense of issues as input. The generative AI model generates educational training content based on this input data. Specifically, the server performs the following processes:

[0486] 1. Data Reception

[0487] The server receives the on-site situation and the employee's sense of challenge sent from the terminal, such as data on the work situation on the manufacturing line, the operation status of trains, the difficulty of the work, and the need for safety measures.

[0488] 2. Data Analysis

[0489] The server analyzes the received data and converts it into a format suitable for the generative AI model, for example by converting text data into numerical data and extracting necessary features.

[0490] 3. Content Creation

[0491] The server generates educational and training content using a generative AI model, which creates optimal educational and training content based on the on-site situation and perceived issues.

[0492] 4. Content submission

[0493] The server transmits the generated educational / training content to the terminal, specifically, by converting it into a data format suitable for display on a visual display device.

[0494] Terminal

[0495] The terminal provides an interface for users to input information about the on-site situation and the issues the target employees face. The data entered by the user is sent from the terminal to the server. After the generation AI on the server generates the educational training content, the terminal transfers the content to the visual display device. The specific operation is as follows:

[0496] 1. Data Entry

[0497] The terminal provides an interface for the user to input information about the situation at the site and the issues faced by the target employees. For example, data is entered using text boxes and drop-down menus.

[0498] 2. Data transmission

[0499] The terminal transmits the input data to the server. Specifically, the terminal transmits the data by using an HTTP request.

[0500] 3. Receiving Content

[0501] The terminal receives the educational and training content transmitted from the server and prepares the received data for transfer to the visual display device.

[0502] 4. Content Display

[0503] The terminal transfers the received educational / training content to the visual display device for display, specifically, by transmitting data for displaying the content on the display of the visual display device.

[0504] User

[0505] The user uses the terminal to input the on-site situation and the issues the target employees feel. For example, they can input information such as "The speed of Line A is too fast" or "New employees are not used to the work." The input data is sent to the server, and the generation AI generates educational and training content. The generated content is displayed to the user through a visual display device. The specific operation is as follows:

[0506] 1. Data Entry

[0507] The user uses the terminal to input information about the on-site situation and issues, such as "The speed of Line A is too fast" or "New employees are not used to the work."

[0508] 2. Data verification

[0509] The user reviews the data they entered to ensure it is accurate and, if necessary, corrects it.

[0510] 3. Training

[0511] The user wears the visual display device and receives the generated educational and training content. Specifically, the user views the content displayed on the display of the visual display device and performs operations according to the instructions.

[0512] Examples

[0513] Example 1

[0514] The user inputs the work situation of the production line as "Line A is running too fast" and the employee's sense of challenge as "New employees are not used to the work." The generation AI on the server uses this information to generate educational training content for new employees. The generated content is shown to new employees through a visual display device, helping them with training on work procedures and safety measures.

[0515] Example 2

[0516] The user inputs the train operation status as "frequent delays" and the target employee's sense of the problem as "efficiency in traffic management is necessary." The generation AI uses this information to generate educational and training content on the efficiency of traffic management. The generated content is shown to traffic management personnel via a visual display device, supporting training on efficient traffic management methods.

[0517] Examples of prompt statements

[0518] "Please enter the operation status of the production line. Example: Line A is running fast."

[0519] "Please enter the problem that the target employee feels. Example: New employees are not familiar with the work."

[0520] "Please enter the train operation status. Example: Frequent delays."

[0521] "Please enter the issues the target employees face. Example: Traffic management needs to be made more efficient."

[0522] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0523] Step 1:

[0524] Data entry (terminal)

[0525] The terminal provides an interface for the user to input the situation at the site and the perceived issues of the target employees. For example, the user inputs information such as "The speed of Line A is fast" or "New employees are not familiar with the work." The input data is saved in the terminal via text boxes and drop-down menus. The input data is text information about the situation at the site and perceived issues.

[0526] Step 2:

[0527] Data transmission (terminal)

[0528] The terminal transmits the input data on the field situation and the sense of issues to the server. Specifically, the data is transmitted using an HTTP request. The transmitted data is in a structured data format such as JSON. The input data is the text information transmitted to the server, and the output is the data transmitted to the server.

[0529] Step 3:

[0530] Data reception (server)

[0531] The server receives data on the field situation and issues sent from the terminal. The received data is analyzed on the server. The input data is the text information sent from the terminal, and the output is the data ready for analysis.

[0532] Step 4:

[0533] Data analysis (server)

[0534] The server analyzes the received data and converts it into a format suitable for the generative AI model. For example, it converts text data into numerical data and extracts the necessary features. The input data is the received text information, and the output is numerical data to be input into the generative AI model.

[0535] Step 5:

[0536] Content generation (server)

[0537] The server generates educational training content using a generative AI model. The generative AI model creates optimal educational training content based on the on-site situation and sense of issues. The input data is analyzed numerical data, and the output is the generated educational training content.

[0538] Step 6:

[0539] Content transmission (server)

[0540] The server transmits the generated educational / training content to the terminal. Specifically, it converts the content into a data format suitable for display on a visual display device and transmits it. The input data is the generated educational / training content, and the output is the content data transmitted to the terminal.

[0541] Step 7:

[0542] Content reception (terminal)

[0543] The terminal receives the educational and training content transmitted from the server. The terminal prepares the received data for transfer to the visual display device. The input data is the content data transmitted from the server, and the output is the data ready for transfer to the visual display device.

[0544] Step 8:

[0545] Content display (terminal)

[0546] The terminal transfers the received educational and training content to the visual display device for display. Specifically, the terminal transmits data for displaying the content on the display of the visual display device. The input data is the data ready to be transferred to the visual display device, and the output is the educational and training content displayed on the visual display device.

[0547] Step 9:

[0548] Attending training (user)

[0549] The user wears the visual display device and takes the generated educational and training content. Specifically, the user watches the content displayed on the display of the visual display device and performs operations according to the instructions. The input data is the educational and training content displayed on the visual display device, and the output is the user's progress in taking the training.

[0550] (Application example 1)

[0551] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0552] In order to effectively provide educational and training content based on the situation at the workplace and the sense of issues of the target employees, a means of generating and displaying appropriate information in real time is required. However, with conventional systems, it was difficult to provide educational and training content that fully reflected the situation at the workplace and the sense of issues of each employee, and the effectiveness of the training was limited. In addition, because the means of display were limited, it was not possible to provide a flexible training environment.

[0553] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0554] In this invention, the server includes a generating AI means for generating educational training content that takes into account the on-site situation and the sense of issues of the target employee, a VR goggle means for displaying the educational training content generated by the generating AI, and a smart glasses means for displaying the educational training content generated by the generating AI. This makes it possible to generate customized educational training content based on the on-site situation and the sense of issues of each employee in real time and provide it through a flexible display means.

[0555] "Site conditions" refers to the current work progress and environmental conditions at the work site.

[0556] "Target Employees" refers to employees who are intended to receive the educational and training content.

[0557] "Sense of challenge" refers to the concerns felt by the employees, such as the difficulty of the work and the need for safety measures.

[0558] "Educational and training content" refers to educational materials and training programs aimed at improving the skills and knowledge of target employees.

[0559] "Generative AI" refers to artificial intelligence that takes on-site conditions and the target employees' sense of challenges as input and generates educational training content based on that.

[0560] "VR goggles" refers to a head-mounted display for displaying a virtual reality environment.

[0561] "Smart glasses" refers to a glasses-type device that has augmented reality and information display functions.

[0562] The system for carrying out the present invention includes a generating AI means for generating educational training content that takes into account the on-site situation and the sense of issues of the target employees, and a VR goggle means and a smart glasses means for displaying the educational training content generated by the generating AI. A specific embodiment of this system will be described below.

[0563] System Programming

[0564] The server receives the on-site situation and the target employee's sense of challenge as input, and runs a generation AI that generates educational training content based on that. The generation AI uses OpenAI's API to send prompts and generate educational training content. The generated content is displayed on the VR goggles and smart glasses.

[0565] Processing Description

[0566] The server first obtains the on-site situation and the target employee's sense of challenge as input data. This input data includes, for example, the working situation on the production line, the difficulty of the work, and the need for safety measures. Next, the server sends a prompt to the generation AI to generate education and training content. This prompt specifically describes the on-site situation and the sense of challenge. The generated education and training content is sent to the VR goggles and smart glasses and displayed to the target employee.

[0567] Hardware and Software Used

[0568] Hardware: Servers, VR goggles, smart glasses

[0569] Software: OpenAI API, Python

[0570] Examples

[0571] For example, if the work situation on a production line is "assembly of parts" and the worker's sense of challenge is "installation of parts is difficult," the generative AI will generate educational and training content including "detailed steps on how to install parts" and "safety measures." The generated content is displayed to the worker through VR goggles or smart glasses.

[0572] Examples of prompt statements

[0573] Site situation: Parts assembly work

[0574] Problem: Difficult to install parts

[0575] Generate educational training content based on this.

[0576] In this way, the server can generate customized educational and training content in real time based on the situation at the workplace and the issues each employee faces, and provide it through flexible display means.

[0577] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0578] Step 1:

[0579] The server receives input data on the on-site situation and the employees' perceptions of the issues. Specifically, it collects information such as the work situation on the production line, the difficulty of the work, and the need for safety measures through sensors and user input. The input data is sent to the server in JSON format.

[0580] Step 2:

[0581] The server generates a prompt sentence based on the acquired on-site situation and problem perception. Specifically, it converts the on-site situation and problem perception into text format and creates a prompt sentence to send to the generation AI. For example, the format might be "On-site situation: Parts assembly work, Problem perception: Parts installation is difficult."

[0582] Step 3:

[0583] The server sends the prompt text to the generation AI and generates the educational training content. Specifically, the server uses the OpenAI API to send the prompt text and receives the generated text data. The generation AI generates appropriate educational training content based on the input prompt text.

[0584] Step 4:

[0585] The server transmits the generated educational and training content to the VR goggles and smart glasses. Specifically, the server converts the generated text data into a display format for the VR goggles and smart glasses and transmits it to the device. The device displays the received content.

[0586] Step 5:

[0587] Users visually check the educational training content through VR goggles or smart glasses. Specifically, they read the content displayed on the device's display and receive the training. Users perform tasks based on the displayed information and acquire the necessary skills and knowledge.

[0588] Example 2

[0589] Next, a description will be given of Example 2 of the embodiment 2. In the following description, the data processing device 12 is referred to as a "server" and the smart glasses 214 are referred to as a "terminal".

[0590] There is a need to efficiently generate educational and training content that takes into account the situation at the workplace and the issues faced by the target employees, and to provide effective training to the target employees. However, with conventional systems, it was difficult to generate content that appropriately reflected the situation at the workplace and the issues faced, and there were also limited means to effectively display the generated content. This made it difficult to provide training that met the specific situation at the workplace and the needs of each individual employee.

[0591] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0592] In this invention, the server includes an interface means for the user to input the on-site situation and the sense of issues of the target employee, a means for the server to receive the input data and generate a prompt sentence for the generative AI model, and a means for the server to transmit the generated education and training content to the terminal. This makes it possible to efficiently generate education and training content that reflects the on-site situation and the sense of issues, and to provide effective training to the target employee through the VR goggles.

[0593] "On-site conditions" refer to the specific states and conditions in a particular business environment, such as the working conditions on a manufacturing line or the operation status of a train.

[0594] "Target Employees" refers to the specific employees intended to receive the educational and training content.

[0595] "Sense of challenge" refers to specific problems or issues in the work, such as the difficulty of the work or the need for safety measures as perceived by the employees.

[0596] "Educational and training content" refers to content that is created based on on-site conditions and issues, and is intended for the education and training of target employees.

[0597] "Generative AI means" refers to the function of artificial intelligence that receives on-site conditions and sense of issues as input and generates educational training content based on that.

[0598] "VR goggles means" refers to a device that displays the educational and training content generated by the generating AI means and provides training in a virtual reality environment to target employees.

[0599] "Interface means" refers to the input devices and software of the system that allow the user to input the on-site situation and the target employees' sense of issues.

[0600] A "prompt sentence" is an instruction sentence to be input into a generative AI model, and refers to text that specifically describes the on-site situation and the sense of the problem.

[0601] "Terminal" refers to a device that receives educational and training content sent from the server and displays it on the VR goggles.

[0602] This invention is a system that generates educational training content based on the on-site situation and the employee's sense of issues, and displays it to the employee through VR goggles. A specific embodiment of this system will be described below.

[0603] The user inputs the on-site situation and the employee's perception of the issues through the system interface. For example, the speed of the production line, the difficulty of the work, the need for safety measures, etc. The interface can be a web browser or dedicated application software.

[0604] The server receives the on-site situation and sense of the problem input by the user, and generates a prompt sentence to be input into the generative AI model based on this information. An example of a generative AI model is OpenAI's GPT-4. An example of a prompt sentence is, "The speed of the manufacturing line is so fast that new employees are having trouble remembering the work procedures. Please generate educational training content for this."

[0605] The server inputs the generated prompt sentence into the generative AI model, and the generative AI model generates educational training content based on the prompt sentence. The generated educational training content is, for example, VR content that provides detailed explanations of work procedures on a manufacturing line.

[0606] The server transmits the generated educational and training content to the terminal. The terminal is a device (e.g., a PC or a smartphone) that is linked to the VR goggles. The terminal displays the received educational and training content on the VR goggles.

[0607] Users (target employees) wear VR goggles and receive training through the educational and training content displayed. For example, they can learn the work procedures of a manufacturing line while experiencing them in a virtual environment. This provides a realistic training experience that reflects the actual situation and challenges on the ground.

[0608] This system makes it possible to efficiently generate educational and training content that reflects on-site conditions and issues, and to provide effective training to target employees through VR goggles.

[0609] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0610] Step 1:

[0611] The user inputs the situation on-site and the target employee's perception of the issues.

[0612] Through the system's interface, users input information about the on-site situation and the issues the target employees face. For example, they input information such as the speed of the production line, the difficulty of the work, and the need for safety measures. The input data is sent to the server. An example of input data is, "The speed of the production line is so fast that new employees are having trouble learning the work procedures."

[0613] Step 2:

[0614] The server receives the input data and generates a prompt for the generative AI model.

[0615] The server receives data on the on-site situation and problem perception sent by the user. Next, based on this information, it generates a prompt sentence to be input to the generative AI model. Specifically, the server analyzes the input data and generates a prompt sentence such as, "The speed of the manufacturing line is fast, and new employees are having difficulty learning the work procedures. Please generate educational training content for this." The generated prompt sentence is sent to the generative AI model.

[0616] Step 3:

[0617] The server inputs prompt sentences into the generative AI model to generate educational training content.

[0618] The server inputs the generated prompt sentence into a generative AI model (e.g., OpenAI's GPT-4). The generative AI model generates educational training content based on the prompt sentence. Specifically, the generative AI model analyzes the prompt sentence and generates VR content that explains the work procedures of the manufacturing line in detail. The generated educational training content is returned to the server.

[0619] Step 4:

[0620] The server transmits the generated educational / training content to the terminal.

[0621] The server receives the educational training content generated from the generative AI model and transmits it to the terminal. The terminal is a device (e.g., a PC or a smartphone) that is linked to the VR goggles. The transmitted educational training content is stored in the terminal.

[0622] Step 5:

[0623] The terminal displays educational training content through VR goggles.

[0624] The terminal displays the received educational and training content on the VR goggles. Specifically, the terminal transmits the content to the VR goggles, and the VR goggles provide realistic images and sounds to the target employees.

[0625] Step 6:

[0626] Users (target employees) receive training using VR goggles.

[0627] Users (target employees) wear VR goggles and receive training through the educational and training content displayed. For example, they can learn the work procedures of a manufacturing line while experiencing them in a virtual environment. This provides a realistic training experience that reflects the actual situation and challenges on the ground.

[0628] (Application example 2)

[0629] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the smart glasses 214 will be referred to as a "terminal".

[0630] There is a need to efficiently generate educational and training content based on the on-site work situation and the workers' sense of issues, and provide it to workers at the appropriate time. However, with conventional systems, it was difficult to quickly generate training content that reflected the real-time situation at the site, and it was also difficult to provide content that responded to the individual sense of issues of workers. This resulted in problems such as insufficient improvement of workers' skills and thorough implementation of safety measures.

[0631] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0632] In this invention, the server includes a generating AI means for generating educational training content that takes into account the on-site situation and the target employee's sense of problem, a VR goggle means for displaying the educational training content generated by the generating AI, a data collection means for collecting data from sensors in the factory, an input means for the worker to input the sense of problem, and a transmission means for transmitting the generated educational training content to the VR goggles. This makes it possible to quickly generate and provide educational training content based on the real-time situation of the on-site situation and the individual sense of problem of the worker.

[0633] "Site conditions" refers to the real-time work situation and environmental conditions in a factory or work site.

[0634] "Target employees' sense of issues" refers to individual issues and concerns felt by workers, such as the difficulty of the work and the need for safety measures.

[0635] "Educational and training content" refers to information and teaching materials for education and training aimed at improving worker skills and ensuring thorough safety measures.

[0636] "Generative AI means" refers to the function of artificial intelligence that takes on-site conditions and the target employees' sense of challenges as input and generates educational training content based on that.

[0637] "VR goggle means" refers to virtual reality goggles for displaying the educational and training content generated by the generation AI to workers.

[0638] "Data collection means" refers to the function of collecting information on on-site work conditions in real time using sensors and cameras within the factory.

[0639] "Input means" refers to devices such as smartphones or tablets that workers use to input their own perceptions of issues.

[0640] "Transmission means" refers to the communication function for transmitting the generated educational and training content to the VR goggles.

[0641] A system for implementing this invention includes a generation AI means for generating educational and training content that takes into account the on-site situation and the target employee's sense of challenges, a VR goggles means for displaying the educational and training content generated by the generation AI, a data collection means for collecting data from sensors within the factory, an input means for workers to input their sense of challenges, and a transmission means for transmitting the generated educational and training content to the VR goggles.

[0642] First, the server collects information on the work situation in the factory in real time using sensors and cameras. This makes it possible to grasp the real-time situation at the factory. Next, workers use their smartphones or tablets to input their own sense of the issues. This sense of the issues includes the difficulty of the work and the need for safety measures.

[0643] The collected information on the site conditions and the workers' sense of challenges are input into the generation AI. Based on this data, the generation AI generates optimal educational and training content. This generation AI uses a programming language such as Python. The generated educational and training content is sent to the VR goggles via a transmission means.

[0644] The VR goggles display the generated educational and training content to the workers, allowing them to receive educational and training based on the real-time situation at the site and their own sense of issues. This allows workers to efficiently improve their skills and ensure safety measures.

[0645] As a concrete example, consider a situation where the robot on the manufacturing line is operating normally, but the worker is feeling uneasy about the new operating procedure. In this case, the worker inputs the sense of challenge that the new operating procedure is difficult. Based on this information, the generative AI generates educational training content that makes it easy for the worker to understand the new operating procedure.

[0646] Examples of prompts for a generative AI model include:

[0647] Site situation: The robots on the manufacturing line are working normally.

[0648] Challenges: Workers find the new operating procedures difficult.

[0649] Use this information to generate educational training content that helps workers understand new operating procedures.

[0650] In this way, it becomes possible to quickly generate and provide educational and training content based on the real-time situation on-site and the individual challenges faced by workers.

[0651] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0652] Step 1:

[0653] The server collects information on the status of on-site work in real time using sensors and cameras in the factory. It receives data from the sensors and cameras as input and generates data indicating the status of on-site work as output. This data includes, for example, the operating status of the manufacturing line and the operating status of robots.

[0654] Step 2:

[0655] The user inputs their own sense of the problem using a smartphone or tablet. As input, the system receives information such as the difficulty of the task as perceived by the user and the need for safety measures, and as output, generates data indicating the sense of the problem. This data includes specific sense of the problem, such as "the new operating procedure is difficult."

[0656] Step 3:

[0657] The server inputs the collected on-site situation data and the user's problem perception data into the generation AI. It receives the on-site situation data and problem perception data as input, and generates educational training content as output. This generation AI is implemented using a programming language such as Python, and generates optimal educational training content based on the input data.

[0658] Step 4:

[0659] The server transmits the generated educational / training content to the VR goggles through a transmission means. As an input, the server receives the generated educational / training content, and as an output, generates data to be transmitted to the VR goggles. For this transmission means, a communication technology such as Wi-Fi or Bluetooth is used.

[0660] Step 5:

[0661] The terminal (VR goggles) displays the received educational and training content to the user. As input, it receives the transmitted educational and training content, and as output, it generates images and audio to be displayed to the user. This allows the user to receive educational and training based on the real-time situation in the field and their own sense of issues.

[0662] In this way, it becomes possible to rapidly generate and provide educational and training content based on the real-time situation on-site and the user's individual sense of the issues.

[0663] In addition, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0664] "Example 1"

[0665] In one embodiment of the present invention, the generation AI includes an emotion engine. The emotion engine recognizes the user's emotion and uses the emotion as an input for the generation AI. Specifically, the emotion is estimated from the user's facial expression, tone of voice, choice of words, etc., and the result is provided to the generation AI. The generation AI generates educational training content by considering this emotion information together with the on-site situation and the target employee's sense of challenge. For example, if the user feels confused, the generation AI takes the emotion into account and generates educational training content including more detailed explanations and concrete examples. Also, if the user feels excited, the generation AI takes the emotion into account and generates educational training content including more advanced topics. In this way, the present invention combined with the emotion engine makes it possible to provide personalized educational training content according to the user's emotion.

[0666] "Example 2"

[0667] In one embodiment of the present invention, the generation AI includes an emotion engine. The emotion engine recognizes the user's emotion and uses the emotion as an input for the generation AI. Specifically, the emotion is estimated from the user's facial expression, tone of voice, choice of words, etc., and the result is provided to the generation AI. The generation AI generates educational training content by considering this emotion information together with the on-site situation and the target employee's sense of challenge. For example, if the user feels confused, the generation AI takes the emotion into account and generates educational training content including more detailed explanations and concrete examples. Also, if the user feels excited, the generation AI takes the emotion into account and generates educational training content including more advanced topics. In this way, the present invention combined with the emotion engine makes it possible to provide personalized educational training content according to the user's emotion.

[0668] The process flow of each embodiment will be described below.

[0669] "Example 1"

[0670] Step 1: The emotion engine recognizes the user's emotions. Specifically, it infers emotions from the user's facial expressions, tone of voice, choice of words, etc.

[0671] Step 2: The emotion engine provides the recognized emotion information to the generation AI.

[0672] Step 3: The generation AI considers the emotion information along with the on-site situation and the target employee's sense of challenge to generate educational training content. For example, if the user feels confused, the generation AI considers that emotion to generate educational training content that includes more detailed explanations and concrete examples. Also, if the user feels excited, the generation AI considers that emotion to generate educational training content that includes more advanced topics.

[0673] Step 4: The educational and training content generated by the generation AI is sent to the VR goggles so that it can be viewed by the target employees.

[0674] Example 1

[0675] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0676] With conventional education and training systems, it was difficult to generate content that took into account the on-site situation and the issues faced by the target employees, and it was also not possible to provide personalized training content that reflected the user's emotions. As a result, the effectiveness of the training was limited, and there were problems with the level of understanding and satisfaction of the target employees decreasing.

[0677] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0678] In this invention, the server includes a generating AI means for generating educational training content that takes into account the on-site situation and the target employee's sense of problem, a VR goggle means for displaying the educational training content generated by the generating AI, and an emotion engine means for recognizing the user's emotion and using the emotion as an input for the generating AI. This makes it possible to provide personalized educational training content that reflects the user's emotion in addition to the on-site situation and the target employee's sense of problem.

[0679] "On-site conditions" refer to a specific work environment or the progress of work, including, for example, the work conditions on a manufacturing line or the operation status of a railway.

[0680] "The target employee's sense of challenge" refers to the work-related difficulties and areas for improvement that a particular employee feels, including, for example, the difficulty of the work or the need for safety measures.

[0681] "Generative AI" refers to artificial intelligence that automatically generates educational and training content based on input data.

[0682] "VR goggles" refers to a device that uses virtual reality technology to visually display information and is used to display educational training content to target employees.

[0683] An "emotion engine" refers to technology that recognizes emotions from a user's facial expressions, tone of voice, choice of words, etc., and uses that emotional information as input for generative AI.

[0684] "Educational and training content" refers to educational materials and training programs aimed at improving the skills and knowledge of target employees.

[0685] "User" refers to the person who operates the system and inputs the on-site situation and problem perception.

[0686] This invention is a system that provides a generation AI that receives the on-site situation and the subject employee's sense of issues as input and generates educational training content based on that, and a VR goggle that displays that content. It also includes an emotion engine that recognizes the user's emotions and uses those emotions as input for the generation AI.

[0687] System Configuration

[0688] 1. Generation AI

[0689] The generative AI generates educational training content by integrating the on-site situation, the target employees' sense of issues, and the user's emotional information. The software used is a natural language processing library (e.g., spaCy, NLTK) or a machine learning framework (e.g., TensorFlow, PyTorch).

[0690] 2. VR goggles

[0691] The VR goggles display educational training content generated by generative AI to target employees, providing an interactive training experience.

[0692] 3. Emotion Engine

[0693] The emotion engine recognizes emotions from the user's facial expressions, tone of voice, choice of words, etc., and provides the results to the generative AI, which then makes it possible to generate personalized educational and training content that reflects the user's emotions.

[0694] System Operation

[0695] 1. Data Entry

[0696] The user inputs the situation at the site and the employee's perception of the problem. For example, the user inputs information such as "Work on the production line is delayed," "The work is difficult," and "New safety measures are needed" into an input form on the terminal.

[0697] The device transmits the user's facial expressions, tone of voice, choice of words, etc. to the emotion engine. The device's camera and microphone capture the user's facial expressions and voice and transmit the data to the emotion engine in real time.

[0698] 2. Data Processing

[0699] The emotion engine estimates the user's emotions and provides the results to the generation AI. The emotion engine estimates that the user is confused from their facial expression and sends that information to the generation AI.

[0700] The generative AI integrates the on-site situation, the target employees' sense of issues, and the user's emotional information to generate educational and training content.The generative AI generates educational and training content including detailed explanations and concrete examples based on information such as "production line work is delayed," "the work is difficult," "new safety measures are needed," and "users are confused."

[0701] 3. Content Display

[0702] The server transmits the generated educational training content to the VR goggles. The server transfers data via a network to transmit the educational training content received from the generation AI to the VR goggles.

[0703] The VR goggles display the educational training content to the target employees. The VR goggles display the received educational training content in the target employees' field of vision, providing an interactive training experience.

[0704] Examples and prompts

[0705] Examples

[0706] Situation on the ground: Work on the production line is slow.

[0707] Challenges: The work is difficult and new safety measures are needed.

[0708] User sentiment: Confused.

[0709] In this case, the generative AI generates educational and training content with detailed explanations and concrete examples, and displays it on the VR goggles.

[0710] Examples of prompt statements

[0711] Situation on the ground: Work on the production line is slow.

[0712] Challenges: The work is difficult and new safety measures are needed.

[0713] User sentiment: Confused.

[0714] Based on this, generate educational training content with detailed explanations and concrete examples.

[0715] In this way, the system provides personalized educational and training content that takes into account user input and sentiment.

[0716] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0717] Step 1:

[0718] The user inputs the situation on-site and the target employee's perception of the issues.

[0719] Input: On-site situation (e.g., work on the production line is delayed), target employee's sense of challenge (e.g., the work is difficult, new safety measures are needed)

[0720] Output: Data on the situation on the ground and issues

[0721] Specific operations: The user enters information such as "Work on the production line is delayed," "The work is difficult," and "New safety measures are needed" into an input form on the terminal.

[0722] Step 2:

[0723] The device transmits the user's facial expressions, tone of voice, choice of words, etc. to the emotion engine.

[0724] Input: User's facial expressions, tone of voice, and word choice

[0725] Output: Emotion data sent to the emotion engine

[0726] How it works: The device's camera and microphone capture the user's facial expressions and voice, and send that data to the emotion engine in real time.

[0727] Step 3:

[0728] The emotion engine estimates the user's emotions and provides the results to the generative AI.

[0729] Input: User's facial expressions, tone of voice, and word choice

[0730] Output: Estimated emotion data

[0731] Specific operation: The emotion engine infers that the user is confused from their facial expression and sends that information to the generation AI.

[0732] Step 4:

[0733] The generative AI integrates the on-site situation, the target employees' sense of issues, and the user's emotional information to generate educational training content.

[0734] Input: On-site situation, target employee's sense of issues, user's emotional information

[0735] Output: Educational and training content

[0736] Specific operation: Based on information such as "work on the production line is delayed," "the work is difficult," "new safety measures are needed," and "users are confused," the generative AI generates educational and training content including detailed explanations and concrete examples.

[0737] Step 5:

[0738] The server transmits the generated educational and training content to the VR goggles.

[0739] Input: Educational and training content

[0740] Output: Data sent to the VR goggles

[0741] Specific operation: The server transfers data via the network to send the educational training content received from the generation AI to the VR goggles.

[0742] Step 6:

[0743] VR goggles display educational training content to target employees.

[0744] Input: Educational and training content

[0745] Output: Educational training content displayed to target employees

[0746] Specific operation: The VR goggles display the received educational training content in the target employee's field of vision, providing an interactive training experience.

[0747] (Application example 1)

[0748] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0749] In conventional education and training systems, it was difficult to provide content that fully reflected the specific situation at the workplace and the individual issues felt by the target employees. In addition, there was a lack of means to provide personalized training content that took into account the feelings of the target employees, so the effectiveness of the training was limited. This made it difficult to implement efficient and effective education and training, especially at workplaces such as factories.

[0750] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0751] In this invention, the server includes a generating AI means for generating educational training content that takes into account the on-site situation and the target employee's sense of issues, a VR goggle means for displaying the educational training content generated by the generating AI, a sensor means for collecting on-site situations in real time, and an emotion engine means for recognizing the emotions of the target employee. This makes it possible to provide personalized educational training content that takes into account the specific on-site situation, the individual sense of issues of the target employee, and even emotions.

[0752] "Site conditions" refers to the progress of work and the state of the environment at a factory or work site.

[0753] "Target employees" refers to employees or workers who are intended to receive the educational and training content.

[0754] "Sense of challenge" refers to the individual issues and concerns felt by the employees, such as the difficulty of the work or the need for safety measures.

[0755] "Educational and training content" refers to educational materials and training programs aimed at improving the skills and knowledge of target employees.

[0756] "Generative AI" refers to artificial intelligence that takes on-site conditions and the target employees' sense of challenges as input and generates educational training content based on that.

[0757] "VR goggles" refers to a virtual reality device for displaying the generated educational training content to target employees.

[0758] A "sensor" refers to a device or equipment used to collect information on site conditions in real time.

[0759] An "emotion engine" refers to a system that recognizes emotions from the facial expressions and tone of voice of the target employee, and provides that information to the generating AI.

[0760] A system for implementing this invention includes a generation AI means for generating educational training content that takes into account the on-site situation and the target employee's sense of challenges, a VR goggle means for displaying the educational training content generated by the generation AI, a sensor means for collecting on-site conditions in real time, and an emotion engine means for recognizing the emotions of the target employee.

[0761] The server first uses sensors to collect real-time information about the on-site situation. Sensors include cameras and environmental sensors in the factory, and obtain data on the progress of work and the state of the environment.

[0762] Next, voice input and tablets are used to collect the employees' perceptions of the issues they face. The employees can input their own perceptions of the difficulty of the work and the need for safety measures.

[0763] In addition, the system uses an emotion engine to recognize the emotions of the target employee. The emotion engine uses a camera and microphone to analyze the facial expressions and tone of voice of the target employee and estimates their emotions.

[0764] All of this data is input into the Generative AI, which generates optimal educational and training content for the target employees based on the on-site situation, sense of issues, and emotional information. The generated content includes specific operating procedures and safety measures, and helps the target employees improve their skills and acquire knowledge.

[0765] The generated educational and training content is displayed to the target employees through VR goggles, which are virtual reality devices that allow the target employees to visually experience the generated content.

[0766] As a specific example, when learning how to operate a new machine in a factory, if an employee feels that "this operation is difficult," the generative AI will take into account that sense of challenge and the employee's confused expression and generate educational training content that includes more detailed operating procedures and safety measures.

[0767] An example of a prompt is as follows:

[0768] Field situation: learning how to operate new machinery

[0769] Challenge: The task is difficult

[0770] Emotion: Confused

[0771] Content to be generated: Educational and training content including detailed operating procedures and safety measures

[0772] In this way, the system is able to provide personalized educational and training content that takes into account the specific situation on-site, the individual challenges faced by the target employees, and even their emotions.

[0773] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0774] Step 1:

[0775] The server uses sensors to collect real-time information about the site conditions. Sensors include cameras and environmental sensors in the factory, and obtain data about the progress of work and the state of the environment. The input is real-time data from the sensors, and the output is data that indicates the site conditions. The server collects this data and passes it on to the next processing step.

[0776] Step 2:

[0777] The user uses voice input or a tablet to collect the target employees' perceptions of the issues. The target employees input information such as their perceptions of the difficulty of the work and the need for safety measures. The input is data on the target employees' perceptions of the issues, and the output is data indicating the perceptions of the issues. The server collects this data and passes it on to the next processing step.

[0778] Step 3:

[0779] The server uses an emotion engine to recognize the target employee's emotions. The emotion engine uses a camera and microphone to analyze the target employee's facial expressions and tone of voice to infer emotions. The input is data from the camera and microphone, and the output is data indicating emotions. The server collects this data and passes it to the next processing step.

[0780] Step 4:

[0781] The server inputs the collected information on the on-site situation, sense of issues, and emotions into the generation AI. Based on this data, the generation AI generates the optimal educational training content for the target employees. The input is the on-site situation, sense of issues, and emotional information, and the output is the educational training content. The generation AI analyzes the data and generates the appropriate content.

[0782] Step 5:

[0783] The server transmits the generated educational / training content to the VR goggles. The VR goggles display the generated content to the target employee. The input is the generated educational / training content, and the output is the content displayed on the VR goggles. The target employee wears the VR goggles and visually experiences the training content.

[0784] In this way, the system is able to provide personalized educational and training content that takes into account the specific situation on-site, the individual challenges faced by the target employees, and even their emotions.

[0785] Example 2

[0786] Next, a description will be given of Example 2 of the embodiment 2. In the following description, the data processing device 12 is referred to as a "server" and the smart glasses 214 are referred to as a "terminal".

[0787] In conventional education and training systems, it was difficult to provide content that fully reflected the on-site situation and the issues faced by the target employees, and it was not possible to provide personalized training content that responded to the user's emotions. As a result, the effectiveness of the training was limited, and there was a problem of a decrease in the understanding and satisfaction of the target employees.

[0788] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0789] In this invention, the server includes a generating AI means for generating educational training content that takes into account the on-site situation and the target employee's sense of problem, a VR goggle means for displaying the educational training content generated by the generating AI, and an emotion engine means for recognizing the user's emotion and using the emotion as an input for the generating AI. This makes it possible to provide personalized educational training content that takes into account the on-site situation, the target employee's sense of problem, and the user's emotion.

[0790] "On-site conditions" refers to the actual conditions in a specific business environment, such as the working conditions on a manufacturing line or the operation status of a railway.

[0791] "The target employees' sense of challenge" refers to the work-related challenges and problems that specific employees feel, such as the difficulty of the work or the need for safety measures.

[0792] "Generative AI" refers to artificial intelligence that generates educational and training content based on on-site conditions and the issues perceived by target employees.

[0793] "VR goggles" refers to a virtual reality device for visually displaying generated educational and training content.

[0794] An "emotion engine" is a system that recognizes emotions from a user's facial expressions, tone of voice, choice of words, etc., and provides that information to the generative AI.

[0795] "Educational and training content" refers to teaching materials and training materials that enable target employees to learn the knowledge and skills necessary for their jobs.

[0796] "User" refers to employees and participants who use the educational and training content.

[0797] "Server" refers to the computer system that manages and operates the entire system, including the generative AI and emotion engine.

[0798] The present invention is a system that generates educational training content based on the on-site situation and the sense of issues of the target employees, and provides personalized training according to the user's emotions. A specific embodiment of this system will be described below.

[0799] System Configuration

[0800] The system consists of the following main components:

[0801] 1. Generative AI means: Artificial intelligence that generates educational and training content based on on-site conditions and the issues faced by target employees.

[0802] 2. VR goggles means: a virtual reality device that visually displays the generated educational training content.

[0803] 3. Emotion engine means: A system that recognizes emotions from the user's facial expressions, tone of voice, choice of words, etc., and provides that information to the generation AI.

[0804] Hardware and Software Configuration

[0805] The server manages and operates the entire system, including the generation AI and the emotion engine. The generation AI receives the on-site situation and the target employees' sense of issues as input, and generates educational training content. The emotion engine recognizes the user's emotions and provides that information to the generation AI. The VR goggles display the generated educational training content to the user.

[0806] Data processing and calculation

[0807] The server receives data on the on-site situation and perceived issues sent by the user and passes it to the generation AI. The generation AI generates educational training content based on this data. The generated content is sent from the server to the VR goggles and displayed to the user. The emotion engine recognizes the user's emotions and feeds that information back to the generation AI. The generation AI regenerates educational training content as necessary based on the fed-back emotion information.

[0808] Examples

[0809] For example, consider a case where an employee needs to learn how to operate a new machine on a manufacturing line. The user opens a dedicated application on their device and inputs "how to operate a new machine on the manufacturing line" as the on-site situation. They also input "it's difficult to operate" as the sense of challenge. The server receives these data and passes them to the generation AI. The generation AI generates educational training content including detailed operating procedures and safety measures. The generated content is sent to the VR goggles and displayed to the user. When the user puts on the VR goggles and watches the content, the emotion engine recognizes the user's confused facial expression and feeds that information back to the generation AI. The generation AI regenerates the content with more detailed explanations and concrete examples and sends it back to the VR goggles.

[0810] Examples of prompt statements

[0811] Below are examples of prompt sentences that the generative AI uses when generating educational and training content.

[0812] If you have confused employees who need to learn how to operate a new machine on your production line, generate educational training content with detailed explanations and concrete examples.

[0813] In this way, a system is realized in which the server, terminals, and users work together to provide personalized educational and training content that is tailored to the on-site situation, sense of issues, and even the user's emotions.

[0814] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0815] Step 1:

[0816] The user inputs the situation at the site and the issues that need addressing.

[0817] The user opens a dedicated application on the terminal and inputs "How to operate a new machine on the manufacturing line" as the on-site situation. Also, inputs "It's difficult to operate" as the problem. These input data are sent to the server.

[0818] Step 2:

[0819] The server receives the input data and passes it to the generation AI.

[0820] The server receives data on the site situation and issues sent by the user. The received data is passed to the generation AI. At this time, the data format is converted into a format that the generation AI can easily process.

[0821] Step 3:

[0822] Generative AI generates educational training content.

[0823] The generation AI generates educational and training content based on the received data on the site situation and issues. For example, it generates content that includes "detailed instructions and safety measures on how to operate a new machine." The generated content is returned to the server.

[0824] Step 4:

[0825] The server sends the generated content to the VR goggles.

[0826] The server receives the educational and training content returned by the generation AI and transmits it to the VR goggles, where it is stored in the internal memory of the VR goggles.

[0827] Step 5:

[0828] The user puts on the VR goggles and watches the educational training content.

[0829] The user puts on the VR goggles and watches the saved educational and training content. While watching, the user can learn how to operate a new machine through 3D images.

[0830] Step 6:

[0831] The emotion engine recognizes the user's emotions and provides feedback to the generative AI.

[0832] The emotion engine recognizes emotions from the user's facial expressions, tone of voice, choice of words, etc. For example, if the user is confused, it will feed that information back to the generating AI.

[0833] Step 7:

[0834] Generative AI will recreate content based on emotion information (if necessary).

[0835] The AI ​​regenerates the educational and training content as necessary based on the feedback of emotional information. For example, if the user is confused, it generates content that includes more detailed explanations and concrete examples. The regenerated content is sent again via the server to the VR goggles and displayed to the user.

[0836] (Application example 2)

[0837] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the smart glasses 214 will be referred to as a "terminal".

[0838] There is a need to collect information on the work situation at the site and the workers' sense of issues in real time, generate optimal education and training content based on that information, and provide it to workers effectively. However, with conventional systems, it was difficult to provide personalized education and training content that fully considered the situation at the site and the feelings of the workers. This resulted in problems such as a decrease in the workers' level of understanding and work efficiency.

[0839] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0840] In this invention, the server includes a generating AI means for generating educational training content that takes into account the on-site situation and the subject employee's sense of problem, a VR goggle means for displaying the educational training content generated by the generating AI, a data collection means for collecting the on-site work situation and the subject employee's sense of problem in real time, a content generation means for the generating AI to generate optimal educational training content based on the collected data, and a display means for displaying the generated content to the worker through the VR goggles. This makes it possible to provide personalized educational training content that takes into account the on-site situation and the worker's feelings.

[0841] "Site status" refers to information such as the current progress of work at the work site and the operating status of machines.

[0842] "The target employees' sense of challenge" refers to the workers' subjective assessment of the difficulty of the work, the need for safety measures, etc.

[0843] "Generative AI means" refers to the function of artificial intelligence that takes on-site conditions and the target employees' sense of challenges as input and generates educational training content based on that.

[0844] "VR goggle means" refers to virtual reality goggles for displaying educational and training content generated by the generation AI.

[0845] "Data collection means" refers to devices such as sensors and cameras used to collect information on on-site work conditions and workers' sense of challenges in real time.

[0846] "Content generation means" refers to the function whereby the generative AI generates optimal educational and training content based on collected data.

[0847] "Display means" refers to the function for displaying the generated educational and training content to workers through VR goggles.

[0848] In order to implement this invention, the following system configuration is used: The server includes a generating AI means for generating educational training content that takes into account the on-site situation and the sense of issues of the target employees, a VR goggle means for displaying the educational training content generated by the generating AI, a data collection means for collecting on-site work conditions and the sense of issues of the workers in real time, a content generation means for the generating AI to generate optimal educational training content based on the collected data, and a display means for displaying the generated content to the workers through the VR goggles.

[0849] Hardware and Software Used

[0850] Hardware:

[0851] Camera: Used to collect on-site information.

[0852] VR goggles: Used to display the generated educational and training content.

[0853] Sensors: Used to collect real-time information on work conditions and worker challenges.

[0854] software:

[0855] OpenCV: An image processing library used to collect scene conditions.

[0856] EmotionRecognizer: An emotion engine used to recognize the emotions of workers.

[0857] ContentGenerator: Generative AI. Used to generate educational and training content.

[0858] VRDisplay: Used to display content on VR goggles.

[0859] Data processing and calculation

[0860] The server first collects information from cameras and sensors about the current situation on the site. For example, the camera captures the progress of the production line and the operating status of the machines, and processes the images using OpenCV. Next, the server uses EmotionRecognizer to recognize the emotions of the workers from their facial expressions and tone of voice, and collects their sense of the issues.

[0861] The collected data is input into the ContentGenerator, and the generative AI generates optimal educational training content. For example, if a worker is confused, the generative AI takes that emotion into account and generates educational training content with more detailed explanations and concrete examples. The generated content is displayed on VR goggles through the VRDisplay, providing feedback and educational training to workers in real time.

[0862] Examples

[0863] For example, when a worker is having trouble assembling a part on a production line, the camera and sensor will collect the situation, and the EmotionRecognizer will recognize the worker's confusion. Based on this, the ContentGenerator will generate a 3D model and simulation that explains the part installation procedure in detail, and display it to the worker through the VR goggles.

[0864] Examples of prompt statements

[0865] Scene: Parts assembly on a production line. A worker is having trouble fitting the part.

[0866] Generated Content: Generate 3D models and simulations that detail part installation steps.

[0867] In this way, it becomes possible to provide personalized educational and training content that takes into account the situation on-site and the emotions of workers.

[0868] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0869] Step 1:

[0870] The server collects on-site conditions using cameras and sensors. Specifically, the camera captures the progress of the production line and the operating status of the machines, and obtains data from the sensors. The input is camera images and sensor data, and the output is data on the on-site conditions. OpenCV is used to process the images and analyze the on-site conditions.

[0871] Step 2:

[0872] The server uses EmotionRecognizer to recognize the emotions of the workers. Specifically, it analyzes the facial expressions and tone of voice of the workers to estimate emotions such as confusion and excitement. The input is the facial expression and voice data of the workers, and the output is emotion data. Emotions are analyzed using EmotionRecognizer, and the worker's sense of the challenges is collected.

[0873] Step 3:

[0874] The server uses ContentGenerator to generate optimal education and training content based on the collected site situation data and emotion data. Specifically, it generates education and training content including detailed explanations and concrete examples, taking into account the site situation and the emotions of the workers. The input is site situation data and emotion data, and the output is education and training content. The data is analyzed using ContentGenerator to generate optimal content.

[0875] Step 4:

[0876] The server displays the generated educational and training content on the VR goggles via the VRDisplay. Specifically, the server transmits the generated 3D models and simulations to the VR goggles and provides real-time feedback and educational training to workers. The input is the educational and training content, and the output is the content displayed on the VR goggles. The content is displayed using the VRDisplay and provided to workers.

[0877] Step 5:

[0878] Users check the educational training content displayed through the VR goggles and learn work procedures and safety measures. Specifically, they put on the VR goggles and understand the work procedures through the displayed 3D models and simulations. The input is the content displayed on the VR goggles, and the output is the user's improvement in understanding and work efficiency. Users use the VR goggles to learn and then apply what they have learned to their actual work.

[0879] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0880] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0881] Another example of generative AI is Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) are mentioned.

[0882] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0883] [Third embodiment]

[0884] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0885] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0886] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0887] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0888] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0889] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0890] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0891] Fig. 6 shows an example of main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0892] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0893] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0894] In the headset type terminal 314, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0895] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0896] "Example 1"

[0897] As one embodiment of the present invention, there is a system that provides a generation AI that receives as input the on-site situation and the target employee's sense of issues, and VR goggles that display the educational training content generated by the generation AI. The generation AI generates educational training content that takes into account the on-site situation and the target employee's sense of issues. Specifically, the on-site situation may be, for example, the work situation on the production line or the operation status of the railway, and the target employee's sense of issues may be, for example, the difficulty of the work or the need for safety measures, as input. Then, the educational training content is generated based on this information. The generated educational training content is displayed to the target employee through the VR goggles, and supports the training of the target employee.

[0898] "Example 2"

[0899] As one embodiment of the present invention, there is a system that provides a generation AI that receives as input the on-site situation and the target employee's sense of issues, and VR goggles that display the educational training content generated by the generation AI. The generation AI generates educational training content that takes into account the on-site situation and the target employee's sense of issues. Specifically, the on-site situation may be, for example, the work situation on the production line or the operation status of the railway, and the target employee's sense of issues may be, for example, the difficulty of the work or the need for safety measures, as input. Then, the educational training content is generated based on this information. The generated educational training content is displayed to the target employee through the VR goggles, and supports the training of the target employee.

[0900] The process flow of each embodiment will be described below.

[0901] "Example 1"

[0902] Step 1: The generation AI receives as input the on-site situation and the target employee's perception of the problem. For example, the on-site situation may be the work situation on the production line or the train operation status, while for example, the target employee's perception of the problem may be the difficulty of the work or the need for safety measures.

[0903] Step 2: The AI ​​generates training content based on the received on-site situation and the employee's sense of challenge. For example, if the production line is congested, the AI ​​generates training content that includes work procedures and precautions to deal with the congestion.

[0904] Step 3: The generated educational training content is displayed to the target employees through VR goggles. The target employees experience the educational training content through the VR goggles and receive the training.

[0905] Example 1

[0906] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[0907] With conventional education and training systems, it was difficult to provide content that fully reflected the specific situation at the workplace and the individual issues faced by the target employees. As a result, the effectiveness of the training was limited, and it was difficult to acquire skills and knowledge that were suited to actual work. In addition, there was also the problem that it took a lot of time and effort to generate and distribute training content.

[0908] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0909] In this invention, the server includes a generating AI means for generating educational training content that takes into account the on-site situation and the target employee's sense of issues, a visual display device means for displaying the educational training content generated by the generating AI, an input device means for inputting the on-site situation and the target employee's sense of issues, and a communication means for transmitting the generated educational training content to the visual display device. This makes it possible to quickly and efficiently generate and display educational training content that reflects the specific on-site situation and the individual sense of issues of the target employee.

[0910] "On-site conditions" refer to the specific states and conditions in a particular business environment, such as the working conditions on a manufacturing line or the operation status of a train.

[0911] "The target employees' sense of challenge" refers to the work-related challenges and problems that specific employees feel, such as the difficulty of the work or the need for safety measures.

[0912] "Educational and training content" refers to learning materials and training programs aimed at improving employee skills and acquiring knowledge.

[0913] "Generative AI" refers to artificial intelligence that automatically generates educational and training content based on on-site conditions and the issues perceived by target employees.

[0914] A "visual display device" is a device for displaying the generated educational and training content, specifically referring to VR goggles and the like.

[0915] An "input device" refers to a device that provides an interface for users to input on-site conditions and the issues perceived by target employees.

[0916] "Communication means" refers to the technology or protocols used to transmit the generated educational and training content from the server to the visual display device.

[0917] The present invention is a system for generating educational and training content based on the situation at the workplace and the sense of issues faced by target employees, and displaying the content on a visual display device. A specific embodiment of this system will be described below.

[0918] server

[0919] The server hosts the generative AI model and receives the on-site situation and the target employees' sense of issues as input. The generative AI model generates educational training content based on this input data. Specifically, the server performs the following processes:

[0920] 1. Data Reception

[0921] The server receives the on-site situation and the employee's sense of challenge sent from the terminal, such as data on the work situation on the manufacturing line, the operation status of trains, the difficulty of the work, and the need for safety measures.

[0922] 2. Data Analysis

[0923] The server analyzes the received data and converts it into a format suitable for the generative AI model, for example by converting text data into numerical data and extracting necessary features.

[0924] 3. Content Creation

[0925] The server generates educational and training content using a generative AI model, which creates optimal educational and training content based on the on-site situation and perceived issues.

[0926] 4. Content submission

[0927] The server transmits the generated educational / training content to the terminal, specifically, by converting it into a data format suitable for display on a visual display device.

[0928] Terminal

[0929] The terminal provides an interface for users to input information about the on-site situation and the issues the target employees face. The data entered by the user is sent from the terminal to the server. After the generation AI on the server generates the educational training content, the terminal transfers the content to the visual display device. The specific operation is as follows:

[0930] 1. Data Entry

[0931] The terminal provides an interface for the user to input information about the situation at the site and the issues faced by the target employees. For example, data is entered using text boxes and drop-down menus.

[0932] 2. Data transmission

[0933] The terminal transmits the input data to the server. Specifically, the terminal transmits the data by using an HTTP request.

[0934] 3. Receiving Content

[0935] The terminal receives the educational and training content transmitted from the server and prepares the received data for transfer to the visual display device.

[0936] 4. Content Display

[0937] The terminal transfers the received educational / training content to the visual display device for display, specifically, by transmitting data for displaying the content on the display of the visual display device.

[0938] User

[0939] The user uses the terminal to input the on-site situation and the issues the target employees feel. For example, they can input information such as "The speed of Line A is too fast" or "New employees are not used to the work." The input data is sent to the server, and the generation AI generates educational and training content. The generated content is displayed to the user through a visual display device. The specific operation is as follows:

[0940] 1. Data Entry

[0941] The user uses the terminal to input information about the on-site situation and issues, such as "The speed of Line A is too fast" or "New employees are not used to the work."

[0942] 2. Data verification

[0943] The user reviews the data they entered to ensure it is accurate and, if necessary, corrects it.

[0944] 3. Training

[0945] The user wears the visual display device and receives the generated educational and training content. Specifically, the user views the content displayed on the display of the visual display device and performs operations according to the instructions.

[0946] Examples

[0947] Example 1

[0948] The user inputs the work situation of the production line as "Line A is running too fast" and the employee's sense of challenge as "New employees are not used to the work." The generation AI on the server uses this information to generate educational training content for new employees. The generated content is shown to new employees through a visual display device, helping them with training on work procedures and safety measures.

[0949] Example 2

[0950] The user inputs the train operation status as "frequent delays" and the target employee's sense of the problem as "efficiency in traffic management is necessary." The generation AI uses this information to generate educational and training content on the efficiency of traffic management. The generated content is shown to traffic management personnel via a visual display device, supporting training on efficient traffic management methods.

[0951] Examples of prompt statements

[0952] "Please enter the operation status of the production line. Example: Line A is running fast."

[0953] "Please enter the problem that the target employee feels. Example: New employees are not familiar with the work."

[0954] "Please enter the train operation status. Example: Frequent delays."

[0955] "Please enter the issues the target employees face. Example: Traffic management needs to be made more efficient."

[0956] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0957] Step 1:

[0958] Data entry (terminal)

[0959] The terminal provides an interface for the user to input the situation at the site and the perceived issues of the target employees. For example, the user inputs information such as "The speed of Line A is fast" or "New employees are not familiar with the work." The input data is saved in the terminal via text boxes and drop-down menus. The input data is text information about the situation at the site and perceived issues.

[0960] Step 2:

[0961] Data transmission (terminal)

[0962] The terminal transmits the input data on the field situation and the sense of issues to the server. Specifically, the data is transmitted using an HTTP request. The transmitted data is in a structured data format such as JSON. The input data is the text information transmitted to the server, and the output is the data transmitted to the server.

[0963] Step 3:

[0964] Data reception (server)

[0965] The server receives data on the field situation and issues sent from the terminal. The received data is analyzed on the server. The input data is the text information sent from the terminal, and the output is the data ready for analysis.

[0966] Step 4:

[0967] Data analysis (server)

[0968] The server analyzes the received data and converts it into a format suitable for the generative AI model. For example, it converts text data into numerical data and extracts the necessary features. The input data is the received text information, and the output is numerical data to be input into the generative AI model.

[0969] Step 5:

[0970] Content generation (server)

[0971] The server generates educational training content using a generative AI model. The generative AI model creates optimal educational training content based on the on-site situation and sense of issues. The input data is analyzed numerical data, and the output is the generated educational training content.

[0972] Step 6:

[0973] Content transmission (server)

[0974] The server transmits the generated educational / training content to the terminal. Specifically, it converts the content into a data format suitable for display on a visual display device and transmits it. The input data is the generated educational / training content, and the output is the content data transmitted to the terminal.

[0975] Step 7:

[0976] Content reception (terminal)

[0977] The terminal receives the educational and training content transmitted from the server. The terminal prepares the received data for transfer to the visual display device. The input data is the content data transmitted from the server, and the output is the data ready for transfer to the visual display device.

[0978] Step 8:

[0979] Content display (terminal)

[0980] The terminal transfers the received educational and training content to the visual display device for display. Specifically, the terminal transmits data for displaying the content on the display of the visual display device. The input data is the data ready to be transferred to the visual display device, and the output is the educational and training content displayed on the visual display device.

[0981] Step 9:

[0982] Attending training (user)

[0983] The user wears the visual display device and takes the generated educational and training content. Specifically, the user watches the content displayed on the display of the visual display device and performs operations according to the instructions. The input data is the educational and training content displayed on the visual display device, and the output is the user's progress in taking the training.

[0984] (Application example 1)

[0985] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[0986] In order to effectively provide educational and training content based on the situation at the workplace and the sense of issues of the target employees, a means of generating and displaying appropriate information in real time is required. However, with conventional systems, it was difficult to provide educational and training content that fully reflected the situation at the workplace and the sense of issues of each employee, and the effectiveness of the training was limited. In addition, because the means of display were limited, it was not possible to provide a flexible training environment.

[0987] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0988] In this invention, the server includes a generating AI means for generating educational training content that takes into account the on-site situation and the sense of issues of the target employee, a VR goggle means for displaying the educational training content generated by the generating AI, and a smart glasses means for displaying the educational training content generated by the generating AI. This makes it possible to generate customized educational training content based on the on-site situation and the sense of issues of each employee in real time and provide it through a flexible display means.

[0989] "Site conditions" refers to the current work progress and environmental conditions at the work site.

[0990] "Target Employees" refers to employees who are intended to receive the educational and training content.

[0991] "Sense of challenge" refers to the concerns felt by the employees, such as the difficulty of the work and the need for safety measures.

[0992] "Educational and training content" refers to educational materials and training programs aimed at improving the skills and knowledge of target employees.

[0993] "Generative AI" refers to artificial intelligence that takes on-site conditions and the target employees' sense of challenges as input and generates educational training content based on that.

[0994] "VR goggles" refers to a head-mounted display for displaying a virtual reality environment.

[0995] "Smart glasses" refers to a glasses-type device that has augmented reality and information display functions.

[0996] The system for carrying out the present invention includes a generating AI means for generating educational training content that takes into account the on-site situation and the sense of issues of the target employees, and a VR goggle means and a smart glasses means for displaying the educational training content generated by the generating AI. A specific embodiment of this system will be described below.

[0997] System Programming

[0998] The server receives the on-site situation and the target employee's sense of challenge as input, and runs a generation AI that generates educational training content based on that. The generation AI uses OpenAI's API to send prompts and generate educational training content. The generated content is displayed on the VR goggles and smart glasses.

[0999] Processing Description

[1000] The server first obtains the on-site situation and the target employee's sense of challenge as input data. This input data includes, for example, the working situation on the production line, the difficulty of the work, and the need for safety measures. Next, the server sends a prompt to the generation AI to generate education and training content. This prompt specifically describes the on-site situation and the sense of challenge. The generated education and training content is sent to the VR goggles and smart glasses and displayed to the target employee.

[1001] Hardware and Software Used

[1002] Hardware: Servers, VR goggles, smart glasses

[1003] Software: OpenAI API, Python

[1004] Examples

[1005] For example, if the work situation on a production line is "assembly of parts" and the worker's sense of challenge is "installation of parts is difficult," the generative AI will generate educational and training content including "detailed steps on how to install parts" and "safety measures." The generated content is displayed to the worker through VR goggles or smart glasses.

[1006] Examples of prompt statements

[1007] Site situation: Parts assembly work

[1008] Problem: Difficult to install parts

[1009] Generate educational training content based on this.

[1010] In this way, the server can generate customized educational and training content in real time based on the situation at the workplace and the issues each employee faces, and provide it through flexible display means.

[1011] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1012] Step 1:

[1013] The server receives input data on the on-site situation and the employees' perceptions of the issues. Specifically, it collects information such as the work situation on the production line, the difficulty of the work, and the need for safety measures through sensors and user input. The input data is sent to the server in JSON format.

[1014] Step 2:

[1015] The server generates a prompt sentence based on the acquired on-site situation and problem perception. Specifically, it converts the on-site situation and problem perception into text format and creates a prompt sentence to send to the generation AI. For example, the format might be "On-site situation: Parts assembly work, Problem perception: Parts installation is difficult."

[1016] Step 3:

[1017] The server sends the prompt text to the generation AI and generates the educational training content. Specifically, the server uses the OpenAI API to send the prompt text and receives the generated text data. The generation AI generates appropriate educational training content based on the input prompt text.

[1018] Step 4:

[1019] The server transmits the generated educational and training content to the VR goggles and smart glasses. Specifically, the server converts the generated text data into a display format for the VR goggles and smart glasses and transmits it to the device. The device displays the received content.

[1020] Step 5:

[1021] Users visually check the educational training content through VR goggles or smart glasses. Specifically, they read the content displayed on the device's display and receive the training. Users perform tasks based on the displayed information and acquire the necessary skills and knowledge.

[1022] Example 2

[1023] Next, a second embodiment of the second form will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1024] There is a need to efficiently generate educational and training content that takes into account the situation at the workplace and the issues faced by the target employees, and to provide effective training to the target employees. However, with conventional systems, it was difficult to generate content that appropriately reflected the situation at the workplace and the issues faced, and there were also limited means to effectively display the generated content. This made it difficult to provide training that met the specific situation at the workplace and the needs of each individual employee.

[1025] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1026] In this invention, the server includes an interface means for the user to input the on-site situation and the sense of issues of the target employee, a means for the server to receive the input data and generate a prompt sentence for the generative AI model, and a means for the server to transmit the generated education and training content to the terminal. This makes it possible to efficiently generate education and training content that reflects the on-site situation and the sense of issues, and to provide effective training to the target employee through the VR goggles.

[1027] "On-site conditions" refer to the specific states and conditions in a particular business environment, such as the working conditions on a manufacturing line or the operation status of a train.

[1028] "Target Employees" refers to the specific employees intended to receive the educational and training content.

[1029] "Sense of challenge" refers to specific problems or issues in the work, such as the difficulty of the work or the need for safety measures as perceived by the employees.

[1030] "Educational and training content" refers to content that is created based on on-site conditions and issues, and is intended for the education and training of target employees.

[1031] "Generative AI means" refers to the function of artificial intelligence that receives on-site conditions and sense of issues as input and generates educational training content based on that.

[1032] "VR goggles means" refers to a device that displays the educational and training content generated by the generating AI means and provides training in a virtual reality environment to target employees.

[1033] "Interface means" refers to the input devices and software of the system that allow the user to input the on-site situation and the target employees' sense of issues.

[1034] A "prompt sentence" is an instruction sentence to be input into a generative AI model, and refers to text that specifically describes the on-site situation and the sense of the problem.

[1035] "Terminal" refers to a device that receives educational and training content sent from the server and displays it on the VR goggles.

[1036] This invention is a system that generates educational training content based on the on-site situation and the employee's sense of issues, and displays it to the employee through VR goggles. A specific embodiment of this system will be described below.

[1037] The user inputs the on-site situation and the employee's perception of the issues through the system interface. For example, the speed of the production line, the difficulty of the work, the need for safety measures, etc. The interface can be a web browser or dedicated application software.

[1038] The server receives the on-site situation and sense of the problem input by the user, and generates a prompt sentence to be input into the generative AI model based on this information. An example of a generative AI model is OpenAI's GPT-4. An example of a prompt sentence is, "The speed of the manufacturing line is so fast that new employees are having trouble remembering the work procedures. Please generate educational training content for this."

[1039] The server inputs the generated prompt sentence into the generative AI model, and the generative AI model generates educational training content based on the prompt sentence. The generated educational training content is, for example, VR content that provides detailed explanations of work procedures on a manufacturing line.

[1040] The server transmits the generated educational and training content to the terminal. The terminal is a device (e.g., a PC or a smartphone) that is linked to the VR goggles. The terminal displays the received educational and training content on the VR goggles.

[1041] Users (target employees) wear VR goggles and receive training through the educational and training content displayed. For example, they can learn the work procedures of a manufacturing line while experiencing them in a virtual environment. This provides a realistic training experience that reflects the actual situation and challenges on the ground.

[1042] This system makes it possible to efficiently generate educational and training content that reflects on-site conditions and issues, and to provide effective training to target employees through VR goggles.

[1043] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1044] Step 1:

[1045] The user inputs the situation on-site and the target employee's perception of the issues.

[1046] Through the system's interface, users input information about the on-site situation and the issues the target employees face. For example, they input information such as the speed of the production line, the difficulty of the work, and the need for safety measures. The input data is sent to the server. An example of input data is, "The speed of the production line is so fast that new employees are having trouble learning the work procedures."

[1047] Step 2:

[1048] The server receives the input data and generates a prompt for the generative AI model.

[1049] The server receives data on the on-site situation and problem perception sent by the user. Next, based on this information, it generates a prompt sentence to be input to the generative AI model. Specifically, the server analyzes the input data and generates a prompt sentence such as, "The speed of the manufacturing line is fast, and new employees are having difficulty learning the work procedures. Please generate educational training content for this." The generated prompt sentence is sent to the generative AI model.

[1050] Step 3:

[1051] The server inputs prompt sentences into the generative AI model to generate educational training content.

[1052] The server inputs the generated prompt sentence into a generative AI model (e.g., OpenAI's GPT-4). The generative AI model generates educational training content based on the prompt sentence. Specifically, the generative AI model analyzes the prompt sentence and generates VR content that explains the work procedures of the manufacturing line in detail. The generated educational training content is returned to the server.

[1053] Step 4:

[1054] The server transmits the generated educational / training content to the terminal.

[1055] The server receives the educational training content generated from the generative AI model and transmits it to the terminal. The terminal is a device (e.g., a PC or a smartphone) that is linked to the VR goggles. The transmitted educational training content is stored in the terminal.

[1056] Step 5:

[1057] The terminal displays educational training content through VR goggles.

[1058] The terminal displays the received educational and training content on the VR goggles. Specifically, the terminal transmits the content to the VR goggles, and the VR goggles provide realistic images and sounds to the target employees.

[1059] Step 6:

[1060] Users (target employees) receive training using VR goggles.

[1061] Users (target employees) wear VR goggles and receive training through the educational and training content displayed. For example, they can learn the work procedures of a manufacturing line while experiencing them in a virtual environment. This provides a realistic training experience that reflects the actual situation and challenges on the ground.

[1062] (Application example 2)

[1063] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1064] There is a need to efficiently generate educational and training content based on the on-site work situation and the workers' sense of issues, and provide it to workers at the appropriate time. However, with conventional systems, it was difficult to quickly generate training content that reflected the real-time situation at the site, and it was also difficult to provide content that responded to the individual sense of issues of workers. This resulted in problems such as insufficient improvement of workers' skills and thorough implementation of safety measures.

[1065] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1066] In this invention, the server includes a generating AI means for generating educational training content that takes into account the on-site situation and the target employee's sense of problem, a VR goggle means for displaying the educational training content generated by the generating AI, a data collection means for collecting data from sensors in the factory, an input means for the worker to input the sense of problem, and a transmission means for transmitting the generated educational training content to the VR goggles. This makes it possible to quickly generate and provide educational training content based on the real-time situation of the on-site situation and the individual sense of problem of the worker.

[1067] "Site conditions" refers to the real-time work situation and environmental conditions in a factory or work site.

[1068] "Target employees' sense of issues" refers to individual issues and concerns felt by workers, such as the difficulty of the work and the need for safety measures.

[1069] "Educational and training content" refers to information and teaching materials for education and training aimed at improving worker skills and ensuring thorough safety measures.

[1070] "Generative AI means" refers to the function of artificial intelligence that takes on-site conditions and the target employees' sense of challenges as input and generates educational training content based on that.

[1071] "VR goggle means" refers to virtual reality goggles for displaying the educational and training content generated by the generation AI to workers.

[1072] "Data collection means" refers to the function of collecting information on on-site work conditions in real time using sensors and cameras within the factory.

[1073] "Input means" refers to devices such as smartphones or tablets that workers use to input their own perceptions of issues.

[1074] "Transmission means" refers to the communication function for transmitting the generated educational and training content to the VR goggles.

[1075] A system for implementing this invention includes a generation AI means for generating educational and training content that takes into account the on-site situation and the target employee's sense of challenges, a VR goggles means for displaying the educational and training content generated by the generation AI, a data collection means for collecting data from sensors within the factory, an input means for workers to input their sense of challenges, and a transmission means for transmitting the generated educational and training content to the VR goggles.

[1076] First, the server collects information on the work situation in the factory in real time using sensors and cameras. This makes it possible to grasp the real-time situation at the factory. Next, workers use their smartphones or tablets to input their own sense of the issues. This sense of the issues includes the difficulty of the work and the need for safety measures.

[1077] The collected information on the site conditions and the workers' sense of challenges are input into the generation AI. Based on this data, the generation AI generates optimal educational and training content. This generation AI uses a programming language such as Python. The generated educational and training content is sent to the VR goggles via a transmission means.

[1078] The VR goggles display the generated educational and training content to the workers, allowing them to receive educational and training based on the real-time situation at the site and their own sense of issues. This allows workers to efficiently improve their skills and ensure safety measures.

[1079] As a concrete example, consider a situation where the robot on the manufacturing line is operating normally, but the worker is feeling uneasy about the new operating procedure. In this case, the worker inputs the sense of challenge that the new operating procedure is difficult. Based on this information, the generative AI generates educational training content that makes it easy for the worker to understand the new operating procedure.

[1080] Examples of prompts for a generative AI model include:

[1081] Site situation: The robots on the manufacturing line are working normally.

[1082] Challenges: Workers find the new operating procedures difficult.

[1083] Use this information to generate educational training content that helps workers understand new operating procedures.

[1084] In this way, it becomes possible to quickly generate and provide educational and training content based on the real-time situation on-site and the individual challenges faced by workers.

[1085] The flow of the specific process in the application example 2 will be described with reference to FIG.

[1086] Step 1:

[1087] The server collects information on the status of on-site work in real time using sensors and cameras in the factory. It receives data from the sensors and cameras as input and generates data indicating the status of on-site work as output. This data includes, for example, the operating status of the manufacturing line and the operating status of robots.

[1088] Step 2:

[1089] The user inputs their own sense of the problem using a smartphone or tablet. As input, the system receives information such as the difficulty of the task as perceived by the user and the need for safety measures, and as output, generates data indicating the sense of the problem. This data includes specific sense of the problem, such as "the new operating procedure is difficult."

[1090] Step 3:

[1091] The server inputs the collected on-site situation data and the user's problem perception data into the generation AI. It receives the on-site situation data and problem perception data as input, and generates educational training content as output. This generation AI is implemented using a programming language such as Python, and generates optimal educational training content based on the input data.

[1092] Step 4:

[1093] The server transmits the generated educational / training content to the VR goggles through a transmission means. As an input, the server receives the generated educational / training content, and as an output, generates data to be transmitted to the VR goggles. For this transmission means, a communication technology such as Wi-Fi or Bluetooth is used.

[1094] Step 5:

[1095] The terminal (VR goggles) displays the received educational and training content to the user. As input, it receives the transmitted educational and training content, and as output, it generates images and audio to be displayed to the user. This allows the user to receive educational and training based on the real-time situation in the field and their own sense of issues.

[1096] In this way, it becomes possible to rapidly generate and provide educational and training content based on the real-time situation on-site and the user's individual sense of the issues.

[1097] In addition, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1098] "Example 1"

[1099] In one embodiment of the present invention, the generation AI includes an emotion engine. The emotion engine recognizes the user's emotion and uses the emotion as an input for the generation AI. Specifically, the emotion is estimated from the user's facial expression, tone of voice, choice of words, etc., and the result is provided to the generation AI. The generation AI generates educational training content by considering this emotion information together with the on-site situation and the target employee's sense of challenge. For example, if the user feels confused, the generation AI takes the emotion into account and generates educational training content including more detailed explanations and concrete examples. Also, if the user feels excited, the generation AI takes the emotion into account and generates educational training content including more advanced topics. In this way, the present invention combined with the emotion engine makes it possible to provide personalized educational training content according to the user's emotion.

[1100] "Example 2"

[1101] In one embodiment of the present invention, the generation AI includes an emotion engine. The emotion engine recognizes the user's emotion and uses the emotion as an input for the generation AI. Specifically, the emotion is estimated from the user's facial expression, tone of voice, choice of words, etc., and the result is provided to the generation AI. The generation AI generates educational training content by considering this emotion information together with the on-site situation and the target employee's sense of challenge. For example, if the user feels confused, the generation AI takes the emotion into account and generates educational training content including more detailed explanations and concrete examples. Also, if the user feels excited, the generation AI takes the emotion into account and generates educational training content including more advanced topics. In this way, the present invention combined with the emotion engine makes it possible to provide personalized educational training content according to the user's emotion.

[1102] The process flow of each embodiment will be described below.

[1103] "Example 1"

[1104] Step 1: The emotion engine recognizes the user's emotions. Specifically, it infers emotions from the user's facial expressions, tone of voice, choice of words, etc.

[1105] Step 2: The emotion engine provides the recognized emotion information to the generation AI.

[1106] Step 3: The generation AI considers the emotion information along with the on-site situation and the target employee's sense of challenge to generate educational training content. For example, if the user feels confused, the generation AI considers that emotion to generate educational training content that includes more detailed explanations and concrete examples. Also, if the user feels excited, the generation AI considers that emotion to generate educational training content that includes more advanced topics.

[1107] Step 4: The educational and training content generated by the generation AI is sent to the VR goggles so that it can be viewed by the target employees.

[1108] Example 1

[1109] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1110] With conventional education and training systems, it was difficult to generate content that took into account the on-site situation and the issues faced by the target employees, and it was also not possible to provide personalized training content that reflected the user's emotions. As a result, the effectiveness of the training was limited, and there were problems with the level of understanding and satisfaction of the target employees decreasing.

[1111] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1112] In this invention, the server includes a generating AI means for generating educational training content that takes into account the on-site situation and the target employee's sense of problem, a VR goggle means for displaying the educational training content generated by the generating AI, and an emotion engine means for recognizing the user's emotion and using the emotion as an input for the generating AI. This makes it possible to provide personalized educational training content that reflects the user's emotion in addition to the on-site situation and the target employee's sense of problem.

[1113] "On-site conditions" refer to a specific work environment or the progress of work, including, for example, the work conditions on a manufacturing line or the operation status of a railway.

[1114] "The target employee's sense of challenge" refers to the work-related difficulties and areas for improvement that a particular employee feels, including, for example, the difficulty of the work or the need for safety measures.

[1115] "Generative AI" refers to artificial intelligence that automatically generates educational and training content based on input data.

[1116] "VR goggles" refers to a device that uses virtual reality technology to visually display information and is used to display educational training content to target employees.

[1117] An "emotion engine" refers to technology that recognizes emotions from a user's facial expressions, tone of voice, choice of words, etc., and uses that emotional information as input for generative AI.

[1118] "Educational and training content" refers to educational materials and training programs aimed at improving the skills and knowledge of target employees.

[1119] "User" refers to the person who operates the system and inputs the on-site situation and problem perception.

[1120] This invention is a system that provides a generation AI that receives the on-site situation and the subject employee's sense of issues as input and generates educational training content based on that, and a VR goggle that displays that content. It also includes an emotion engine that recognizes the user's emotions and uses those emotions as input for the generation AI.

[1121] System Configuration

[1122] 1. Generation AI

[1123] The generative AI generates educational training content by integrating the on-site situation, the target employees' sense of issues, and the user's emotional information. The software used is a natural language processing library (e.g., spaCy, NLTK) or a machine learning framework (e.g., TensorFlow, PyTorch).

[1124] 2. VR goggles

[1125] The VR goggles display educational training content generated by generative AI to target employees, providing an interactive training experience.

[1126] 3. Emotion Engine

[1127] The emotion engine recognizes emotions from the user's facial expressions, tone of voice, choice of words, etc., and provides the results to the generative AI, which then makes it possible to generate personalized educational and training content that reflects the user's emotions.

[1128] System Operation

[1129] 1. Data Entry

[1130] The user inputs the situation at the site and the employee's perception of the problem. For example, the user inputs information such as "Work on the production line is delayed," "The work is difficult," and "New safety measures are needed" into an input form on the terminal.

[1131] The device transmits the user's facial expressions, tone of voice, choice of words, etc. to the emotion engine. The device's camera and microphone capture the user's facial expressions and voice and transmit the data to the emotion engine in real time.

[1132] 2. Data Processing

[1133] The emotion engine estimates the user's emotions and provides the results to the generation AI. The emotion engine estimates that the user is confused from their facial expression and sends that information to the generation AI.

[1134] The generative AI integrates the on-site situation, the target employees' sense of issues, and the user's emotional information to generate educational and training content.The generative AI generates educational and training content including detailed explanations and concrete examples based on information such as "production line work is delayed," "the work is difficult," "new safety measures are needed," and "users are confused."

[1135] 3. Content Display

[1136] The server transmits the generated educational training content to the VR goggles. The server transfers data via a network to transmit the educational training content received from the generation AI to the VR goggles.

[1137] The VR goggles display the educational training content to the target employees. The VR goggles display the received educational training content in the target employees' field of vision, providing an interactive training experience.

[1138] Examples and prompts

[1139] Examples

[1140] Situation on the ground: Work on the production line is slow.

[1141] Challenges: The work is difficult and new safety measures are needed.

[1142] User sentiment: Confused.

[1143] In this case, the generative AI generates educational and training content with detailed explanations and concrete examples, and displays it on the VR goggles.

[1144] Examples of prompt statements

[1145] Situation on the ground: Work on the production line is slow.

[1146] Challenges: The work is difficult and new safety measures are needed.

[1147] User sentiment: Confused.

[1148] Based on this, generate educational training content with detailed explanations and concrete examples.

[1149] In this way, the system provides personalized educational and training content that takes into account user input and sentiment.

[1150] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1151] Step 1:

[1152] The user inputs the situation on-site and the target employee's perception of the issues.

[1153] Input: On-site situation (e.g., work on the production line is delayed), target employee's sense of challenge (e.g., the work is difficult, new safety measures are needed)

[1154] Output: Data on the situation on the ground and issues

[1155] Specific operations: The user enters information such as "Work on the production line is delayed," "The work is difficult," and "New safety measures are needed" into an input form on the terminal.

[1156] Step 2:

[1157] The device transmits the user's facial expressions, tone of voice, choice of words, etc. to the emotion engine.

[1158] Input: User's facial expressions, tone of voice, and word choice

[1159] Output: Emotion data sent to the emotion engine

[1160] How it works: The device's camera and microphone capture the user's facial expressions and voice, and send that data to the emotion engine in real time.

[1161] Step 3:

[1162] The emotion engine estimates the user's emotions and provides the results to the generative AI.

[1163] Input: User's facial expressions, tone of voice, and word choice

[1164] Output: Estimated emotion data

[1165] Specific operation: The emotion engine infers that the user is confused from their facial expression and sends that information to the generation AI.

[1166] Step 4:

[1167] The generative AI integrates the on-site situation, the target employees' sense of issues, and the user's emotional information to generate educational training content.

[1168] Input: On-site situation, target employee's sense of issues, user's emotional information

[1169] Output: Educational and training content

[1170] Specific operation: Based on information such as "work on the production line is delayed," "the work is difficult," "new safety measures are needed," and "users are confused," the generative AI generates educational and training content including detailed explanations and concrete examples.

[1171] Step 5:

[1172] The server transmits the generated educational and training content to the VR goggles.

[1173] Input: Educational and training content

[1174] Output: Data sent to the VR goggles

[1175] Specific operation: The server transfers data via the network to send the educational training content received from the generation AI to the VR goggles.

[1176] Step 6:

[1177] VR goggles display educational training content to target employees.

[1178] Input: Educational and training content

[1179] Output: Educational training content displayed to target employees

[1180] Specific operation: The VR goggles display the received educational training content in the target employee's field of vision, providing an interactive training experience.

[1181] (Application example 1)

[1182] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1183] In conventional education and training systems, it was difficult to provide content that fully reflected the specific situation at the workplace and the individual issues felt by the target employees. In addition, there was a lack of means to provide personalized training content that took into account the feelings of the target employees, so the effectiveness of the training was limited. This made it difficult to implement efficient and effective education and training, especially at workplaces such as factories.

[1184] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1185] In this invention, the server includes a generating AI means for generating educational training content that takes into account the on-site situation and the target employee's sense of issues, a VR goggle means for displaying the educational training content generated by the generating AI, a sensor means for collecting on-site situations in real time, and an emotion engine means for recognizing the emotions of the target employee. This makes it possible to provide personalized educational training content that takes into account the specific on-site situation, the individual sense of issues of the target employee, and even emotions.

[1186] "Site conditions" refers to the progress of work and the state of the environment at a factory or work site.

[1187] "Target employees" refers to employees or workers who are intended to receive the educational and training content.

[1188] "Sense of challenge" refers to the individual issues and concerns felt by the employees, such as the difficulty of the work or the need for safety measures.

[1189] "Educational and training content" refers to educational materials and training programs aimed at improving the skills and knowledge of target employees.

[1190] "Generative AI" refers to artificial intelligence that takes on-site conditions and the target employees' sense of challenges as input and generates educational training content based on that.

[1191] "VR goggles" refers to a virtual reality device for displaying the generated educational training content to target employees.

[1192] A "sensor" refers to a device or equipment used to collect information on site conditions in real time.

[1193] An "emotion engine" refers to a system that recognizes emotions from the facial expressions and tone of voice of the target employee, and provides that information to the generating AI.

[1194] A system for implementing this invention includes a generation AI means for generating educational training content that takes into account the on-site situation and the target employee's sense of challenges, a VR goggle means for displaying the educational training content generated by the generation AI, a sensor means for collecting on-site conditions in real time, and an emotion engine means for recognizing the emotions of the target employee.

[1195] The server first uses sensors to collect real-time information about the on-site situation. Sensors include cameras and environmental sensors in the factory, and obtain data on the progress of work and the state of the environment.

[1196] Next, voice input and tablets are used to collect the employees' perceptions of the issues they face. Employees can input their own perceptions of the difficulty of the work and the need for safety measures.

[1197] In addition, the system uses an emotion engine to recognize the emotions of the target employee. The emotion engine uses a camera and microphone to analyze the facial expressions and tone of voice of the target employee and estimates their emotions.

[1198] All of this data is input into the Generative AI, which generates optimal educational and training content for the target employees based on the on-site situation, sense of issues, and emotional information. The generated content includes specific operating procedures and safety measures, and helps the target employees improve their skills and acquire knowledge.

[1199] The generated educational and training content is displayed to the target employees through VR goggles, which are virtual reality devices that allow the target employees to visually experience the generated content.

[1200] As a specific example, when learning how to operate a new machine in a factory, if an employee feels that "this operation is difficult," the generative AI will take into account that sense of challenge and the employee's confused expression and generate educational training content that includes more detailed operating procedures and safety measures.

[1201] An example of a prompt is as follows:

[1202] Field situation: learning how to operate new machinery

[1203] Challenge: The task is difficult

[1204] Emotion: Confused

[1205] Content to be generated: Educational and training content including detailed operating procedures and safety measures

[1206] In this way, the system is able to provide personalized educational and training content that takes into account the specific situation on-site, the individual challenges faced by the target employees, and even their emotions.

[1207] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1208] Step 1:

[1209] The server uses sensors to collect real-time information about the site conditions. Sensors include cameras and environmental sensors in the factory, and obtain data about the progress of work and the state of the environment. The input is real-time data from the sensors, and the output is data that indicates the site conditions. The server collects this data and passes it on to the next processing step.

[1210] Step 2:

[1211] The user uses voice input or a tablet to collect the target employees' perceptions of the issues. The target employees input information such as their perceptions of the difficulty of the work and the need for safety measures. The input is data on the target employees' perceptions of the issues, and the output is data indicating the perceptions of the issues. The server collects this data and passes it on to the next processing step.

[1212] Step 3:

[1213] The server uses an emotion engine to recognize the target employee's emotions. The emotion engine uses a camera and microphone to analyze the target employee's facial expressions and tone of voice to infer emotions. The input is data from the camera and microphone, and the output is data indicating emotions. The server collects this data and passes it to the next processing step.

[1214] Step 4:

[1215] The server inputs the collected information on the on-site situation, sense of issues, and emotions into the generation AI. Based on this data, the generation AI generates the optimal educational training content for the target employees. The input is the on-site situation, sense of issues, and emotional information, and the output is the educational training content. The generation AI analyzes the data and generates the appropriate content.

[1216] Step 5:

[1217] The server transmits the generated educational / training content to the VR goggles. The VR goggles display the generated content to the target employee. The input is the generated educational / training content, and the output is the content displayed on the VR goggles. The target employee wears the VR goggles and visually experiences the training content.

[1218] In this way, the system is able to provide personalized educational and training content that takes into account the specific situation on-site, the individual challenges faced by the target employees, and even their emotions.

[1219] Example 2

[1220] Next, a second embodiment of the second form will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1221] In conventional education and training systems, it was difficult to provide content that fully reflected the on-site situation and the issues faced by the target employees, and it was not possible to provide personalized training content that responded to the user's emotions. As a result, the effectiveness of the training was limited, and there was a problem of a decrease in the understanding and satisfaction of the target employees.

[1222] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1223] In this invention, the server includes a generating AI means for generating educational training content that takes into account the on-site situation and the target employee's sense of problem, a VR goggle means for displaying the educational training content generated by the generating AI, and an emotion engine means for recognizing the user's emotion and using the emotion as an input for the generating AI. This makes it possible to provide personalized educational training content that takes into account the on-site situation, the target employee's sense of problem, and the user's emotion.

[1224] "On-site conditions" refers to the actual conditions in a specific business environment, such as the working conditions on a manufacturing line or the operation status of a railway.

[1225] "The target employees' sense of challenge" refers to the work-related challenges and problems that specific employees feel, such as the difficulty of the work or the need for safety measures.

[1226] "Generative AI" refers to artificial intelligence that generates educational and training content based on on-site conditions and the issues perceived by target employees.

[1227] "VR goggles" refers to a virtual reality device for visually displaying generated educational and training content.

[1228] An "emotion engine" is a system that recognizes emotions from a user's facial expressions, tone of voice, choice of words, etc., and provides that information to the generative AI.

[1229] "Educational and training content" refers to teaching materials and training materials that enable target employees to learn the knowledge and skills necessary for their jobs.

[1230] "User" refers to employees and participants who use the educational and training content.

[1231] "Server" refers to the computer system that manages and operates the entire system, including the generative AI and emotion engine.

[1232] The present invention is a system that generates educational training content based on the on-site situation and the sense of issues of the target employees, and provides personalized training according to the user's emotions. A specific embodiment of this system will be described below.

[1233] System Configuration

[1234] The system consists of the following main components:

[1235] 1. Generative AI means: Artificial intelligence that generates educational and training content based on on-site conditions and the issues faced by target employees.

[1236] 2. VR goggles means: a virtual reality device that visually displays the generated educational training content.

[1237] 3. Emotion engine means: A system that recognizes emotions from the user's facial expressions, tone of voice, choice of words, etc., and provides that information to the generation AI.

[1238] Hardware and Software Configuration

[1239] The server manages and operates the entire system, including the generation AI and the emotion engine. The generation AI receives the on-site situation and the target employees' sense of issues as input, and generates educational training content. The emotion engine recognizes the user's emotions and provides that information to the generation AI. The VR goggles display the generated educational training content to the user.

[1240] Data processing and calculation

[1241] The server receives data on the on-site situation and perceived issues sent by the user and passes it to the generation AI. The generation AI generates educational training content based on this data. The generated content is sent from the server to the VR goggles and displayed to the user. The emotion engine recognizes the user's emotions and feeds that information back to the generation AI. The generation AI regenerates educational training content as necessary based on the fed-back emotion information.

[1242] Examples

[1243] For example, consider a case where an employee needs to learn how to operate a new machine on a manufacturing line. The user opens a dedicated application on their device and inputs "how to operate a new machine on the manufacturing line" as the on-site situation. They also input "it's difficult to operate" as the sense of challenge. The server receives these data and passes them to the generation AI. The generation AI generates educational training content including detailed operating procedures and safety measures. The generated content is sent to the VR goggles and displayed to the user. When the user puts on the VR goggles and watches the content, the emotion engine recognizes the user's confused facial expression and feeds that information back to the generation AI. The generation AI regenerates the content with more detailed explanations and concrete examples and sends it back to the VR goggles.

[1244] Examples of prompt statements

[1245] Below are examples of prompt sentences that the generative AI uses when generating educational and training content.

[1246] If you have confused employees who need to learn how to operate a new machine on your production line, generate educational training content with detailed explanations and concrete examples.

[1247] In this way, a system is realized in which the server, terminals, and users work together to provide personalized educational and training content that is tailored to the on-site situation, sense of issues, and even the user's emotions.

[1248] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1249] Step 1:

[1250] The user inputs the situation at the site and the issues that need addressing.

[1251] The user opens a dedicated application on the terminal and inputs "How to operate a new machine on the manufacturing line" as the on-site situation. Also, inputs "It's difficult to operate" as the problem. These input data are sent to the server.

[1252] Step 2:

[1253] The server receives the input data and passes it to the generation AI.

[1254] The server receives data on the site situation and issues sent by the user. The received data is passed to the generation AI. At this time, the data format is converted into a format that the generation AI can easily process.

[1255] Step 3:

[1256] Generative AI generates educational training content.

[1257] The generation AI generates educational and training content based on the received data on the site situation and issues. For example, it generates content that includes "detailed instructions and safety measures on how to operate a new machine." The generated content is returned to the server.

[1258] Step 4:

[1259] The server sends the generated content to the VR goggles.

[1260] The server receives the educational and training content returned by the generation AI and transmits it to the VR goggles, where it is stored in the internal memory of the VR goggles.

[1261] Step 5:

[1262] The user puts on the VR goggles and watches the educational training content.

[1263] The user puts on the VR goggles and watches the saved educational and training content. While watching, the user can learn how to operate a new machine through 3D images.

[1264] Step 6:

[1265] The emotion engine recognizes the user's emotions and provides feedback to the generative AI.

[1266] The emotion engine recognizes emotions from the user's facial expressions, tone of voice, choice of words, etc. For example, if the user is confused, it will feed that information back to the generating AI.

[1267] Step 7:

[1268] Generative AI will recreate content based on emotion information (if necessary).

[1269] The AI ​​regenerates the educational and training content as necessary based on the feedback of emotional information. For example, if the user is confused, it generates content that includes more detailed explanations and concrete examples. The regenerated content is sent again via the server to the VR goggles and displayed to the user.

[1270] (Application example 2)

[1271] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1272] There is a need to collect information on the work situation at the site and the workers' sense of issues in real time, generate optimal education and training content based on that information, and provide it to workers effectively. However, with conventional systems, it was difficult to provide personalized education and training content that fully considered the situation at the site and the feelings of the workers. This resulted in problems such as a decrease in the workers' level of understanding and work efficiency.

[1273] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1274] In this invention, the server includes a generating AI means for generating educational training content that takes into account the on-site situation and the subject employee's sense of problem, a VR goggle means for displaying the educational training content generated by the generating AI, a data collection means for collecting the on-site work situation and the subject employee's sense of problem in real time, a content generation means for the generating AI to generate optimal educational training content based on the collected data, and a display means for displaying the generated content to the worker through the VR goggles. This makes it possible to provide personalized educational training content that takes into account the on-site situation and the worker's feelings.

[1275] "Site status" refers to information such as the current progress of work at the work site and the operating status of machines.

[1276] "The target employees' sense of challenge" refers to the workers' subjective assessment of the difficulty of the work, the need for safety measures, etc.

[1277] "Generative AI means" refers to the function of artificial intelligence that takes on-site conditions and the target employees' sense of challenges as input and generates educational training content based on that.

[1278] "VR goggle means" refers to virtual reality goggles for displaying educational and training content generated by the generation AI.

[1279] "Data collection means" refers to devices such as sensors and cameras used to collect information on on-site work conditions and workers' sense of challenges in real time.

[1280] "Content generation means" refers to the function whereby the generative AI generates optimal educational and training content based on collected data.

[1281] "Display means" refers to the function for displaying the generated educational and training content to workers through VR goggles.

[1282] In order to implement this invention, the following system configuration is used: The server includes a generating AI means for generating educational training content that takes into account the on-site situation and the sense of issues of the target employees, a VR goggle means for displaying the educational training content generated by the generating AI, a data collection means for collecting on-site work conditions and the sense of issues of the workers in real time, a content generation means for the generating AI to generate optimal educational training content based on the collected data, and a display means for displaying the generated content to the workers through the VR goggles.

[1283] Hardware and Software Used

[1284] Hardware:

[1285] Camera: Used to collect on-site information.

[1286] VR goggles: Used to display the generated educational and training content.

[1287] Sensors: Used to collect real-time information on work conditions and worker challenges.

[1288] software:

[1289] OpenCV: An image processing library used to collect scene conditions.

[1290] EmotionRecognizer: An emotion engine used to recognize the emotions of workers.

[1291] ContentGenerator: Generative AI. Used to generate educational and training content.

[1292] VRDisplay: Used to display content on VR goggles.

[1293] Data processing and calculation

[1294] The server first collects information from cameras and sensors about the current situation on the site. For example, the camera captures the progress of the production line and the operating status of the machines, and processes the images using OpenCV. Next, the server uses EmotionRecognizer to recognize the emotions of the workers from their facial expressions and tone of voice, and collects their sense of the issues.

[1295] The collected data is input into the ContentGenerator, and the generative AI generates optimal educational training content. For example, if a worker is confused, the generative AI takes that emotion into account and generates educational training content with more detailed explanations and concrete examples. The generated content is displayed on VR goggles through the VRDisplay, providing feedback and educational training to workers in real time.

[1296] Examples

[1297] For example, when a worker is having trouble assembling a part on a production line, the camera and sensor will collect the situation, and the EmotionRecognizer will recognize the worker's confusion. Based on this, the ContentGenerator will generate a 3D model and simulation that explains the part installation procedure in detail, and display it to the worker through the VR goggles.

[1298] Examples of prompt statements

[1299] Scene: Parts assembly on a production line. A worker is having trouble fitting the part.

[1300] Generated Content: Generate 3D models and simulations that detail part installation steps.

[1301] In this way, it becomes possible to provide personalized educational and training content that takes into account the situation on-site and the emotions of workers.

[1302] The flow of the specific process in the application example 2 will be described with reference to FIG.

[1303] Step 1:

[1304] The server collects on-site conditions using cameras and sensors. Specifically, the camera captures the progress of the production line and the operating status of the machines, and obtains data from the sensors. The input is camera images and sensor data, and the output is data on the on-site conditions. OpenCV is used to process the images and analyze the on-site conditions.

[1305] Step 2:

[1306] The server uses EmotionRecognizer to recognize the emotions of the workers. Specifically, it analyzes the facial expressions and tone of voice of the workers to estimate emotions such as confusion and excitement. The input is the facial expression and voice data of the workers, and the output is emotion data. Emotions are analyzed using EmotionRecognizer, and the worker's sense of the challenges is collected.

[1307] Step 3:

[1308] The server uses ContentGenerator to generate optimal education and training content based on the collected site situation data and emotion data. Specifically, it generates education and training content including detailed explanations and concrete examples, taking into account the site situation and the emotions of the workers. The input is site situation data and emotion data, and the output is education and training content. The data is analyzed using ContentGenerator to generate optimal content.

[1309] Step 4:

[1310] The server displays the generated educational and training content on the VR goggles via the VRDisplay. Specifically, the server transmits the generated 3D models and simulations to the VR goggles and provides real-time feedback and educational training to workers. The input is the educational and training content, and the output is the content displayed on the VR goggles. The content is displayed using the VRDisplay and provided to workers.

[1311] Step 5:

[1312] Users check the educational training content displayed through the VR goggles and learn work procedures and safety measures. Specifically, they put on the VR goggles and understand the work procedures through the displayed 3D models and simulations. The input is the content displayed on the VR goggles, and the output is the user's improvement in understanding and work efficiency. Users use the VR goggles to learn and then apply what they have learned to their actual work.

[1313] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1314] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1315] Another example of generative AI is Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) are mentioned.

[1316] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1317] [Fourth embodiment]

[1318] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1319] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1320] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[1321] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. In addition, the microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1322] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[1323] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[1324] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[1325] The control target 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, legs, etc. The posture and behavior of the robot 414 are controlled by controlling the motors of the arms, hands, legs, etc. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1326] Fig. 8 shows an example of main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1327] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1328] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1329] In the robot 414, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1330] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1331] "Example 1"

[1332] As one embodiment of the present invention, there is a system that provides a generation AI that receives as input the on-site situation and the target employee's sense of issues, and VR goggles that display the educational training content generated by the generation AI. The generation AI generates educational training content that takes into account the on-site situation and the target employee's sense of issues. Specifically, the on-site situation may be, for example, the work situation on the production line or the operation status of the railway, and the target employee's sense of issues may be, for example, the difficulty of the work or the need for safety measures, as input. Then, the educational training content is generated based on this information. The generated educational training content is displayed to the target employee through the VR goggles, and supports the training of the target employee.

[1333] "Example 2"

[1334] As one embodiment of the present invention, there is a system that provides a generation AI that receives as input the on-site situation and the target employee's sense of issues, and VR goggles that display the educational training content generated by the generation AI. The generation AI generates educational training content that takes into account the on-site situation and the target employee's sense of issues. Specifically, the on-site situation may be, for example, the work situation on the production line or the operation status of the railway, and the target employee's sense of issues may be, for example, the difficulty of the work or the need for safety measures, as input. Then, the educational training content is generated based on this information. The generated educational training content is displayed to the target employee through the VR goggles, and supports the training of the target employee.

[1335] The process flow of each embodiment will be described below.

[1336] "Example 1"

[1337] Step 1: The generation AI receives as input the on-site situation and the target employee's perception of the problem. For example, the on-site situation may be the work situation on the production line or the train operation status, while for example, the target employee's perception of the problem may be the difficulty of the work or the need for safety measures.

[1338] Step 2: The AI ​​generates training content based on the received on-site situation and the employee's sense of challenge. For example, if the production line is congested, the AI ​​generates training content that includes work procedures and precautions to deal with the congestion.

[1339] Step 3: The generated educational training content is displayed to the target employees through VR goggles. The target employees experience the educational training content through the VR goggles and receive the training.

[1340] Example 1

[1341] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1342] With conventional education and training systems, it was difficult to provide content that fully reflected the specific situation at the workplace and the individual issues faced by the target employees. As a result, the effectiveness of the training was limited, and it was difficult to acquire skills and knowledge that were suited to actual work. In addition, there was also the problem that it took a lot of time and effort to generate and distribute training content.

[1343] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1344] In this invention, the server includes a generating AI means for generating educational training content that takes into account the on-site situation and the target employee's sense of issues, a visual display device means for displaying the educational training content generated by the generating AI, an input device means for inputting the on-site situation and the target employee's sense of issues, and a communication means for transmitting the generated educational training content to the visual display device. This makes it possible to quickly and efficiently generate and display educational training content that reflects the specific on-site situation and the individual sense of issues of the target employee.

[1345] "On-site conditions" refer to the specific states and conditions in a particular business environment, such as the working conditions on a manufacturing line or the operation status of a train.

[1346] "The target employees' sense of challenge" refers to the work-related challenges and problems that specific employees feel, such as the difficulty of the work or the need for safety measures.

[1347] "Educational and training content" refers to learning materials and training programs aimed at improving employee skills and acquiring knowledge.

[1348] "Generative AI" refers to artificial intelligence that automatically generates educational and training content based on on-site conditions and the issues perceived by target employees.

[1349] A "visual display device" is a device for displaying the generated educational and training content, specifically referring to VR goggles and the like.

[1350] An "input device" refers to a device that provides an interface for users to input on-site conditions and the issues perceived by target employees.

[1351] "Communication means" refers to the technology or protocols used to transmit the generated educational and training content from the server to the visual display device.

[1352] The present invention is a system for generating educational and training content based on the situation at the workplace and the sense of issues faced by target employees, and displaying the content on a visual display device. A specific embodiment of this system will be described below.

[1353] server

[1354] The server hosts the generative AI model and receives the on-site situation and the target employees' sense of issues as input. The generative AI model generates educational training content based on this input data. Specifically, the server performs the following processes:

[1355] 1. Data Reception

[1356] The server receives the on-site situation and the employee's sense of challenge sent from the terminal, such as data on the work situation on the manufacturing line, the operation status of trains, the difficulty of the work, and the need for safety measures.

[1357] 2. Data Analysis

[1358] The server analyzes the received data and converts it into a format suitable for the generative AI model, for example by converting text data into numerical data and extracting necessary features.

[1359] 3. Content Creation

[1360] The server generates educational and training content using a generative AI model, which creates optimal educational and training content based on the on-site situation and perceived issues.

[1361] 4. Content Submission

[1362] The server transmits the generated educational / training content to the terminal, specifically, by converting it into a data format suitable for display on a visual display device.

[1363] Terminal

[1364] The terminal provides an interface for users to input information about the on-site situation and the issues the target employees face. The data entered by the user is sent from the terminal to the server. After the generation AI on the server generates the educational training content, the terminal transfers the content to the visual display device. The specific operation is as follows:

[1365] 1. Data Entry

[1366] The terminal provides an interface for the user to input information about the situation at the site and the issues faced by the target employees. For example, data is entered using text boxes and drop-down menus.

[1367] 2. Data transmission

[1368] The terminal transmits the input data to the server. Specifically, the terminal transmits the data by using an HTTP request.

[1369] 3. Receiving Content

[1370] The terminal receives the educational and training content transmitted from the server and prepares the received data for transfer to the visual display device.

[1371] 4. Content Display

[1372] The terminal transfers the received educational / training content to the visual display device for display, specifically, by transmitting data for displaying the content on the display of the visual display device.

[1373] User

[1374] The user uses the terminal to input the on-site situation and the issues the target employees feel. For example, they can input information such as "The speed of Line A is too fast" or "New employees are not used to the work." The input data is sent to the server, and the generation AI generates educational and training content. The generated content is displayed to the user through a visual display device. The specific operation is as follows:

[1375] 1. Data Entry

[1376] The user uses the terminal to input information about the on-site situation and issues, such as "The speed of Line A is too fast" or "New employees are not used to the work."

[1377] 2. Data verification

[1378] The user reviews the data they entered to ensure it is accurate and, if necessary, corrects it.

[1379] 3. Training

[1380] The user wears the visual display device and receives the generated educational and training content. Specifically, the user views the content displayed on the display of the visual display device and performs operations according to the instructions.

[1381] Examples

[1382] Example 1

[1383] The user inputs the work situation of the production line as "Line A is running too fast" and the employee's sense of challenge as "New employees are not used to the work." The generation AI on the server uses this information to generate educational training content for new employees. The generated content is shown to new employees through a visual display device, helping them with training on work procedures and safety measures.

[1384] Example 2

[1385] The user inputs the train operation status as "frequent delays" and the target employee's sense of the problem as "efficiency in traffic management is necessary." The generation AI uses this information to generate educational and training content on the efficiency of traffic management. The generated content is shown to traffic management personnel via a visual display device, supporting training on efficient traffic management methods.

[1386] Examples of prompt statements

[1387] "Please enter the operation status of the production line. Example: Line A is running fast."

[1388] "Please enter the problem that the target employee feels. Example: New employees are not familiar with the work."

[1389] "Please enter the train operation status. Example: Frequent delays."

[1390] "Please enter the issues the target employees face. Example: Traffic management needs to be made more efficient."

[1391] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1392] Step 1:

[1393] Data entry (terminal)

[1394] The terminal provides an interface for the user to input the situation at the site and the perceived issues of the target employees. For example, the user inputs information such as "The speed of Line A is fast" or "New employees are not familiar with the work." The input data is saved in the terminal via text boxes and drop-down menus. The input data is text information about the situation at the site and perceived issues.

[1395] Step 2:

[1396] Data transmission (terminal)

[1397] The terminal transmits the input data on the field situation and the sense of issues to the server. Specifically, the data is transmitted using an HTTP request. The transmitted data is in a structured data format such as JSON. The input data is the text information transmitted to the server, and the output is the data transmitted to the server.

[1398] Step 3:

[1399] Data reception (server)

[1400] The server receives data on the field situation and issues sent from the terminal. The received data is analyzed on the server. The input data is the text information sent from the terminal, and the output is the data ready for analysis.

[1401] Step 4:

[1402] Data analysis (server)

[1403] The server analyzes the received data and converts it into a format suitable for the generative AI model. For example, it converts text data into numerical data and extracts the necessary features. The input data is the received text information, and the output is numerical data to be input into the generative AI model.

[1404] Step 5:

[1405] Content generation (server)

[1406] The server generates educational training content using a generative AI model. The generative AI model creates optimal educational training content based on the on-site situation and sense of issues. The input data is analyzed numerical data, and the output is the generated educational training content.

[1407] Step 6:

[1408] Content transmission (server)

[1409] The server transmits the generated educational / training content to the terminal. Specifically, it converts the content into a data format suitable for display on a visual display device and transmits it. The input data is the generated educational / training content, and the output is the content data transmitted to the terminal.

[1410] Step 7:

[1411] Content reception (terminal)

[1412] The terminal receives the educational and training content transmitted from the server. The terminal prepares the received data for transfer to the visual display device. The input data is the content data transmitted from the server, and the output is the data ready for transfer to the visual display device.

[1413] Step 8:

[1414] Content display (terminal)

[1415] The terminal transfers the received educational and training content to the visual display device for display. Specifically, the terminal transmits data for displaying the content on the display of the visual display device. The input data is the data ready to be transferred to the visual display device, and the output is the educational and training content displayed on the visual display device.

[1416] Step 9:

[1417] Training course (user)

[1418] The user wears the visual display device and takes the generated educational and training content. Specifically, the user watches the content displayed on the display of the visual display device and performs operations according to the instructions. The input data is the educational and training content displayed on the visual display device, and the output is the user's progress in taking the training.

[1419] (Application example 1)

[1420] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1421] In order to effectively provide educational and training content based on the situation at the workplace and the sense of issues of the target employees, a means of generating and displaying appropriate information in real time is required. However, with conventional systems, it was difficult to provide educational and training content that fully reflected the situation at the workplace and the sense of issues of each employee, and the effectiveness of the training was limited. In addition, because the means of display were limited, it was not possible to provide a flexible training environment.

[1422] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1423] In this invention, the server includes a generating AI means for generating educational training content that takes into account the on-site situation and the sense of issues of the target employee, a VR goggle means for displaying the educational training content generated by the generating AI, and a smart glasses means for displaying the educational training content generated by the generating AI. This makes it possible to generate customized educational training content based on the on-site situation and the sense of issues of each employee in real time and provide it through a flexible display means.

[1424] "Site conditions" refers to the current work progress and environmental conditions at the work site.

[1425] "Target Employees" refers to employees who are intended to receive the educational and training content.

[1426] "Sense of challenge" refers to the concerns felt by the employees, such as the difficulty of the work and the need for safety measures.

[1427] "Educational and training content" refers to educational materials and training programs aimed at improving the skills and knowledge of target employees.

[1428] "Generative AI" refers to artificial intelligence that takes on-site conditions and the target employees' sense of challenges as input and generates educational training content based on that.

[1429] "VR goggles" refers to a head-mounted display for displaying a virtual reality environment.

[1430] "Smart glasses" refers to a glasses-type device that has augmented reality and information display functions.

[1431] The system for carrying out the present invention includes a generating AI means for generating educational training content that takes into account the on-site situation and the sense of issues of the target employees, and a VR goggle means and a smart glasses means for displaying the educational training content generated by the generating AI. A specific embodiment of this system will be described below.

[1432] System Programming

[1433] The server receives the on-site situation and the target employee's sense of challenge as input, and runs a generation AI that generates educational training content based on that. The generation AI uses OpenAI's API to send prompts and generate educational training content. The generated content is displayed on the VR goggles and smart glasses.

[1434] Processing Description

[1435] The server first obtains the on-site situation and the target employee's sense of challenge as input data. This input data includes, for example, the working situation on the production line, the difficulty of the work, and the need for safety measures. Next, the server sends a prompt to the generation AI to generate education and training content. This prompt specifically describes the on-site situation and the sense of challenge. The generated education and training content is sent to the VR goggles and smart glasses and displayed to the target employee.

[1436] Hardware and Software Used

[1437] Hardware: Servers, VR goggles, smart glasses

[1438] Software: OpenAI API, Python

[1439] Examples

[1440] For example, if the work situation on a production line is "assembly of parts" and the worker's sense of challenge is "installation of parts is difficult," the generative AI will generate educational and training content including "detailed steps on how to install parts" and "safety measures." The generated content is displayed to the worker through VR goggles or smart glasses.

[1441] Examples of prompt statements

[1442] Site situation: Parts assembly work

[1443] Problem: Difficult to install parts

[1444] Generate educational training content based on this.

[1445] In this way, the server can generate customized educational and training content in real time based on the situation at the workplace and the issues each employee faces, and provide it through flexible display means.

[1446] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1447] Step 1:

[1448] The server receives input data on the on-site situation and the employees' perceptions of the issues. Specifically, it collects information such as the work situation on the production line, the difficulty of the work, and the need for safety measures through sensors and user input. The input data is sent to the server in JSON format.

[1449] Step 2:

[1450] The server generates a prompt sentence based on the acquired on-site situation and problem perception. Specifically, it converts the on-site situation and problem perception into text format and creates a prompt sentence to send to the generation AI. For example, the format might be "On-site situation: Parts assembly work, Problem perception: Parts installation is difficult."

[1451] Step 3:

[1452] The server sends the prompt text to the generation AI and generates the educational training content. Specifically, the server uses the OpenAI API to send the prompt text and receives the generated text data. The generation AI generates appropriate educational training content based on the input prompt text.

[1453] Step 4:

[1454] The server transmits the generated educational and training content to the VR goggles and smart glasses. Specifically, the server converts the generated text data into a display format for the VR goggles and smart glasses and transmits it to the device. The device displays the received content.

[1455] Step 5:

[1456] Users visually check the educational training content through VR goggles or smart glasses. Specifically, they read the content displayed on the device's display and receive the training. Users perform tasks based on the displayed information and acquire the necessary skills and knowledge.

[1457] Example 2

[1458] Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1459] There is a need to efficiently generate educational and training content that takes into account the situation at the workplace and the issues faced by the target employees, and to provide effective training to the target employees. However, with conventional systems, it was difficult to generate content that appropriately reflected the situation at the workplace and the issues faced, and there were also limited means to effectively display the generated content. This made it difficult to provide training that met the specific situation at the workplace and the needs of each individual employee.

[1460] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1461] In this invention, the server includes an interface means for the user to input the on-site situation and the sense of issues of the target employee, a means for the server to receive the input data and generate a prompt sentence for the generative AI model, and a means for the server to transmit the generated education and training content to the terminal. This makes it possible to efficiently generate education and training content that reflects the on-site situation and the sense of issues, and to provide effective training to the target employee through the VR goggles.

[1462] "On-site conditions" refer to the specific states and conditions in a particular business environment, such as the working conditions on a manufacturing line or the operation status of a train.

[1463] "Target Employees" refers to the specific employees intended to receive the educational and training content.

[1464] "Sense of challenge" refers to specific problems or issues in the work, such as the difficulty of the work or the need for safety measures as perceived by the employees.

[1465] "Educational and training content" refers to content that is created based on on-site conditions and issues, and is intended for the education and training of target employees.

[1466] "Generative AI means" refers to the function of artificial intelligence that receives on-site conditions and sense of issues as input and generates educational training content based on that.

[1467] "VR goggles means" refers to a device that displays the educational and training content generated by the generating AI means and provides training in a virtual reality environment to target employees.

[1468] "Interface means" refers to the input devices and software of the system that allow the user to input the on-site situation and the target employees' sense of issues.

[1469] A "prompt sentence" is an instruction sentence to be input into a generative AI model, and refers to text that specifically describes the on-site situation and the sense of the problem.

[1470] "Terminal" refers to a device that receives educational and training content sent from the server and displays it on the VR goggles.

[1471] This invention is a system that generates educational training content based on the on-site situation and the employee's sense of issues, and displays it to the employee through VR goggles. A specific embodiment of this system will be described below.

[1472] The user inputs the on-site situation and the employee's perception of the issues through the system interface. For example, the speed of the production line, the difficulty of the work, the need for safety measures, etc. The interface can be a web browser or dedicated application software.

[1473] The server receives the on-site situation and sense of the problem input by the user, and generates a prompt sentence to be input into the generative AI model based on this information. An example of a generative AI model is OpenAI's GPT-4. An example of a prompt sentence is, "The speed of the manufacturing line is so fast that new employees are having trouble remembering the work procedures. Please generate educational training content for this."

[1474] The server inputs the generated prompt sentence into the generative AI model, and the generative AI model generates educational training content based on the prompt sentence. The generated educational training content is, for example, VR content that provides detailed explanations of work procedures on a manufacturing line.

[1475] The server transmits the generated educational and training content to the terminal. The terminal is a device (e.g., a PC or a smartphone) that is linked to the VR goggles. The terminal displays the received educational and training content on the VR goggles.

[1476] Users (target employees) wear VR goggles and receive training through the educational and training content displayed. For example, they can learn the work procedures of a manufacturing line while experiencing them in a virtual environment. This provides a realistic training experience that reflects the actual situation and challenges on the ground.

[1477] This system makes it possible to efficiently generate educational and training content that reflects on-site conditions and issues, and to provide effective training to target employees through VR goggles.

[1478] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1479] Step 1:

[1480] The user inputs the situation on-site and the target employee's perception of the issues.

[1481] Through the system's interface, users input information about the on-site situation and the issues the target employees face. For example, they input information such as the speed of the production line, the difficulty of the work, and the need for safety measures. The input data is sent to the server. An example of input data is, "The speed of the production line is so fast that new employees are having trouble learning the work procedures."

[1482] Step 2:

[1483] The server receives the input data and generates a prompt for the generative AI model.

[1484] The server receives data on the on-site situation and problem perception sent by the user. Next, based on this information, it generates a prompt sentence to be input to the generative AI model. Specifically, the server analyzes the input data and generates a prompt sentence such as, "The speed of the manufacturing line is fast, and new employees are having difficulty learning the work procedures. Please generate educational training content for this." The generated prompt sentence is sent to the generative AI model.

[1485] Step 3:

[1486] The server inputs prompt sentences into the generative AI model to generate educational training content.

[1487] The server inputs the generated prompt sentence into a generative AI model (e.g., OpenAI's GPT-4). The generative AI model generates educational training content based on the prompt sentence. Specifically, the generative AI model analyzes the prompt sentence and generates VR content that explains the work procedures of the manufacturing line in detail. The generated educational training content is returned to the server.

[1488] Step 4:

[1489] The server transmits the generated educational / training content to the terminal.

[1490] The server receives the educational training content generated from the generative AI model and transmits it to the terminal. The terminal is a device (e.g., a PC or a smartphone) that is linked to the VR goggles. The transmitted educational training content is stored in the terminal.

[1491] Step 5:

[1492] The terminal displays educational training content through VR goggles.

[1493] The terminal displays the received educational and training content on the VR goggles. Specifically, the terminal transmits the content to the VR goggles, and the VR goggles provide realistic images and sounds to the target employees.

[1494] Step 6:

[1495] Users (target employees) receive training using VR goggles.

[1496] Users (target employees) wear VR goggles and receive training through the educational and training content displayed. For example, they can learn the work procedures of a manufacturing line while experiencing them in a virtual environment. This provides a realistic training experience that reflects the actual situation and challenges on the ground.

[1497] (Application example 2)

[1498] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1499] There is a need to efficiently generate educational and training content based on the on-site work situation and the workers' sense of issues, and provide it to workers at the appropriate time. However, with conventional systems, it was difficult to quickly generate training content that reflected the real-time situation at the site, and it was also difficult to provide content that responded to the individual sense of issues of workers. This resulted in problems such as insufficient improvement of workers' skills and thorough implementation of safety measures.

[1500] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1501] In this invention, the server includes a generating AI means for generating educational training content that takes into account the on-site situation and the target employee's sense of problem, a VR goggle means for displaying the educational training content generated by the generating AI, a data collection means for collecting data from sensors in the factory, an input means for the worker to input the sense of problem, and a transmission means for transmitting the generated educational training content to the VR goggles. This makes it possible to quickly generate and provide educational training content based on the real-time situation of the on-site situation and the individual sense of problem of the worker.

[1502] "Site conditions" refers to the real-time work situation and environmental conditions in a factory or work site.

[1503] "Target employees' sense of issues" refers to individual issues and concerns felt by workers, such as the difficulty of the work and the need for safety measures.

[1504] "Educational and training content" refers to information and teaching materials for education and training aimed at improving worker skills and ensuring thorough safety measures.

[1505] "Generative AI means" refers to the function of artificial intelligence that takes on-site conditions and the target employees' sense of challenges as input and generates educational training content based on that.

[1506] "VR goggle means" refers to virtual reality goggles for displaying the educational and training content generated by the generation AI to workers.

[1507] "Data collection means" refers to the function of collecting information on on-site work conditions in real time using sensors and cameras within the ...

Claims

1. A data collection means for collecting data indicating a work situation at a site and a user's sense of problems; A means for generating educational and training content based on the collected data using a generation AI; A display means for displaying the generated educational / training content to the user; A system including:

2. the generating means generates the educational training content based on a prompt sentence corresponding to the emotion information of the user recognized by an emotion engine. The system of claim 1 .

3. The display means displays the educational training content by VR. The system of claim 1 .

Images